Open in app

Sign In

Write

Sign In

SoniaComp
SoniaComp

240 Followers

Home

About

Pinned

Getting started with Pyspark (rdd, spark SQL)— A 10 minute tutorial

with “Google Colaboratory” and “Word Count Example” — You can check complete code at the link below Google Colaboratory — WordCountExample colab.research.google.com code description 1. Spark Environment Setup - Install Java, Spark, and Findspark - Set Environment Variables - Start a SparkSession 2. Loading data into Spark - Create your own RDD - Import data from outside 3. …

Pyspark

3 min read

Getting started with Pyspark (rdd, spark SQL)— A 10 minute tutorial
Getting started with Pyspark (rdd, spark SQL)— A 10 minute tutorial
Pyspark

3 min read


Oct 17, 2022

장고 ORM 과 최적화 기법

ORM, 장고 ORM, N+1 문제, Eager Loading — ORM 데이터베이스 시스템을 직접 다루지 않고도 데이터베이스를 활용할 수 있도록 하는 편리하고 강력한 인터페이스 간단한 것을 쉽게, 어려운 것을 가능하게 해줌 => ORM은 객체와 관계형 데이터베이스의 데이터를 매핑해주는 것 => 데이터베이스와 객체지향 프로그래밍 언어간의 호환되지 않는 데이터를 변환하는 프로그래밍 기법 객체와 관계형 데이터베이스의 데이터를 자동으로 매핑해주는 …

3 min read

3 min read


Jun 26, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(5) Security — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage 3. Data Processing 4. Analysis and Visualization 5. Security Security Cognito Using…

AWS

6 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

6 min read


Jun 26, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(4) Analysis and Visualization (Data Warehouse and QuickInsight) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage 3. Data Processing 4. Analysis and Visualization Data Lake - Lake Formation Analysis…

AWS

10 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

10 min read


Jun 25, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(3) Data Processing (AWS EMR and AWS ETL) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage 3. Data Processing - Glue - EMR - Lambda - AWS Data Pipeline - Sage…

AWS

8 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

8 min read


Jun 22, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(2) Data Storage (EBS, S3…) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage - Data Format - EBS, S3, DynamoDB, RDS - Architecture - Other services…

AWS

6 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

6 min read


Jun 20, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(1) Data Ingestion (AWS Kinesis and other related services) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion - Kinesis Tuning - Amazon Managed Streaming for Apache Kafka Security - Kinesis…

AWS

6 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

6 min read


May 5, 2022

AWS, Google— Data Engineer Interview Questions

Hello 👋 I’m a Junior Data Engineer in Asia. In my opinion, many Asian companies are struggling to bring data architecture into their business. So, as a junior data engineer, I have some trouble figuring our how to develop my career. I tried to find out what kind of problem…

Interview Questions

4 min read

Interview Questions

4 min read


Apr 10, 2022

드루이드 (Druid)

핵심 개념과 물리적인 구조, 드루이드와 다른 시스템(ES, Spark, Key-Value Storage)과의 차이점 — 목차 1. 드루이드 2. 장점 3. 주요 특징 4. 물리적 구조 5. 다른 빅데이터 솔루션과의 차이점 1. 드루이드 (Druid) 공식문서: https://druid.apache.org/ Github: https://github.com/apache/druid Druid is a high performance real-time analytics …

Apache Druid

9 min read

드루이드 (Druid)
드루이드 (Druid)
Apache Druid

9 min read


Feb 6, 2022

“데이터 엔지니어”로서 출발

고민과 계획.. 앞으로 어떻게 하지..? — 클라우드 덕후 삶의 시작 내가 클라우드 덕후가 된 이유 클라우드 조!아!soniacomp.medium.com 2020년 좋은 기회로, “클라우드 아키텍트” 직무로 인턴을 시작하게 됐고, 유연하고 다양한 솔루션에 반해서, “클라우드 아키텍트”를 목표로 삼게 되었다. 하고 싶었던 게 있었기 때문에, 이전 회사에서 프론트엔드를 시작으로 백엔드 서버 개발을 할 때에도, 틈틈이 클라우드 공부를 하면서, 자격증도 따고, 사내 클 …

Lets Go

4 min read

Lets Go

4 min read

SoniaComp

SoniaComp

240 Followers

Data Engineer interested in constructing Data-Driven Architecture with Cloud Service (https://www.linkedin.com/in/sonia-comp/)

Following
  • Kidong Lee

    Kidong Lee

  • Ryan Kim

    Ryan Kim

  • naljin

    naljin

  • Amit Singh Rathore

    Amit Singh Rathore

  • Joao Marques @ Data Beyond Ltd

    Joao Marques @ Data Beyond Ltd

See all (34)

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech