Open in app

Sign in

Write

Sign in

SoniaComp
SoniaComp

241 Followers

Home

About

Pinned

Getting started with Pyspark (rdd, spark SQL)— A 10 minute tutorial

with “Google Colaboratory” and “Word Count Example” — You can check complete code at the link below Google Colaboratory — WordCountExample colab.research.google.com code description 1. Spark Environment Setup - Install Java, Spark, and Findspark - Set Environment Variables - Start a SparkSession 2. Loading data into Spark - Create your own RDD - Import data from outside 3. …

Pyspark

3 min read

Getting started with Pyspark (rdd, spark SQL)— A 10 minute tutorial
Getting started with Pyspark (rdd, spark SQL)— A 10 minute tutorial
Pyspark

3 min read


Jun 11

개춘기… 개발자 사춘기 극복 일지 🥸

개발자 4년차에 접어들며… — 프론트 엔지니어로 시작해, 풀스택 엔지니어를 거쳐 데이터 엔지니어로 먹고 산지 벌써 3년을 꽉 채우게 되었다. 그동안 연봉도 높아지고, 이제 더 이상 귀여움 받으며 배우고 성장하는데 급급한 주니어가 아니게 되었다. 나도 이제 회사에서 밥값을 하기 위해서는 시스템의 개발과 개선을 주도해야하는 연차가 되었고, 내가 …

3 min read

개춘기… 개발자 사춘기 극복 일지 🥸
개춘기… 개발자 사춘기 극복 일지 🥸

3 min read


Oct 17, 2022

장고 ORM 과 최적화 기법

ORM, 장고 ORM, N+1 문제, Eager Loading — ORM 데이터베이스 시스템을 직접 다루지 않고도 데이터베이스를 활용할 수 있도록 하는 편리하고 강력한 인터페이스 간단한 것을 쉽게, 어려운 것을 가능하게 해줌 => ORM은 객체와 관계형 데이터베이스의 데이터를 매핑해주는 것 => 데이터베이스와 객체지향 프로그래밍 언어간의 호환되지 않는 데이터를 변환하는 프로그래밍 기법 객체와 관계형 데이터베이스의 데이터를 자동으로 매핑해주는 …

3 min read

3 min read


Jun 26, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(5) Security — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage 3. Data Processing 4. Analysis and Visualization 5. Security Security Cognito Using…

AWS

6 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

6 min read


Jun 26, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(4) Analysis and Visualization (Data Warehouse and QuickInsight) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage 3. Data Processing 4. Analysis and Visualization Data Lake - Lake Formation Analysis…

AWS

10 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

10 min read


Jun 25, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(3) Data Processing (AWS EMR and AWS ETL) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage 3. Data Processing - Glue - EMR - Lambda - AWS Data Pipeline - Sage…

AWS

8 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

8 min read


Jun 22, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(2) Data Storage (EBS, S3…) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion 2. Data Storage - Data Format - EBS, S3, DynamoDB, RDS - Architecture - Other services…

AWS

6 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

6 min read


Jun 20, 2022

AWS Certified Data Analytics(DAS-C01) — Certification Summary

(1) Data Ingestion (AWS Kinesis and other related services) — Amazon’s data services can be divided into five categories: data ingestion, storage, processing, analysis and visualization, and security. This article is part of a series, each dealing with each of the five topics above. 1. Data Ingestion - Kinesis Tuning - Amazon Managed Streaming for Apache Kafka Security - Kinesis…

AWS

6 min read

AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS Certified Data Analytics(DAS-C01) — Certification Summary
AWS

6 min read


May 5, 2022

AWS, Google— Data Engineer Interview Questions

Hello 👋 I’m a Junior Data Engineer in Asia. In my opinion, many Asian companies are struggling to bring data architecture into their business. So, as a junior data engineer, I have some trouble figuring our how to develop my career. I tried to find out what kind of problem…

Interview Questions

4 min read

Interview Questions

4 min read


Apr 10, 2022

드루이드 (Druid)

핵심 개념과 물리적인 구조, 드루이드와 다른 시스템(ES, Spark, Key-Value Storage)과의 차이점 — 목차 1. 드루이드 2. 장점 3. 주요 특징 4. 물리적 구조 5. 다른 빅데이터 솔루션과의 차이점 1. 드루이드 (Druid) 공식문서: https://druid.apache.org/ Github: https://github.com/apache/druid Druid is a high performance real-time analytics …

Apache Druid

9 min read

드루이드 (Druid)
드루이드 (Druid)
Apache Druid

9 min read

SoniaComp

SoniaComp

241 Followers

Data Engineer interested in Data Infrastructure Powering Fintech Innovation (https://www.linkedin.com/in/sonia-comp/)

Following
  • Amit Singh Rathore

    Amit Singh Rathore

  • Kidong Lee

    Kidong Lee

  • ART NYC

    ART NYC

  • Ryan Kim

    Ryan Kim

  • naljin

    naljin

See all (38)

Help

Status

About

Careers

Blog

Privacy

Terms

Text to speech

Teams