1.
Programming
Time & Space Complexity
HyperLogLog Algorithm
Vectorization using Python
SQL Window Functions
Binary Search Tree Traversal
OOPs Concepts in nutshell
Python : *args vs **kwargs
Python List vs Tuple vs Dictionary
Spark Structured Streaming
Basic code : Spark streaming
Spark Memory and Optimizer
Basic definition of Spark
2.
System Design
OSI Model Layer
Datawarehouse Schema
Slowly Changing Dimension
Database Keys
ACID in Database
CAP Theorem in Nutshell
REST vs GRPC
HTTP Long Polling
Database Sharding
About Kafka
Synchronous and Asynchronous process
Event Driven model
All about Eventual and Strongly Consistency
Demystify Lambda and Kappa Architecture
Vertical vs Horizontal Scaling
What is Caching and its importance
About Zookeeper
Load Balancer
Demystifying HTTP and HTTPS
Authentication vs Authorization
Types of Authentication
About REST and SOAP
Session & Token flow
3.
Product Architecture
Tips on Scaling System
Design Cloud Storage App
Design Photo Sharing App
Design Chat Messenger App
Design a API Rate limiter
Design Video Sharing App
4.
AI ML Engineering
Activation function
Bagging & Boosting
Confusion Matrix
Feature Engineering
AWS : Athena + Sagemaker
Word2Vec Algorithm
Trees in ML World
5.
Cloud Computing
AWS Well Architected Framework
AWS Basic Terminologies
AWS : Search using Kendra
AWS : Redshift + S3
AWS : REST API + Athena (Gateway & Lambda)
AWS : Lambda + API gateway
AWS : Lambda + Athena and S3
AWS : PySpark on EMR Cluster
AWS : S3 + Glue and Athena
Brief on AWS IAM
AWS : ElasticSearch Service
Brief Intro AWS Kinesis
AWS : Setup Cloud Budgets
AWS Monitoring Services
AWS : Refresh CloudFront caches
Comparison AWS technologies
More
Personal Projects
Publications
Linkedin
GitHub
About the Blog
>
Cloud Computing
> AWS : S3 + Glue and Athena
7 simple steps to integrate S3, Glue and Athena
1. Upload any Dataset on S3
2. Setup Glue Role
3. Attach S3 and Glue Role
4. Create RoleName -
AWSGlueServiceRoleDefault
Check the Glue role (highlighted)
5. Glue Crawler Creation - Step by Step
Glue Crawler is created !!
6. Run the Glue Crawler
7. Go to Athena and run query on database / table
aws
cloud
architechture
glue
etl
data
s3
AWS : S3 + Glue and Athena
7 simple steps to integrate S3, Glue and Athena
1. Upload any Dataset on S3
2. Setup Glue Role
Select Glue from the list
3. Attach S3 and Glue Role
Tags are optional
4. Create RoleName -
AWSGlueServiceRoleDefault
Check the Glue role (highlighted)
5. Glue Crawler Creation - Step by Step
Glue Crawler is created !!
6. Run the Glue Crawler
7. Go to Athena and run query on database / table
You have the data 😊
Note :
The steps mentioned above is for POC.
In Production or any organization , CloudFormation template and proper IAM roles would be utilized (Concept of least privilege)