Yogender PalDecomposing a monolithic database for microservice architectureSchema design and its application for bounded context in domain-driven development·9 min read·Oct 29, 2023----
Yogender PalinDevOps.devDeep dive into Databricks delta-lakesTechnologies used: Python, PySpark, and SQL on Databricks notebooks·13 min read·Oct 27, 2023--1--1
Yogender PalAce your SQL interviews by doing these 10 advanced queriesI have used PostgreSQL, but these queries can be used in any SQL engine. I assume you have already installed the SQL engine on your local…·15 min read·Oct 24, 2023----
Yogender PalInstalling Postgres-11 on Python-3.11Installing postgres11 in public.ecr.aws/lambda/python:3.11, but underlying Linux is Amazon Linux 2, so applicable for all instances using…·2 min read·Aug 19, 2023----
Yogender PalAdhere your Pandas data analytics class to the single responsibility and open close principle of…Do you impeccably organize your Panda’s methods in a class like this?·1 min read·Sep 9, 2022----
Yogender PalSpark Stream Processing with Amazon EMR using Apache Kafka streams running in Amazon MSKA gentle introduction to the “data in motion” with AWS using Apache Spark on Amazon EMR and Apache Kafka on Amazon MSK·11 min read·Jun 30, 2022--1--1
Yogender PalSpark Structured Streaming with Kafka all on your laptopStream data, perform data wrangling on streamed data, (stream processing), and batch and incremental queries on the stream·6 min read·Jun 23, 2022----
Yogender PalinDevOps.devCreate AWS EC2 instance using Terraform with custom Ubuntu Amazon Machine Image (AMI) having…Terraform is an infrastructure as code (IaC) tool that allows you to build, change, and version infrastructure safely and efficiently. This…·8 min read·May 28, 2022----
Yogender PalPySpark UDFs, Spark-NLP, and scrapping HTML files on spark clusters — a complete ETL process for…This is a beginner to pro guide to deal with PySpark clusters. Complete jupyter notebook can be found here…·4 min read·Apr 9, 2022----
Yogender PalConfigure GitHub Actions with AWS — for your CI/CD pipelineYou want to automate things by building CI/CD pipeline using GitHub Actions accessing AWS resources. Just an example you want to create a…·4 min read·Feb 11, 2022--1--1