Clean Code - Robert C. Martin

Locust - Python Performance Testing Library

Locust Git Repository

Terraform

Kubernetes , Prometheus and Grafana

Multi thread in Python

Async in Python

Versioning

java.lang.NoSuchMethodError: org.apache.spark.sql.AnalysisException.(Ljava/lang/String;Lscala/Option;Lscala/Option

Tenacity with Asyncio - Defensive Programming

Python : Find Sum using Parallel Procesing using Common Aggregator

Pyspark : Compression

PySpark Cache

About Me

TriggerBlock is a cool tech blog that's all about helping you out with tutorials, problem-solving, hacks, and quick fixes for different coding languages and techie stuff like Java, Python, Scala, Kubernetes, and more. They've got plenty of articles on testing too. Check it out!

via GIPHY

Configure Pyspark

 Pyspark

Versioning

ETL : Delta Lake

Delta Lake 

Azure Synapse 

ETL

Build Jar : Add multiple class files in the Jar using Bazel

 Add multiple class files in the Jar using Bazel 

Azure Synapse - Working with Custom jar Scala (Bazel)

 Working with Custom jar in Azure 

There is 2 Jars :

1. Big Fat Jar or uber Jar (Contains main class  )
2. Build Dependency Jar/ Library to be used in Synapse 

Big Fat Jar or uber Jar:

• Contains main class
• Contains all the Dependency
• Contains manifest File
• $ bazel build build_name_deploy.jar
• Can be run as : build_name_deploy.jar
• Eg: https://github.com/j-thepac/BazelTestProject
• Ref - https://docs.bazel.build/versions/main/be/java.html

Build Dependency Jar/ Library

• Does not Contain main class
• Does not contain dependencies
• Note : Download external dependencies from Maven and upload it separately (refer bewlo link)
• $ bazel build build_name.jar

Once u have jars Created based on ure need , use link - https://docs.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-manage-scala-packages to upload to Azure