Course Content

​

Module 1: Introduction to Databricks

1. What is Databricks?

2. Why Databricks? Key features & benefits

3. Databricks vs Hadoop vs Snowflake vs AWS EMR

4. Understanding the Lakehouse Architecture

5. Workspaces, users, groups, and notebooks

6. Databricks pricing & cluster types

Module 2: Databricks Workspace & Environment Setup

1. Navigating the Databricks UI

2. Creating and managing notebooks (Python, SQL, Scala)

3. Repos & Git integration

4. Databricks File System (DBFS)

5. Accessing data using Mount Points

6. Working with Databricks Utilities (DBUtils)

Module 3: Databricks Clusters & Compute

1. Types of clusters (Standard, High Concurrency, Single Node)

2. Cluster setup, autoscaling, spot instances

3. Jobs and Job Clusters

4. Pools for cost optimization

5. Cluster policies and security

Module 4: Data Engineering with Databricks

1. ETL vs ELT on Databricks

2. Reading/Writing data from ADLS/S3/GCS

3. Transformations with PySpark

4. Databricks Delta: Delta Lake essentials

5. Delta Tables, ACID Transactions, Time Travel, Vacuum

6. Auto Loader for streaming data ingestion

7. Data pipelines with Workflows

Module 5: Databricks SQL & BI Analytics

1. SQL Warehouses (formerly SQL Endpoints)

2. Writing analytical queries

3. Creating dashboards

4. Materialized views & querying Delta Tables

5. Performance optimization & query caching

6. Connecting Power BI/Tableau to Databricks

Module 6: Delta Lake Deep Dive

1. Delta Architecture

2. Delta schema enforcement & evolution

3. Delta Live Tables (DLT)

4. Medallion Architecture (Bronze, Silver, Gold)

5. Optimize & Z-Order commands

6. Handling slowly changing dimensions (SCD Type 1/2)

Module 7: Machine Learning & AI with Databricks

1. Databricks ML runtime

2. Feature engineering with Feature Store

3. MLflow for experiment tracking

4. AutoML in Databricks

5. Building ML pipelines

6. Model deployment & serving

7. Integrating with external ML libraries (TensorFlow, PyTorch, Scikit-Learn)