Production-ready PySpark ETL template for Databricks — medallion architecture, DABs, tests, DQX, CI/CD, and agentic development with Claude Code.

Lightweight Python library and CLI that generates Databricks Workflows definition from your dbt project — each dbt model, test, seed, and snapshot runs as a separate task.

End-to-end SaaS subscription data platform using AWS, Databricks (Delta Lake), and Snowflake featuring streaming ingestion, CDC/SCD modeling, medallion architecture, and Iceberg-based analytics.

This Repository has like a goal, build generic solution for use on any projects

Built a real-time banking analytics platform for streaming transaction processing, fraud detection, and KPI monitoring.

Built an end-to-end retail lakehouse using the Medallion architecture with batch and streaming pipelines for scalable business analytics.

Developed a metadata-driven ETL framework that automates configurable data ingestion, SCD processing, and data quality using generic pipelines.

Managing and monitoring jobs in Databricks can be a daunting task, especially as the number of jobs grows. To streamline this process, we can automate the extraction of job metadata and store it in a centralized table for easy access and monitoring.

This azure databricks project implements a modern data engineering pipeline on Azure using the Medallion Architecture (Bronze → Silver → Gold) to process NYC taxi trip data.

AI-agent skill for migrating Matillion ETL pipelines to Databricks

Medallion pipeline over the public NYC TLC taxi dataset on Databricks Free Edition.

Commercial sales analytics platform built on Databricks and dbt. Medallion architecture, Delta Lake, automated Workflows and a live AI/BI dashboard, with governed gold data marts covering the full deal lifecycle from pipeline stage through activity and rep performance.

This project built an automated, end-to-end data pipeline from raw CSVs to real-time Databricks dashboards for Rideeasy Cab Aggregator.