Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

151 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MLOps Pipeline Platform

Build Status Python License

A full-stack MLOps platform that lets anyone regardless of ML experience train, compare, deploy, and run inference on machine learning models through a guided step-by-step web interface. Built on Django, ZenML, MLflow, and AWS.

Can be run locally without AWS using local file storage for models and artifacts.

Demo

🎥 Watch the full app walkthrough

Features

  • Upload a CSV and train multiple ML models through a step-by-step UI
  • Experiment tracking and model registry via MLflow
  • Deploy models as persistent HTTPS inference endpoints
  • Run batch predictions and store results
  • Session-based Guest accounts. Registered users retain full history
  • CI pipeline via GitHub Actions to AWS ECR

Architecture

The app runs as a multi-container system via Docker Compose on an AWS EC2 instance:

Service Technology Purpose
Web Server Django + Gunicorn Frontend, auth, API views
ML Tracking MLflow Experiment logging, model registry
ML Pipelines ZenML Pipeline orchestration
App Database PostgreSQL User data, models, files
ZenML Database MySQL ZenML metadata
File Storage AWS S3 CSVs, plots, static files
Reverse Proxy nginx HTTPS routing between containers

Tech Stack

  • Backend: Django, Gunicorn
  • ML: Scikit-learn, ZenML, MLflow
  • Cloud: AWS (EC2, S3, ECR, SSM Parameter Store)
  • Infrastructure: Docker, Docker Compose, nginx
  • CI: GitHub Actions
  • Databases: PostgreSQL, MySQL
  • Languages: Python 3.13, Bash

Getting Started

Prerequisites

  • Docker and Docker Compose
  • AWS account with ECR, S3, and SSM configured (OPTIONAL, CAN USE LOCAL FILE STORAGE)

Local Setup

  1. Clone the repository
   git clone https://github.com/reyrey112/MLAPP.git
   cd MLAPP
  1. Add your environment variables
   cp .env.example .env
   # Fill in your values
  1. Run the app
   docker-compose up -d
  1. Visit http://localhost:8000

Project Structure

MLAPP/
├── MLapp/              # Django project settings
├── processing/         # Django app (views, models, templates)
├── MLOps/
│   ├── pipelines/      # ZenML pipeline definitions
│   └── steps/          # ZenML step definitions
├── .github/workflows/  # CI/CD GitHub Actions
├── docker-compose.yaml
├── django_dockerfile
├── nginx.conf.template
└── ec2_startup_script.sh

Key Engineering Decisions

Why ZenML alongside MLflow? MLflow handles experiment tracking and model registry well, but ZenML provides pipeline orchestration, step-level caching, and artifact lineage that MLflow alone doesn't offer. Together they cover the full MLOps lifecycle from training to deployment.

As a pipeline orchestrator, ZenML allows integrations between many different softwares (local and cloud-based) and can create fully customizable software stacks for different pipeline executions that can be changed seamlessly, making it perfect for fitting into anybody's current tech stack.

Roadmap

  • Automate EC2 startup via User Data and crontab
  • A more robust Continuous deployment pipeline for new ECR images
  • TensorFlow model support
  • Expanded visualization library

License

MIT

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages