ARGUS is a production-shaped fleet telemetry platform: devices stream into a Kafka-compatible bus, Ray and Flink harden the data path, Iceberg + Dagster form the lakehouse spine, and drift / OPA incidents / OpenTelemetry close the loop for operators — with a read-only AI copilot for investigation. Clone it, docker compose up, and you get the same contracts and images that target EKS via Terraform, Helm, and Argo CD.
Docs: Getting Started · Demo Script (5 min) · Case Study · Known gaps and what I'd build next · ADRs · CHANGELOG v1.0.0
flowchart LR
Sim["simulator / SDKs"] --> Kafka["Redpanda / MSK"]
Kafka --> Ray["Ray ingest"]
Ray --> QA["stream-processor QA"]
QA --> Iceberg["Iceberg + Trino"]
QA --> Drift["drift-monitor"]
Iceberg --> Dagster["Dagster / MLflow"]
Drift --> IE["incident-engine + OPA"]
IE --> GW["api-gateway"]
Iceberg --> GW
GW --> UI["dashboard"]
GW --> Copilot["ai-copilot"]
IE --> OTel["Prometheus / Grafana / Loki / Jaeger"]
simulator/SDKs → telemetry.raw → Ray → normalized → QA ─┬→ validated → Iceberg / drift
├→ quarantine → Iceberg DLQ
└→ qa_metrics → incidents → gateway → UI
Full design: ARCHITECTURE.md · docs/architecture.md
git clone https://github.com/hamidmatiny/Argus.git && cd Argus
cp .env.example .env
echo "NEXTAUTH_SECRET=$(openssl rand -base64 32)" >> .env
echo "NEXTAUTH_URL=http://localhost:3002" >> .env
echo "AUTH_DEMO_OFFLINE=true" >> .env
docker compose up -d --build| Open | URL |
|---|---|
| Dashboard | http://localhost:3002 — login operator / operator |
| Grafana | http://localhost:3001 — admin / argus |
| Gateway health | http://localhost:8099/health |
| Redpanda Console | http://localhost:8087 |
curl -s http://localhost:8099/v1/ping
curl -s -X POST http://localhost:8099/v1/telemetry/query \
-H 'Content-Type: application/json' -H 'X-API-Key: demo-viewer' \
-d '{"sql":"SELECT vehicle_id FROM telemetry LIMIT 5","limit":5}'Tear down: docker compose down -v
Dashboard — Overview (placeholder)
docs/assets/screenshots/dashboard-overview.png — TODO: capture Overview with live throughput after login.
Dashboard — Incidents (placeholder)
docs/assets/screenshots/dashboard-incidents.png — TODO: open incidents + breaker state.
Grafana — SLOs (placeholder)
docs/assets/screenshots/grafana-slos.png — TODO: QA pass ratio / gateway latency panels.
Also see dashboard/docs/screenshots/*.png.txt markers.
| Component | Role | README |
|---|---|---|
shared/ |
Contracts (Avro / Proto / JSON Schema) | README |
ingestion/ |
Simulator + Ray normalize | README |
stream-processor/ |
Streaming QA gate | README |
drift-monitor/ |
KS + Evidently drift | README |
lakehouse/ |
Iceberg writers + Trino | README |
orchestration/ |
Dagster + MLflow | README |
simulation/ |
Scenario synthetic sensors (Sources→Transforms→Sinks) | README |
incident-engine/ |
OPA + circuit breakers | README |
api-gateway/ |
Authn/z edge API | README |
observability/ |
Metrics, logs, traces, alerts | README |
dashboard/ |
Next.js operator UI | README |
ai-copilot/ |
Read-only RAG assistant | README |
sdk/python/ |
argus-sdk |
README |
sdk/typescript/ |
@argus/sdk |
README |
cli/ |
argusctl |
README |
infra/ |
Terraform · Helm · Argo CD | README |
tests/e2e/ |
Smoke · load · chaos | README |
docs/ |
MkDocs site · ADRs · demo | README |
| Category | Technologies |
|---|---|
| Streaming | Apache Kafka API, Redpanda, Amazon MSK, Ray, Apache Flink / PyFlink |
| Storage | Apache Iceberg, MinIO, Amazon S3, AWS Glue, Trino, Parquet |
| Orchestration | Dagster, Feast (optional); AWS Lambda + Step Functions + EventBridge + SQS (serverless ETL demo — not the production Kafka path; see ADR 007) |
| ML | MLflow, Evidently, SciPy KS tests, embeddings (hash / OpenAI); scenario synthetic sensors (classical renderer proxies — see simulation/) |
| Observability | OpenTelemetry, Prometheus, Grafana, Loki, Promtail, Jaeger, Alertmanager |
| Security | Keycloak (OIDC), OPA/Rego, API keys + RBAC, Trivy, Syft SBOM, Semgrep |
| Frontend | Next.js (App Router), TypeScript, Tailwind, NextAuth, Recharts |
| Infra | Docker Compose, Terraform, Helm, Argo CD, Amazon EKS, IRSA |
| AI | Qdrant, tool-calling LLM agents (OpenAI / Anthropic / mock), RAG runbooks |
| Languages | Python 3.12, Go 1.22, TypeScript / Node 22 |
| Quality | pytest, Go test, Vitest, Playwright, k6, GitHub Actions |
CI coverage gate: ≥ 65% on gated packages (see CONTRIBUTING.md).
pip install -r docs/requirements.txt
mkdocs serve # http://127.0.0.1:8000Apache License 2.0 — see LICENSE.