docs(lab8): add monitoring dashboard and alert runbook

app/Dockerfile

@@ -0,0 +1,58 @@
+# syntax=docker/dockerfile:1
+
+FROM golang:1.24-alpine AS builder
+
+WORKDIR /src
+
+COPY go.mod ./
+RUN go mod download
+
+COPY . .
+
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
+    go build -trimpath -ldflags="-s -w" -o /out/quicknotes .
+
+RUN cat > /tmp/healthcheck.go <<'EOF'
+package main
+
+import (
+	"net/http"
+	"os"
+	"time"
+)
+
+func main() {
+	client := http.Client{Timeout: 2 * time.Second}
+	resp, err := client.Get("http://127.0.0.1:8080/health")
+	if err != nil {
+		os.Exit(1)
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		os.Exit(1)
+	}
+}
+EOF
+
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
+    go build -trimpath -ldflags="-s -w" -o /out/healthcheck /tmp/healthcheck.go
+
+# Create an empty /data directory that will be copied into the runtime image
+RUN mkdir -p /out/data
+
+FROM gcr.io/distroless/static:nonroot
+
+WORKDIR /
+
+COPY --from=builder /out/quicknotes /quicknotes
+COPY --from=builder /out/healthcheck /healthcheck
+
+# Make /data exist in the image and be owned by distroless nonroot UID/GID 65532.
+# When Docker creates a fresh named volume, it initializes it from this directory.
+COPY --from=builder --chown=65532:65532 /out/data /data
+
+USER nonroot:nonroot
+
+EXPOSE 8080
+
+ENTRYPOINT ["/quicknotes"]

compose.yaml

@@ -0,0 +1,60 @@
+services:
+  quicknotes:
+    build:
+      context: ./app
+      dockerfile: Dockerfile
+    image: quicknotes:lab6
+    ports:
+      - "8080:8080"
+    environment:
+      ADDR: ":8080"
+      DATA_PATH: "/data/notes.json"
+      SEED_PATH: "/seed.json"
+    volumes:
+      - quicknotes-data:/data
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "/healthcheck"]
+      interval: 10s
+      timeout: 3s
+      retries: 3
+      start_period: 5s
+    cap_drop:
+      - ALL
+    read_only: true
+    tmpfs:
+      - /tmp
+    security_opt:
+      - no-new-privileges:true
+
+  prometheus:
+    image: prom/prometheus:v3.11.3
+    ports:
+      - "9090:9090"
+    volumes:
+      - ./monitoring/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
+      - ./monitoring/prometheus/alerts.yml:/etc/prometheus/alerts.yml:ro
+    command:
+      - "--config.file=/etc/prometheus/prometheus.yml"
+    depends_on:
+      quicknotes:
+        condition: service_healthy
+    restart: unless-stopped
+
+  grafana:
+    image: grafana/grafana:13.0.3
+    ports:
+      - "3000:3000"
+    environment:
+      GF_SECURITY_ADMIN_USER: "admin"
+      GF_SECURITY_ADMIN_PASSWORD: "lab8-admin-please-change"
+      GF_USERS_DEFAULT_THEME: "dark"
+    volumes:
+      - ./monitoring/grafana/provisioning:/etc/grafana/provisioning:ro
+      - ./monitoring/grafana/dashboards:/var/lib/grafana/dashboards:ro
+    depends_on:
+      - prometheus
+    restart: unless-stopped
+
+volumes:
+  quicknotes-data:

docs/runbook/high-error-rate.md

@@ -0,0 +1,43 @@
+# Runbook: QuickNotes High Error Rate
+
+## What this alert means
+
+More than 5% of QuickNotes HTTP requests have returned 4xx or 5xx responses for at least 5 minutes.
+
+## Triage steps
+
+1. Open the QuickNotes Golden Signals dashboard in Grafana and check whether the error ratio is still above 5%.
+2. Check whether traffic changed sharply at the same time. A real traffic spike plus errors may indicate a user-facing incident.
+3. In Prometheus, inspect the request metric by status code to identify whether the errors are mostly 4xx or 5xx.
+4. Check the QuickNotes container logs:
+
+       docker compose logs quicknotes --tail=100
+
+5. Confirm the service is reachable:
+
+       curl -s http://localhost:8080/health
+
+6. Check whether recent deploy/config changes happened:
+
+       git log --oneline -5
+       docker compose ps
+
+## Mitigations
+
+1. Restart the QuickNotes service if it is returning 5xx responses or appears stuck:
+
+       docker compose restart quicknotes
+
+2. Roll back the most recent configuration or image change if the issue started immediately after a deploy.
+3. If malformed client traffic is causing a flood of 4xx responses, temporarily block or rate-limit the offending client/source if available.
+4. If the data volume is suspected to be corrupt, stop the service and inspect `/data/notes.json` before deleting or replacing anything.
+
+## Post-incident
+
+After the incident is mitigated:
+
+1. Write a short postmortem using the Lecture 1 postmortem template.
+2. Record the timeline: detection time, start time, mitigation time, and resolution time.
+3. Identify whether users were affected and for how long.
+4. Add or improve tests, alerts, dashboard panels, or validation so this failure is easier to detect next time.
+5. Update this runbook if any step was missing or misleading.

labs/lab1.md

@@ -91,9 +91,20 @@ git config --global commit.gpgsign true
 git config --global tag.gpgsign true
 ```
 
-Tell the platform your SSH key is a **signing key**:
-- GitHub: Settings → SSH and GPG keys → **New SSH key**, key type **Signing Key**
-- GitLab: Profile → SSH Keys → tick "Usage type: Authentication & signing"
+Now register the key on the platform. GitHub treats **Authentication** and **Signing** as *separate* roles for the same key, so you add it under both:
+
+- **Authentication Key** — lets you `clone` / `fetch` / `push` over SSH (`git@github.com:…`). If you cloned over HTTPS, or have never seen `ssh -T git@github.com` greet you by name, you don't have one configured yet — add it now or the `upstream` SSH remote will fail in Lab 2.
+- **Signing Key** — gives your commits the **Verified** badge.
+
+- 🐙 GitHub: Settings → SSH and GPG keys → **New SSH key** → add the **same** `~/.ssh/id_ed25519.pub` **twice**, once with Key type **Authentication Key** and once with **Signing Key**.
+- 🦊 GitLab: Profile → SSH Keys → a single key with **Usage type: Authentication & signing** covers both.
+
+Confirm authentication works before moving on:
+
+```bash
+ssh -T git@github.com
+# expect: Hi YOUR_USERNAME! You've successfully authenticated...
+```
 
 ### 1.4: Make a Signed Commit
 
@@ -303,7 +314,8 @@ In `submissions/lab1.md`:
 ## Common Pitfalls
 
 - 🪤 **PR template doesn't auto-populate** — make sure the template is on `main` *before* opening the PR
-- 🪤 **Commits show "Unverified"** — the SSH key must be added as a *Signing Key* on GitHub (not just an authentication key)
+- 🪤 **Commits show "Unverified"** — the key must also be added as a **Signing Key** on GitHub; an Authentication Key alone won't verify commits (they're separate roles — see §1.3)
+- 🪤 **`git@github.com: Permission denied (publickey)` on clone/fetch/push** — the *reverse* gap: your key is registered for signing but not as an **Authentication Key**. Add it as Authentication too (§1.3) and confirm with `ssh -T git@github.com`. Quick unblock for the *public* upstream: `git remote set-url upstream https://github.com/inno-devops-labs/DevOps-Intro.git`
 - 🪤 **`git push` rejected on `main`** — that's the bonus rule working as designed; push to `feature/lab1` instead
 - 🪤 **`gpg.format=ssh` ignored** — confirm Git ≥ 2.34: `git --version`
 - 🪤 **Pushed to the wrong branch** — `git switch feature/lab1` before `git push`

labs/lab2.md

@@ -223,6 +223,7 @@ git bisect reset
 
 ## Common Pitfalls
 
+- 🪤 **`git@github.com: Permission denied (publickey)` on `git fetch upstream`** — *not* a remote-config bug (the error is at the SSH layer, before Git reads the repo). Your key isn't registered for **authentication** on GitHub — and a **Signing Key** (Lab 1) does *not* count for auth, they're separate roles. Add the same `~/.ssh/id_ed25519.pub` as an **Authentication Key** (Lab 1 §1.3), verify with `ssh -T git@github.com`, then re-run. To unblock right now, the public upstream fetches over HTTPS with no key: `git remote set-url upstream https://github.com/inno-devops-labs/DevOps-Intro.git`
 - 🪤 **`reset --hard` without committing first** — your *uncommitted* edits really *are* gone (reflog only saves committed work). Always check `git status` first
 - 🪤 **`tag -v` says "no signature"** — you used `git tag NAME` instead of `git tag -a -s NAME -m "..."`
 - 🪤 **Rebase conflicts** — resolve, then `git rebase --continue`. Never `git rebase --skip` unless you know what you're skipping

labs/lab3.md

@@ -160,6 +160,23 @@ Tips:
 - GitLab: `parallel:matrix:`
 - Set `fail-fast: false` (GH) or equivalent so a single bad cell doesn't cancel the others — you want to *see* which combo broke
 
+> ⚠️ **The matrix renames your checks — update branch protection (1.6) or your PR blocks forever.** A matrixed `test` job reports as `test (1.23)` and `test (1.24)`; the old required check named `test` will sit at *"Expected — Waiting for status to be reported"* indefinitely, even though every real check is green. Two fixes:
+>
+> 1. **Quick:** in the branch-protection rule, replace `vet`/`test` with the matrixed names (`vet (1.23)`, `vet (1.24)`, `test (1.23)`, `test (1.24)`).
+> 2. **Robust (recommended):** add one aggregation job and require *only* it — then the matrix can change freely without touching protection settings:
+>
+> ```yaml
+> ci-ok:
+>   if: always()
+>   needs: [vet, test, lint]
+>   runs-on: ubuntu-24.04
+>   steps:
+>     - run: |
+>         test "${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}" = "false"
+> ```
+>
+> The `if: always()` matters — without it, a failed `needs` job *skips* `ci-ok`, and a skipped required check lets the PR through on some configurations.
+
 ### 2.3: Skip docs-only changes
 
 Edit your trigger so the pipeline runs **only** when something in `app/` or your CI config itself changes. README edits should not burn 4 minutes of CI time.
@@ -179,6 +196,8 @@ Capture wall-clock times from the CI UI for three scenarios:
 
 > 💡 To get a clean baseline, temporarily disable each optimization with a commit, take a screenshot of the run time, then restore.
 
+> 🧪 **Expect the cache rows to be boring — that's the finding, not a failure.** QuickNotes has **zero third-party dependencies** (look at `app/go.mod` — no `require` block, no `go.sum`), so the module cache has nothing to store and total wall-clock barely moves with `cache: true` vs `cache: false`. Most of your 60–80 s is runner provisioning, checkout, and the Go toolchain download — none of which `setup-go`'s cache touches. Report what you measured and *explain why* (that's design question **f** in disguise). To see where caching *would* pay, compare the **per-step** durations (`setup-go`, `go test`) instead of job totals, and note which step a real dependency-heavy project would save on.
+
 ### 2.5: Document
 
 In `submissions/lab3.md`:
@@ -284,6 +303,8 @@ Answer in 4-6 sentences:
 - 🪤 **Forgot `working-directory` (or `cd app`) for Go commands** — Go modules live in `app/`, not the repo root; commands run from the root will fail with "no Go files"
 - 🪤 **`fail-fast: true` (the GH Actions default) in a matrix** — one fail cancels the others; you can't see *which* combo broke
 - 🪤 **Branch protection set on someone else's fork's `main`** — you can only protect *your* fork's `main`. The upstream course repo has its own protection
+- 🪤 **PR stuck on "Expected — Waiting for status to be reported" after adding the matrix** — the matrix renamed `test` → `test (1.23)`/`test (1.24)`, but branch protection still requires the old `test` context, which will never report again. Update the required-check names or switch to the `ci-ok` aggregation job (see §2.2)
+- 🪤 **"Caching didn't speed anything up"** — on a zero-dependency module that's the *correct* result, not a mistake (see §2.4); don't pad the timing table with numbers you didn't observe
 - 🪤 **`golangci-lint` version not pinned** — "latest" pulls a new release tomorrow that may flag your code with new rules. Pin `v2.5.0` exactly
 - 🪤 **GitLab CI: incorrect anchor syntax** (`<<: *name`) — GitLab is strict; use the in-platform CI Lint tool (`Project → CI/CD → Editor → Validate`)
 - 🪤 **Cache hits expire after 7 days of inactivity on GH** — that's expected; the cache key is what protects you against poisoning