Keep fresh database migrations idempotent

[Y1fe1Zh0u]

Root cause

The initial_schema revision creates tables from current ORM metadata on a fresh database. The logical-deletion models already contain the deleted_at columns and active partial indexes, but add_agent_model_deleted_at attempted to create the same objects unconditionally. Fresh deployments therefore failed with PostgreSQL DuplicateColumnError before application startup.

What changed

• Inspect each logical-deletion column and index before creating it.
• Handle partial schemas object by object instead of using one migration-wide early return.
• Guard downgrade operations against missing objects.
• Add focused migration tests for missing, complete, and partial schemas.
• Add a standalone Drone stage that starts an empty PostgreSQL database and runs alembic upgrade head without starting the backend application, Redis, or frontend.
• Verify both deleted_at columns, both active partial indexes, and the Alembic head after migration.
• Preserve upgrade coverage when v1.11.2 is the source release by building its database through the parent revision and applying the corrected head before starting the old application.

Impact

Fresh installations can reach the current Alembic head even though the initial revision has already materialized the current logical-deletion schema. Existing databases still receive any missing columns or indexes during normal upgrades.

Validation

• 12 scoped backend tests passed.
• Ruff passed for changed Python files.
• Shell syntax validation passed.
• Docker Compose configuration validation passed.
• Drone and Compose YAML parsing passed.
• git diff --check passed.
• Drone build 377 passed:
  • fresh-database-migration-test
  • local-deploy-test
  • local-upgrade-test from v1.11.2
  • cleanup-ci-images

The successful upgrade reached add_agent_model_deleted_at head and LangGraph checkpoint version 9 while preserving the database and workspace sentinels.

Follow-up scope

This hotfix does not rewrite initial_schema or change dependency resolution:

1. Replace the dynamic ORM-backed initial revision with an audited static baseline in a separate migration-governance PR.
2. Pin Alembic, SQLAlchemy, and asyncpg exact versions in a separate reproducible-build PR.

Those stability improvements require broader audits and are intentionally separated from the deployment-blocking fix.