Skip to content

Fix leak of model_partitions catalog rows when deleting a partitioned model#9711

Merged
nishantmonu51 merged 1 commit into
mainfrom
nishant/leak-partition-on-delete
Jul 20, 2026
Merged

Fix leak of model_partitions catalog rows when deleting a partitioned model#9711
nishantmonu51 merged 1 commit into
mainfrom
nishant/leak-partition-on-delete

Conversation

@nishantmonu51

Copy link
Copy Markdown
Collaborator
  • Deleting a partitioned model that had produced output dropped the output table and returned immediately, skipping clearPartitions. The model's rows in the model_partitions catalog table were orphaned forever, since each model gets a fresh PartitionsModelId and clearPartitions is the only caller of DeleteModelPartitions.
  • The deletion branch in ModelReconciler.Reconcile now falls through after a successful output drop, so clearPartitions runs on every delete path.
  • Added a regression test (TestPartitionsClearedOnDelete) that fails on the old code (all partition rows orphaned) and passes with the fix.

Steps to reproduce:

  1. In Rill Developer, create a partitioned incremental model and let it build:
    type: model
    incremental: true
    materialize: true
    partitions:
      sql: SELECT range AS num FROM range(0, 50)
    sql: SELECT {{ .partition.num }} AS num
  2. Verify partition rows exist in the catalog: sqlite3 <project>/tmp/meta.db "SELECT COUNT(*) FROM model_partitions;" returns 50.
  3. Delete the model file and wait for reconcile (the output table is dropped).
  4. Re-run the query from step 2. Before this fix, all 50 rows remain orphaned, and repeated create/delete churn grows the catalog DB unbounded. With the fix, the count returns to 0.

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

In the model reconciler's deletion branch, models that had produced output
returned immediately after dropping the output table, skipping clearPartitions.
Since each model gets a fresh partitions model ID, the orphaned model_partitions
rows could never be reclaimed, growing the catalog DB on create/delete churn.

Now clearPartitions runs on every delete path after a successful output drop.
@nishantmonu51
nishantmonu51 requested a review from k-anshul July 20, 2026 06:52
@nishantmonu51 nishantmonu51 changed the title Fix leak of \model_partitions\ catalog rows when deleting a partitioned model Fix leak of model_partitions catalog rows when deleting a partitioned model Jul 20, 2026

@begelundmuller begelundmuller left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nishantmonu51
nishantmonu51 merged commit 06ecbe7 into main Jul 20, 2026
16 checks passed
@nishantmonu51
nishantmonu51 deleted the nishant/leak-partition-on-delete branch July 20, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants