Refactor#153
Open
kathsherratt wants to merge 38 commits into
Open
Conversation
Plan to replace the per-target stratified GAMM fit (separate cases/deaths bam() models) with a single model carrying epi_target as a fixed factor, plus a plan-only design for a data-revision covariate sourced from the Hub archive anomalies.csv. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
README.html and README_files/ are Quarto render output, not source. Remove them from tracking; they are regenerated from README.qmd. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- New title emphasising the role of model structure in predictive performance (README.qmd, manuscript.qmd). - Switch manuscript render format pdf -> html. - Reformat author/affiliation block to numbered style. - Wire in the supplement include; drop the references callout wrapper. - Update TODO callouts (survival/proportional methods note). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Recast the abstract, background, and discussion around a single framing: forecast performance varies from two sources — the forecast-generating process (modeller choices) and the target-generating process (epidemic dynamics, surveillance, data revisions). We adjust for the latter to isolate the former. - Replace "data/controlled direct effect" language with "adjusted partial association", matching what the model actually estimates. - Update the headline result to the overlapping-CIs framing: no single structural approach dominated. - Collapse duplicated paragraphs and remove the large HTML-commented decision block in the discussion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Condense the eligibility and model-classification paragraphs. - Fix typos (heirarchical -> hierarchical, dymamics -> dynamics). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Name the model a generalised additive mixed model (GAMM) throughout. - Fix the trend claim: forecasting was harder during increasing trends for both cases and deaths (previously misstated deaths as harder in decline). - Restate the model-structure result as overlapping CIs around the grand mean rather than "attenuated to null". - Condense the country-target paragraph; fix typos; note the identity-link robustness check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Deparse results$formula so the multi-line model formula renders as a single string. - Replace "controlled direct effect" with the adjusted partial-association wording used elsewhere; drop the redundant supplement title. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Previously fit separate models for cases and deaths. Now fit one model with epi_target as a fixed factor, giving a direct deaths-vs-cases contrast. Simplifies code, removes faceting from plots and tables, and unifies covariate effects across targets.
The joint-model refactor replaced per-target diagnostic plots (check_Cases.pdf / check_Deaths.pdf) with a single check_joint.pdf, but the supplement still include_graphics()'d the old filenames. Collapse the two (A) Cases / (B) Deaths sub-panels into one combined include. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The primary GAMM uses gaussian(link="log") on the raw WIS. The log link models the score multiplicatively but assumes Gaussian errors on the skewed raw scale, leaving heavily right-skewed residuals (skew ~5.5) and a wild QQ plot. - analysis-model.R: filter NA wis explicitly (was dropped silently by bam); save observed-vs-fitted (fit_obs.rds). Primary effects unchanged (max abs diff 0). Hoist shared joint formula to file scope. - sensitivity/model-logresp.R: refit the joint model on log(WIS) with an identity link. Residual skew improves ~5.5 -> ~-0.8; effect directions preserved (focal cor ~0.96), model-structure conclusions identical. - plot-model-results.R: add plot_fit_obs() observed-vs-fitted helper. - supplement: add observed-vs-fitted figure, honest note on residual skew, and a log-response sensitivity sub-section. - results.qmd: note the residual skew and point to the sensitivity check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CLAUDE.md references Plan.md as the change log but the file never existed. Create it and record the joint-target refactor and the model-fit diagnostics / log-response sensitivity work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The supplement set eval=FALSE globally, so every chunk showed code but produced no output. Enabling evaluation surfaced three latent errors in chunks that had never run: - eval=FALSE -> eval=TRUE in the setup chunk. - flowchart path: output/plots -> output/log/plots. - supplement-metadata: table_confint() now returns n_forecasts, not Forecasts; alias on select. - plot_fit_obs: free-scale facets are incompatible with coord_equal; apply coord_equal only when unfacetted. Supplement figures and tables (including the new fit-obs and log-response diagnostics) now render. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… supplement Main-text figures, Table 2, and prose previously showed partial effects on both the raw log-WIS scale and the exponentiated ratio scale. Log-scale coefficients are uninterpretable to an epi audience; the multiplicative ratio (1 = grand-mean WIS) is the natural quantity and matches RR/IRR conventions. - plot-model-results.R: plot_effects() and plot_models() now plot exp(value) with exp() CI bounds, reference line at 1.0, scale_y_log10(), axis label "Performance ratio (vs average model)". - analysis-descriptive.R (print_table2): drop raw ci_text columns; ratio columns only; caption reworded to ratio framing. - _results.qmd: inline Trend/Variant/Deaths stats value_ci -> ratio_ci; Fig 2/3 captions and methods paragraph reworded for ratios. - _supplement.qmd: add raw log-scale coefficient table so dropped main-text numbers remain available for reviewers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fixing inconsistencies between model specs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
apologies for the absolute mess of a PR. keeping as a separate branch for comparison but got tangled up so this is really an all-in-one overhaul of main. sorry.