Track the deferred gradient for analytic per-series scaling (ADR-0066/ADR-0475): no gradient job_type can fit a scaled column

[wshlavacek]

Analytic per-series scaling (scale, added in #479) has no gradient, so any fit whose experiment scales a column is unavailable to both gradient job types. This is a tracking request for the deferred derivative, not a bug report against #479 — that issue is closed and the feature works.

Current behaviour

pybnf/objective.py:~633:

if col_name in self._scaled_columns:
    raise GradientNotSupported(
        "Analytic per-series scaling ('scale', #479) on column '%s' is not differentiable "
        "on the gradient path (its optimal scale depends on theta through the whole "
        "series); use a gradient-free step." % col_name)

and the surrounding docstring:

Analytic per-series scaling (ADR-0066, #479) has a deliberately deferred gradient: its profiled c* depends on θ through the whole series (an implicit-function derivative not yet implemented), so a scaled column raises GradientNotSupported here and the fit falls back to a gradient-free step (ADR-0475). The much simpler floor gradient is deferred the same way in pybnf.gradient.assembly.

The reasoning is sound: c* is profiled out of the whole series, so dc*/dθ is an implicit-function term rather than a local derivative.

What we observe

Fitting reduced_onoff (Jaruszewicz-Błońska 2023), whose experiment analytically scales column A20, on bngsim 0.12.0:

• job_type = trf — refuses:
  Gradient-based fitting (job_type = trf) cannot differentiate this fit's objective: Analytic per-series scaling ('scale', #479) on column 'A20' is not differentiable on the gradient path
• job_type = lbfgs — refuses identically, with 2 recorded gradient fallbacks

So there is currently no gradient job type available for a scaled fit. That is consistent with the design note; this issue just asks for it to be trackable.

Why we are asking

We are benchmarking PyBNF's gradient path for a paper, and this row is one of six. We need something citable and accurate for the caption. Citing "#479" reads as though that issue is open and the feature is broken, which misrepresents it — #479 landed the feature and it works on the derivative-free path. What is unimplemented is the profiled scale's derivative.

An open tracking issue would let us cite the actual gap, and would also let a user who hits GradientNotSupported find a status rather than a closed feature ticket.

Possible dispositions, in case it is useful

• implement dc*/dθ via the implicit function theorem on the profiling condition, which for a linear-in-c least-squares profile has a closed form;
• or declare it permanently out of scope and say so in the message, so the refusal reads as a design boundary rather than a pending item.

Either way, having the message point at an open issue rather than a closed one would help.

Environment

• PyBNF e8cb481d (editable install)
• bngsim 0.12.0, core built from f4b24ac69138
• macOS 15, Python 3.12