Add Nori tabular regression plugin

[ajanbekzat]

Summary

nori_regression is a new Processing Engine plugin that predicts a numeric field in an InfluxDB 3 measurement from other columns on the same rows with Nori, Synthefy's in-context-learning tabular regression model, served through the Baseten gateway. It trains on the rows where the target field is present and predicts (imputes) the rows where it is null, writing the predicted values back as a new measurement. It runs as a scheduled trigger (on an interval) or an HTTP trigger (on-demand).

What Nori is (for reviewers not familiar with it)

Nori is a tabular regression foundation model. You send it labeled feature rows (X_train, y_train) and query rows (X_test) in a single request, and it predicts a value for each query row in one forward pass, with no training or fine-tuning. It has no notion of time or row ordering: it only sees numeric feature rows.

How the plugin works

1. Reads a window of the measurement: the target field plus the feature_fields columns, filtered by tags.
2. Splits rows into the training set (target present) and the prediction set (target null, all features present).
3. Sends X_train/y_train/X_test to Nori (task: "regression").
4. Writes each prediction back at its own row's timestamp to <measurement>_regressed.

Typical uses: backfill a field that dropped out, impute a metric from correlated ones (for example, predict pressure from temp and humidity), or derive an expensive-to-measure field from cheaper ones recorded alongside it.

Trigger types

• Scheduled (process_scheduled_call): run on an interval; re-imputes any rows still missing the target each tick.
• HTTP (process_request): on-demand at /api/v3/engine/<path>; supports an explicit start_time/end_time window.

Configuration highlights

• measurement, field: the source table and the target field to predict (required).
• feature_fields: the feature columns (X), ./,-separated (required).
• window, or start_time + end_time: which rows to read.
• tags: filter to a single series (required if the measurement holds several series).
• model: the Nori gateway slug (synthefy/nori default, or synthefy/nori-30m).
• output_measurement, target_database, dry_run, min_history, config_file_path.

Full parameter tables are in the README.

Authentication

The Nori gateway key is a secret and is never read from trigger arguments or the request body (both are logged). It is resolved from, in order: an incoming request header (X-Nori-Api-Key, or Authorization: Api-Key on the HTTP trigger), then the NORI_API_KEY environment variable on the InfluxDB host. Keys are obtained from the Synthefy console (https://console.synthefy.com/); a single key works for all Nori model variants.

One InfluxDB-specific note for the HTTP trigger: InfluxDB's own HTTP layer parses the Authorization header and only accepts the Bearer/Token/Basic schemes, so Authorization: Api-Key is rejected before the plugin runs. Use the custom X-Nori-Api-Key header instead. This is documented in the README.

Testing

Verified end-to-end against influxdb:3-core v3.10.1 with a production key:

• On a sensors table where pressure = 1000 + 0.5*temp - 0.3*humidity + noise(sd 0.8), the plugin imputed the 24 rows missing pressure from temp and humidity. Predictions matched the true relationship to within the training noise (absolute errors 0.02 to 0.71).
• A multi-series regression without a qualifying tags filter fails loud (a would-collide guard) rather than silently dropping values; a wrong or missing key returns a clean failed rather than crashing.

Conformance to repo conventions

• scripts/validate_readme.py --plugins nori_regression: 0 errors.
• scripts/validate_influxdb3_syntax.py --readme influxdata/nori_regression/README.md: 0 errors, all CLI and API usage validated.
• Follows the organization/plugin_name/ layout with the JSON metadata docstring, README, manifest.toml, and requirements.txt, plus a plugin_library.json entry (author Synthefy).
• Writes go through write_sync/write_sync_to_db (InfluxDB 3.8.2+, the same pattern as the mqtt_subscriber and synthefy_forecasting plugins) so write errors surface during trigger execution instead of being swallowed by the buffered writer.
• The Nori API key is read only from a request header or the NORI_API_KEY environment variable, never from trigger arguments or the request body.

Files added

• influxdata/nori_regression/nori_regression.py: the plugin (metadata docstring plus implementation).
• influxdata/nori_regression/README.md: usage, configuration, and troubleshooting.
• influxdata/nori_regression/manifest.toml: packaging metadata.
• influxdata/nori_regression/requirements.txt: Python dependencies.
• influxdata/library/plugin_library.json: registry entry (the plugin lives in the influxdata/ namespace, authored by Synthefy).

Limitations

• One series per run; run multiple triggers for multiple series (the plugin fails loud if a run resolves to more than one series). Multi-series imputation in a single run is a possible enhancement.
• Imputes only rows where the target is null; it does not overwrite existing values.
• Prediction quality depends on how well the feature columns explain the target.
• Each call is a billed gateway request; a larger window sends more rows and tokens.

[Aliaksei-Kharlap]

Influx uses the Authorization header to authorize incoming requests, so it is not possible to use it for passing other access tokens. Therefore, it is appropriate to remove any references to it—and the associated logic—from the plugin

[Aliaksei-Kharlap]

Since Influx allows field names to contain a dot ("."), this argument format will be parsed incorrectly. There is also a parsing limitation regarding the --trigger-arguments parameter when using the CLI: it does not allow commas (",") within argument values, as doing so leads to incorrect parsing of the trigger arguments. A space is the preferred delimiter

[Aliaksei-Kharlap]

Same remark as for the feature_fields arg

[Aliaksei-Kharlap]

The documentation describes this parameter and the option to use a TOML configuration, but the plugin code lacks this logic

[Aliaksei-Kharlap]

eries.rolling(N).mean()/.std() includes the current point, so the roll_mean/roll_std for row t already contain y[t] — the value
we're trying to predict. This leaks the label into training (and contradicts the docstring)

[ajanbekzat]

@Aliaksei-Kharlap thanks for the review, and for the detailed comments. One larger change first: we have refocused this plugin to be regression-only and renamed it (nori_forecasting to nori_regression). We removed the time-series forecast mode to keep the plugin focused on Nori's core capability, tabular regression. The plugin now predicts a target field from other columns on the same rows and fills (imputes) the rows where the target is null.

How each comment is addressed:

1. Rolling-window label leakage (rolling(N).mean() includes the current point). Resolved: that feature-engineering code was removed along with the forecast mode.
2. Authorization header. Removed. The key is no longer read from the incoming Authorization header (InfluxDB uses that header for its own request authorization, so it never reached the plugin anyway); it is resolved only from an X-Nori-Api-Key header or the NORI_API_KEY environment variable. Both the references and the logic are gone.
3. feature_fields delimiter. Now space-separated (for example temp humidity). It no longer splits on . (InfluxDB field names can contain a dot) or on , (which --trigger-arguments uses to split argument pairs). A JSON list is also accepted in the HTTP request body.
4. tags delimiter. Same fix: space-separated key:val pairs.
5. config_file_path / TOML configuration. Removed from the metadata and README, since the plugin did not implement it.

CI is green. This is ready for another review, thank you.