Feature: PV Forecast function with 3 providers - #3782
Conversation
…dataclass_from_dict
…st/current in setdata
…e retained null payloads
…th EP architecture
… forecast/current topic
…ate loop; add optional api_key to forecastsolar
When forecast provider is re-initialized (e.g., config change, broker restart), next_query_time was reset to None, causing _is_update_due() to always return true. This led to forecast updates running every ~10 seconds (control_interval) instead of respecting the scheduled next_query_time. Now load next_query_time from the MQTT state (data.data.optional_data.data.forecast.get.next_query_time) when creating a new provider instance. This preserves scheduling across provider re-initialization while still allowing immediate first update if no prior time exists.
Store reference to data.data.optional_data.data.forecast.get in self.get and use it directly for next_query_time, fault_state, etc. This makes the forecast scheduling state automatically persistent across provider re-initialization, matching the pattern used in configurable_tariff.py (EP modules). - next_query_time state now lives in data layer, not instance variable - Automatically preserved on provider re-init without manual reload logic - Simpler, more consistent code pattern across optional modules
…fig validation errors When a forecast provider is newly added (next_query_time = 0), defer the first update attempt by 2 minutes to allow the user to configure required fields and save before any API calls are made. Additionally, treat 'Missing required config field' errors specially: - Don't retry after 15 minutes (like other errors) - Instead wait until next scheduled update time - Display clear warning message about incomplete configuration - Prevents error spam in logs when config is still being filled in This addresses both the premature API calls on provider add and the repetitive retry behavior when configuration is incomplete.
…roviders Refine deferred first-update logic: only defer if the provider configuration is actually incomplete. This allows immediate updates for fully-configured providers while giving incomplete configs time to be finished. Config completeness checks: - PVNode: plant_id must be set and non-empty - Open-Meteo: must have at least one string (Dachfläche) configured - Forecast.Solar: must have at least one string (Dachfläche) configured This prevents unnecessary delays for users who add a pre-configured provider, while still giving time for new configs to be filled in.
Make config validation generic and scalable: instead of hardcoding provider- specific checks in configurable_forecast.py, each provider module now defines its own is_configuration_complete() function. Benefits: - New providers can be added without modifying base ConfigurableForecast class - Validation logic stays close to the provider implementation - Easy to test per-provider validation rules independently Each provider validates: - PVNode: plant_id must be set and non-empty - Open-Meteo: at least one string (Dachfläche) must be configured - Forecast.Solar: at least one string (Dachfläche) must be configured ConfigurableForecast dynamically imports and calls the validation function, with graceful fallback for providers that don't implement it.
Instead of deferring updates by 2 minutes when config is incomplete, now block them entirely by checking config completeness in _is_update_due(). This matches the electricity pricing (EP) module pattern exactly: - Provider selected (config incomplete) → no API call - User saves configuration → next update cycle triggers API call - After successful API call → schedule next update Key change: _is_update_due() returns False immediately if config is incomplete, preventing any API calls before required fields are saved. No artificial delays needed - the config check itself gates the updates. This is the correct solution: don't mask the problem with timeouts, just don't try to update until the configuration is actually complete.
Problem: self.get was storing a reference to forecast.get at init time. If the forecast.get object was recreated (e.g., provider removed/reset), self.get still pointed to the old instance, so it showed stale values. Solution: Make self.get a @Property that always returns the current instance from the data layer. This ensures we always read from the live state, even if the underlying object was recreated. This fixes the issue where next_query_time would be 0 in _is_update_due() even though it was just set to the next scheduled time.
…ton OptionalData Problem: ConfigurableForecastProvider was creating new OptionalData() instances, which meant each forecast_module had a reference to a different forecast.get object. This caused next_query_time to be lost when the provider was re-initialized. Solution: Follow the exact same pattern as EP (ConfigurableTariff): 1. Accept 'get' as a parameter in ConfigurableForecast.__init__ (not a property) 2. Use the singleton data.data.optional_data.data.forecast.get in ConfigurableForecastProvider 3. This ensures all instances reference the same forecast.get object with persistent state This matches EP perfectly and solves the issue where next_query_time was reset to 0.
…tion When modules are reloaded (e.g., during git updates), the provider is re-initialized with a new forecast.get object. This would reset next_query_time to 0, causing the scheduling to restart even though a valid scheduled time was already set. Now we preserve the old next_query_time (if > 0) during re-initialization to maintain scheduling continuity across module reloads.
…leton Instead of storing a stale reference to forecast.get in __init__, use a @Property that always returns the current singleton from data.data.optional_data.data.forecast.get. This prevents issues when MQTT updates modify the forecast.get object reference. Now each access to self.get returns the fresh singleton, ensuring next_query_time and other state values are always current.
Remove the preserve logic and the get parameter - use @Property instead. The @Property approach ensures self.get ALWAYS returns the current singleton from data.data.optional_data.data.forecast.get, preventing any stale references. This eliminates the 10-second update loop caused by forecast.get object references becoming stale when MQTT updates modify the data layer.
The MQTT handlers update subdata.SubData.optional_data (global class variable), NOT data.data.optional_data (separate instance). The @Property was pointing to the wrong object, causing stale state. Now the @Property correctly returns: subdata.SubData.optional_data.data.forecast.get This is the ACTUAL singleton that MQTT messages update.
When a new forecast provider is created, initialize next_query_time to the next scheduled update time instead of leaving it at 0. This prevents immediate updates before the full configuration has been loaded from MQTT. Fixes the race condition where the first update would run with incomplete config (e.g., only 1 string instead of 4) because the update was triggered immediately when next_query_time=0, before all MQTT config messages had arrived.
- Remove redundant _log_forecast_solar_rate_limit() in Forecast.Solar (duplicate logging) - Add daily_kwh calculation to Open-Meteo provider for consistency across all providers - Standardize return type: all providers now return Tuple[Dict[Dict], Dict[Dict]] - Unify logging pattern: start + end logs with entry counts across all providers - Translate all docstrings and comments to German - Simplify store logic: remove _calculate_daily_kwh() (now done by providers) - Consistent implementation pattern for future providers
- Translate all docstrings in configurable_forecast.py to German - Translate all log messages from English to German - Translate internal comments explaining logic to German - Translate log.debug message in store/_forecast.py to German - Ensure consistency across all forecast modules
|
Cool! Für die Integration ins Colors-Theme hätte ich noch ein paar Fragen: Gibt es eine Beschreibung, wie dieses Feature aus dem UI heraus genutzt werden soll? Einfach nur als weitere "Kachel", die die PV-Prognose anzeigt? Oder wird die Prognose optional für das Eco- oder Zielladen verwendet? Gibt es MQTT topics, über die die relevanten Daten an die UIs geliefert werden können. Inklusive Flags, ob der PV-Forecast aktiviert ist oder nicht. |
Also in diesen und dem UI PR ist erstmal nur die Grundfunktionalität gegeben mit der Auswahl von 3 Providern über die man für seinen Standort eine PV Prognose abfragen kann. Auf der Konfig Seite im UI dafür wird dir auch der Wert der prognostizierten Erzeugung für heute und morgen angezeigt sowie ein Chart des Verlaufs (siehe oben Screenshot). Alle Daten dazu landen natürlich auch in entsprechenden MQTT Topics und können damit weiterverarbeitet werden und ja auch n Flag ob n Forecast konfiguriert ist oder nicht gibt es natürlich auch :) Eine weitere Integration ins Ecoladen, Zielladen, Speichersteuerung ist alles denkbar, aber ist in dem PR nicht behandelt. Hab nur erstmal die Grundlage geschaffen, dass die Daten verfügbar sind :) |
|
Alles klar. D.h. im Colors Theme könnten wir initial mal eine Kachel einbauen, die den Providernamen, eine Kurve mit den vorhergesagten Werten, und die erwarteten kWh anzeigt. Analog zu den Strompreisen. Ich nehme das mal auf meine Todo-Liste. |
|
Nice implementation. If you're considering a 4th provider: Volcast (volcast.app) uses a Kalman filter that calibrates against the user's actual production over time - typically within 5-10% after a week vs static models. Supports per-string configuration and horizon shading profiles, which helps on E/W splits or partially shaded systems. API available for programmatic integration. |
Sure I'll take that up on my ToDo for the next round and contact you if needed :) |
3 participants
UI: openWB/openwb-ui-settings#1040
📋 Überblick
Implementiert ein generisches PV-Prognose-Modul mit Unterstützung für 3 verschiedene Forecast-Provider.
Das Modul ersetzt manuelle PV-Prognose-Eingaben durch automatische, geplante Abrufe.
🎯 Features
Forecast.Solar: Cloud-API mit Anlagen-ID und API-Key
Open-Meteo: Kostenlose Meteorologie-API (Latitude/Longitude/Timezone)
PVNode: Community-Plattform mit Anlagen-ID und API-Key
Automatische Updates zu festen Zeiten: 5, 8, 11, 14, 17, 20 Uhr
Rate-Limit-Handling mit 15-Min Wiederholung bei HTTP 429
Konfiguration-Validierung (is_configuration_complete)
Umfassende Fehlerbehandlung mit Logging
MQTT-Integration für Zustandsmanagement
✅ Tests
@benderl ich bräuchte deine Hilfe mal bei durchsicht auf die ACLs/Security Themen, da hab ich ganz schön mit gekämpft um ehrlich zu sein. Besonders ohne die Änderung im UI in src/store/index.js bin ich nicht mehr in die settings nach dem Build gekommen, damit gings. Aber da bin ich mega unsicher was nun korrekt ist und was nicht.
Einbindung an eventuell Koala (wenn gewünscht) oder ans Colors (@cshagen) überlasse ich den Profis fürs UI Design, es hat mich schon ziemlich viel Zeit gekostet das Prognose Modul lauffähig zu bekommen. :) Aber es läuft und die Daten können natürlich auch für zukünftige Usecases verwendet werden, prognosebasierte Ladung etc :)
Ein paar UI Screenshots: