From b9a5a1cc0e9533f70793c35971dbf36751674daf Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 11:05:05 +0300 Subject: [PATCH 1/2] Consolidation C-1: proxyclient loses its internal header First step of the module consolidation (owner decision 2026-07-04: consolidate now; linting may be disabled selectively where the known clangd bug produces false errors). Order is consumers-first: a library's headers can only be deleted once no other package's headers still include them, so consolidation walks the dependency chain from the top; the proxy client is the topmost consumer and the smallest. - LibProxyClientApi.hpp (673 lines) merged into streamrproxyclient.cpp, which now imports the four sibling modules (streamr.dht, streamr.logger, streamr.trackerlessnetwork, streamr.utils) instead of textually including their headers. Third-party libraries, the standard library and the public C API header remain textual includes. - streamr_enable_imports(streamrproxyclient) so the implementation is scanned for module dependencies. - StreamrModules.cmake (canonical + synced copies): the Android NDK version floor is now a hard configure-time error instead of a textual fall-back - the fall-back required the internal headers that consolidation deletes. - Two NOLINT(bugprone-exception-escape) suppressions: the checker cannot see through imported module interfaces (known pattern from the import-using test files). - MODERNIZATION.md: the owner decision, the revised order and the C-1 record. Verified: Release build green, proxyclient tests 15/15, standalone package build green (synthesized module compilation in the consumer build dir), package lint green. Co-Authored-By: Claude Fable 5 --- MODERNIZATION.md | 38 +- cmake/StreamrModules.cmake | 22 +- packages/streamr-dht/StreamrModules.cmake | 22 +- .../streamr-eventemitter/StreamrModules.cmake | 22 +- packages/streamr-json/StreamrModules.cmake | 22 +- .../CMakeLists.txt | 9 + .../StreamrModules.cmake | 22 +- .../src/LibProxyClientApi.hpp | 674 ----------------- .../src/streamrproxyclient.cpp | 676 +++++++++++++++++- packages/streamr-logger/StreamrModules.cmake | 22 +- .../streamr-proto-rpc/StreamrModules.cmake | 22 +- .../StreamrModules.cmake | 22 +- packages/streamr-utils/StreamrModules.cmake | 22 +- 13 files changed, 816 insertions(+), 779 deletions(-) delete mode 100644 packages/streamr-libstreamrproxyclient/src/LibProxyClientApi.hpp diff --git a/MODERNIZATION.md b/MODERNIZATION.md index 2b15c413..78deb99e 100644 --- a/MODERNIZATION.md +++ b/MODERNIZATION.md @@ -770,11 +770,39 @@ verified NDK clang (18 = r27), with the textual fallback below it. StreamID touch REGRESSED +68% from BMI-chain invalidation). - The final IWYU/ODR hygiene of the no-headers end state. -### Recommended path (when unblocked) -Leaf-first and interleaved as originally planned — eventemitter as the -consolidation canary, one package per PR, `bench.sh` measured at every -step (the 2.4 lesson: measure before generalizing), headers deleted -per-package with a grep-enforced "nothing includes them" gate. +### DECISION (owner, 2026-07-04): consolidate NOW +The owner decided to start consolidation immediately, accepting the +clangd limitation: editor diagnostics may show false errors in the +affected pattern, and linting is disabled selectively (per file, with a +comment) where the clangd std-type-unification bug produces false +errors. Test files are the expected main victims; the linter keeps +running everywhere it is correct. + +**Order — consumers first, libraries last (reverses the old +"leaf-first" idea, which was impossible):** a library's headers can +only be deleted once no other package's headers still textually include +them, and during the transition the downstream packages' headers do +exactly that. So consolidation walks the dependency chain from the top: +1. **C-1 streamr-libstreamrproxyclient** ✅ (this PR) — the internal + header `LibProxyClientApi.hpp` merged into `streamrproxyclient.cpp`, + which now imports the four sibling modules; the public C header is + untouched. The Android/NDK version gate became a hard configure + error (the textual fall-back no longer exists). Verified: Release + build + 15/15 tests, standalone package build, package lint green + (two bugprone-exception-escape suppressions — the checker cannot see + through imported module interfaces; known pattern). +2. C-2 streamr-trackerless-network +3. C-3 streamr-dht (the largest) +4. C-4 streamr-proto-rpc +5. C-5 streamr-utils +6. C-6 streamr-logger +7. C-7 streamr-json +8. C-8 streamr-eventemitter (+ final bench.sh metrics and memo closure) + +One package per PR, `bench.sh` measured at the dht step and at the end +(the 2.4 lesson: measure before generalizing); headers deleted +per-package — the compiler itself enforces that nothing still includes +them. ### Interim posture (adopted now) The façade stage is COMPLETE and delivers: uniform `import streamr.` diff --git a/cmake/StreamrModules.cmake b/cmake/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/cmake/StreamrModules.cmake +++ b/cmake/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-dht/StreamrModules.cmake b/packages/streamr-dht/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-dht/StreamrModules.cmake +++ b/packages/streamr-dht/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-eventemitter/StreamrModules.cmake b/packages/streamr-eventemitter/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-eventemitter/StreamrModules.cmake +++ b/packages/streamr-eventemitter/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-json/StreamrModules.cmake b/packages/streamr-json/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-json/StreamrModules.cmake +++ b/packages/streamr-json/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-libstreamrproxyclient/CMakeLists.txt b/packages/streamr-libstreamrproxyclient/CMakeLists.txt index 464906e4..87e3d61c 100644 --- a/packages/streamr-libstreamrproxyclient/CMakeLists.txt +++ b/packages/streamr-libstreamrproxyclient/CMakeLists.txt @@ -27,9 +27,18 @@ set (IOS_SWIFT_PACKAGE_VERSION ${STREAMRPROXYCLIENT_VERSION}) set (ANDROID_LIBRARY_VERSION ${STREAMRPROXYCLIENT_VERSION}) project(streamr-streamrproxyclient CXX) + +# C++ modules support (guards, policies, helpers) — must come after +# project() because it inspects the compiler id. +include(${CMAKE_CURRENT_SOURCE_DIR}/StreamrModules.cmake) + add_library(streamrproxyclient SHARED src/streamrproxyclient.cpp ) +# CONSOLIDATED: the implementation imports the sibling streamr modules +# (the former internal header was merged into the .cpp), so the source +# needs module dependency scanning. +streamr_enable_imports(streamrproxyclient) set_target_properties(streamrproxyclient PROPERTIES VERSION ${SHAREDLIB_VERSION} SOVERSION ${SHAREDLIB_SOVERSION}) find_package(streamr-trackerless-network CONFIG REQUIRED) diff --git a/packages/streamr-libstreamrproxyclient/StreamrModules.cmake b/packages/streamr-libstreamrproxyclient/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-libstreamrproxyclient/StreamrModules.cmake +++ b/packages/streamr-libstreamrproxyclient/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-libstreamrproxyclient/src/LibProxyClientApi.hpp b/packages/streamr-libstreamrproxyclient/src/LibProxyClientApi.hpp deleted file mode 100644 index 357128bb..00000000 --- a/packages/streamr-libstreamrproxyclient/src/LibProxyClientApi.hpp +++ /dev/null @@ -1,674 +0,0 @@ -#ifndef LIB_PROXY_CLIENT_API_HPP -#define LIB_PROXY_CLIENT_API_HPP - -#include -#include // NOLINT -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include "streamr-dht/Identifiers.hpp" -#include "streamr-dht/connection/ConnectionManager.hpp" -#include "streamr-dht/connection/ConnectorFacade.hpp" -#include "streamr-dht/helpers/Connectivity.hpp" -#include "streamr-dht/transport/FakeTransport.hpp" -#include "streamr-logger/SLogger.hpp" -#include "streamr-trackerless-network/logic/proxy/ProxyClient.hpp" -#include "streamr-utils/BinaryUtils.hpp" -#include "streamr-utils/EthereumAddress.hpp" -#include "streamr-utils/SigningUtils.hpp" -#include "streamr-utils/StreamPartID.hpp" -#include "streamrproxyclient.h" -namespace streamr::libstreamrproxyclient { - -using ::dht::ConnectivityMethod; -using ::dht::ConnectivityResponse; -using ::dht::NodeType; -using ::dht::PeerDescriptor; -using streamr::dht::DhtAddress; -using streamr::dht::Identifiers; -using streamr::dht::connection::ConnectionManager; -using streamr::dht::connection::ConnectionManagerOptions; -using streamr::dht::connection::DefaultConnectorFacade; -using streamr::dht::connection::DefaultConnectorFacadeOptions; -using streamr::dht::helpers::Connectivity; -using streamr::dht::transport::FakeEnvironment; -using streamr::dht::transport::FakeTransport; -using streamr::logger::SLogger; -using streamr::trackerlessnetwork::proxy::ConnectingToProxyError; -using streamr::trackerlessnetwork::proxy::ProxyClient; -using streamr::trackerlessnetwork::proxy::ProxyClientOptions; -using streamr::utils::BinaryUtils; -using streamr::utils::EthereumAddress; -using streamr::utils::SigningUtils; -using streamr::utils::StreamPartID; -using streamr::utils::StreamPartIDUtils; -using streamr::utils::toEthereumAddress; - -class ProxyCpp { -private: - std::string ethereumAddressString; - std::string websocketUrlString; - Proxy proxy; - -public: - ProxyCpp( - const std::string& ethereumAddress, const std::string& websocketUrl) { - this->ethereumAddressString = ethereumAddress; - this->websocketUrlString = websocketUrl; - this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); - this->proxy.websocketUrl = this->websocketUrlString.c_str(); - } - ProxyCpp(const ProxyCpp& other) { - this->ethereumAddressString = other.ethereumAddressString; - this->websocketUrlString = other.websocketUrlString; - this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); - this->proxy.websocketUrl = this->websocketUrlString.c_str(); - } - - ProxyCpp(ProxyCpp&& other) noexcept { - this->ethereumAddressString = std::move(other.ethereumAddressString); - this->websocketUrlString = std::move(other.websocketUrlString); - this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); - this->proxy.websocketUrl = this->websocketUrlString.c_str(); - } - - ProxyCpp& operator=(const ProxyCpp& other) { - if (this == &other) { - return *this; - } - this->ethereumAddressString = other.ethereumAddressString; - this->websocketUrlString = other.websocketUrlString; - this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); - this->proxy.websocketUrl = this->websocketUrlString.c_str(); - return *this; - } - - ProxyCpp& operator=(ProxyCpp&& other) noexcept { - if (this == &other) { - return *this; - } - this->ethereumAddressString = std::move(other.ethereumAddressString); - this->websocketUrlString = std::move(other.websocketUrlString); - this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); - this->proxy.websocketUrl = this->websocketUrlString.c_str(); - return *this; - } - [[nodiscard]] const Proxy* getProxy() const { return &this->proxy; } -}; - -class ErrorCpp { -private: - std::string messageString; - std::string codeString; - std::optional proxyCpp; - - Error error; - -public: - ErrorCpp( - const std::string& message, // NOLINT - const std::string& code, - const std::optional& proxyCpp) { - this->proxyCpp = proxyCpp; - - this->messageString = std::string(message); - this->codeString = std::string(code); - this->error.message = this->messageString.c_str(); - this->error.code = this->codeString.c_str(); - this->error.proxy = this->proxyCpp.has_value() - ? this->proxyCpp.value().getProxy() - : nullptr; - } - ErrorCpp(const ErrorCpp& other) { - this->messageString = other.messageString; - this->codeString = other.codeString; - this->proxyCpp = other.proxyCpp; - this->error.message = this->messageString.c_str(); - this->error.code = this->codeString.c_str(); - this->error.proxy = - this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; - } - ErrorCpp(ErrorCpp&& other) noexcept { - this->messageString = std::move(other.messageString); - this->codeString = std::move(other.codeString); - this->proxyCpp = std::move(other.proxyCpp); - this->error.message = this->messageString.c_str(); - this->error.code = this->codeString.c_str(); - this->error.proxy = - this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; - } - ErrorCpp& operator=(const ErrorCpp& other) { - if (this == &other) { - return *this; - } - this->messageString = other.messageString; - this->codeString = other.codeString; - this->proxyCpp = other.proxyCpp; - this->error.message = this->messageString.c_str(); - this->error.code = this->codeString.c_str(); - this->error.proxy = - this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; - return *this; - } - - ErrorCpp& operator=(ErrorCpp&& other) noexcept { - if (this == &other) { - return *this; - } - this->messageString = std::move(other.messageString); - this->codeString = std::move(other.codeString); - this->proxyCpp = std::move(other.proxyCpp); - this->error.message = this->messageString.c_str(); - this->error.code = this->codeString.c_str(); - this->error.proxy = - this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; - return *this; - } - [[nodiscard]] const Error* getError() const { return &this->error; } -}; - -class ProxyResultCpp { -private: - std::vector errorsCppVector; - std::vector successfulCppVector; - std::vector errorsVector; - std::vector successfulVector; - ProxyResult proxyResult; - -public: - ProxyResultCpp( - const std::vector& errorsCpp, - const std::vector& successfulCpp) { - this->errorsCppVector = errorsCpp; - this->successfulCppVector = successfulCpp; - - for (const auto& errorCpp : this->errorsCppVector) { - this->errorsVector.push_back(*errorCpp.getError()); - } - for (const auto& proxyCpp : this->successfulCppVector) { - this->successfulVector.push_back(*proxyCpp.getProxy()); - } - - this->proxyResult.errors = this->errorsVector.data(); - this->proxyResult.successful = this->successfulVector.data(); - this->proxyResult.numErrors = this->errorsVector.size(); - this->proxyResult.numSuccessful = this->successfulVector.size(); - } - [[nodiscard]] const ProxyResult* getProxyResult() const { - return &this->proxyResult; - } -}; - -class LibProxyClientApi { -private: - class ProxyClientWrapper { - private: - uint64_t handle; - std::shared_ptr - proxyClient; - std::shared_ptr fakeTransport; - std::shared_ptr connectionManager; - std::atomic sequenceNumber = 1; - - public: - ProxyClientWrapper( - uint64_t handle, - std::shared_ptr - proxyClient, - std::shared_ptr fakeTransport, - std::shared_ptr connectionManager) - : handle(handle), - proxyClient(std::move(proxyClient)), - fakeTransport(std::move(fakeTransport)), - connectionManager(std::move(connectionManager)) {} - - ~ProxyClientWrapper() { - this->proxyClient->stop(); - this->connectionManager->stop(); - this->fakeTransport->stop(); - } - - std::shared_ptr& - getProxyClient() { - return this->proxyClient; - } - - std::shared_ptr& getConnectionManager() { - return this->connectionManager; - } - - int32_t getNextSequenceNumber() { - return this->sequenceNumber.fetch_add(1); - } - }; - - FakeEnvironment fakeEnvironment; - - std::map> proxyClients; - std::recursive_mutex proxyClientsMutex; - std::map> results; - std::recursive_mutex resultsMutex; - - class InvalidUrlException : public std::runtime_error { - public: - explicit InvalidUrlException(const std::string& message) - : std::runtime_error(message) {} - }; - - static uint64_t createRandomHandle() { return rand(); } - - static PeerDescriptor createLocalPeerDescriptor( - const std::string& ownEthereumAddress) { - PeerDescriptor peerDescriptor; - peerDescriptor.set_nodeid( - BinaryUtils::hexToBinaryString(ownEthereumAddress)); - peerDescriptor.set_type(NodeType::NODEJS); - peerDescriptor.set_publickey(""); - peerDescriptor.set_signature(""); - peerDescriptor.set_region(1); - peerDescriptor.set_ipaddress(0); - return peerDescriptor; - } - - static PeerDescriptor createProxyPeerDescriptor( - std::string_view proxyEthereumAddress, - const std::string& websocketUrl) { - PeerDescriptor proxyPeerDescriptor; - - // this will throw if the address is not valid - auto ethereumAddress = toEthereumAddress(proxyEthereumAddress); - - proxyPeerDescriptor.set_nodeid( - BinaryUtils::hexToBinaryString(ethereumAddress)); - proxyPeerDescriptor.set_type(NodeType::NODEJS); - - const auto parsedUrl = ada::parse(websocketUrl); - - if (!parsedUrl) { - throw InvalidUrlException( - "Invalid websocket URL for proxy server " + - std::string(websocketUrl)); - } - - ConnectivityMethod connectivityMethod; - connectivityMethod.set_host(parsedUrl->get_hostname()); - - if (parsedUrl->has_port()) { - connectivityMethod.set_port( - std::stoul(std::string(parsedUrl->get_port()))); - } else { - if (parsedUrl->get_protocol() == "wss" || - parsedUrl->get_protocol() == "https") { - connectivityMethod.set_port(443); // NOLINT - } else { - connectivityMethod.set_port(80); // NOLINT - } - } - if (parsedUrl->get_protocol() == "wss" || - parsedUrl->get_protocol() == "https") { - connectivityMethod.set_tls(true); - } else { - connectivityMethod.set_tls(false); - } - - proxyPeerDescriptor.mutable_websocket()->CopyFrom(connectivityMethod); - - SLogger::trace( - "Proxy peer descriptor created: " + - proxyPeerDescriptor.DebugString()); - return proxyPeerDescriptor; - } - - static std::shared_ptr createConnectionManager( - const DefaultConnectorFacadeOptions& opts) { - SLogger::trace("Calling connection manager constructor"); - - ConnectionManagerOptions connectionManagerOptions{ - .createConnectorFacade = - [opts]() -> std::shared_ptr { - return std::make_shared(opts); - }}; - return std::make_shared( - std::move(connectionManagerOptions)); - } - - const ProxyResult* addResult( - const std::vector& errorsCpp, - const std::vector& successfulCpp) { - std::shared_ptr result = - std::make_shared(errorsCpp, successfulCpp); - std::scoped_lock lock(this->resultsMutex); - this->results[result->getProxyResult()] = result; - return this->results[result->getProxyResult()]->getProxyResult(); - } - -public: - void proxyClientResultDelete(const ProxyResult* proxyResult) { - std::scoped_lock lock(this->resultsMutex); - this->results.erase(proxyResult); - } - - uint64_t proxyClientNew( - const ProxyResult** proxyResult, - const char* ownEthereumAddress, // NOLINT - const char* streamPartId) { - std::shared_ptr parsedEthereumAddress; - try { - parsedEthereumAddress = std::make_shared( - toEthereumAddress(ownEthereumAddress)); - } catch (const std::runtime_error& e) { - SLogger::error("Error in proxyClientNew: " + std::string(e.what())); - - const auto* result = addResult( - {ErrorCpp( - e.what(), ERROR_INVALID_ETHEREUM_ADDRESS, std::nullopt)}, - {}); - *proxyResult = result; - return 0; - } - std::shared_ptr parsedStreamPartID; - try { - parsedStreamPartID = std::make_shared( - StreamPartIDUtils::parse(streamPartId)); - } catch (const std::invalid_argument& e) { - SLogger::error("Error in proxyClientNew: " + std::string(e.what())); - std::string message = - "Error in proxyClientNew: " + std::string(e.what()); - *proxyResult = addResult( - {ErrorCpp(message, ERROR_INVALID_STREAM_PART_ID, std::nullopt)}, - {}); - return 0; - } - - auto localPeerDescriptor = - createLocalPeerDescriptor(*parsedEthereumAddress); - - auto fakeTransport = - this->fakeEnvironment.createTransport(localPeerDescriptor); - - auto connectionManager = createConnectionManager( - DefaultConnectorFacadeOptions{ - .transport = *fakeTransport, - .createLocalPeerDescriptor = - [localPeerDescriptor]( - const ConnectivityResponse& /* response */) - -> PeerDescriptor { return localPeerDescriptor; }}); - SLogger::trace("Connection manager created, starting it"); - connectionManager->start(); - uint64_t handle = createRandomHandle(); - SLogger::trace("Creating proxy client"); - auto proxyClient = std::make_shared( - handle, - std::make_shared(ProxyClientOptions{ - .transport = *connectionManager, - .localPeerDescriptor = - createLocalPeerDescriptor(*parsedEthereumAddress), - .streamPartId = *parsedStreamPartID, - .connectionLocker = *connectionManager}), - fakeTransport, - connectionManager); - SLogger::trace("Proxy client created, starting it"); - proxyClient->getProxyClient()->start(); - SLogger::trace("Proxy client started"); - - std::scoped_lock lock(this->proxyClientsMutex); - this->proxyClients[handle] = proxyClient; - *proxyResult = addResult({}, {}); - return handle; - } - - void proxyClientDelete( - const ProxyResult** proxyResult, uint64_t clientHandle) { - std::scoped_lock lock(this->proxyClientsMutex); - this->proxyClients.erase(clientHandle); - SLogger::trace("Proxy client erased"); - *proxyResult = addResult({}, {}); - } - - uint64_t proxyClientConnect( - const ProxyResult** proxyResult, - uint64_t clientHandle, - const Proxy* proxies, - size_t numProxies) { - if (numProxies <= 0) { - SLogger::error("No proxies defined, returning error"); - *proxyResult = addResult( - {ErrorCpp( - "No proxies defined", - ERROR_NO_PROXIES_DEFINED, - std::nullopt)}, - {}); - return 0; - } - - std::shared_ptr proxyClient; - { - std::scoped_lock lock(this->proxyClientsMutex); - auto proxyClientIterator = this->proxyClients.find(clientHandle); - - if (proxyClientIterator == this->proxyClients.end()) { - *proxyResult = addResult( - {ErrorCpp( - "Proxy client not found with handle " + - std::to_string(clientHandle), - ERROR_PROXY_CLIENT_NOT_FOUND, - std::nullopt)}, - {}); - return 0; - } - proxyClient = proxyClientIterator->second; - } - - std::vector proxyPeerDescriptors; - for (size_t i = 0; i < numProxies; i++) { - const auto& proxy = proxies[i]; - try { - proxyPeerDescriptors.push_back(createProxyPeerDescriptor( - proxy.ethereumAddress, proxy.websocketUrl)); - } catch (const InvalidUrlException& e) { - *proxyResult = addResult( - {ErrorCpp( - e.what(), - ERROR_INVALID_PROXY_URL, - ProxyCpp(proxy.ethereumAddress, proxy.websocketUrl))}, - {}); - return 0; - } catch (const std::runtime_error& e) { - *proxyResult = addResult( - {ErrorCpp( - e.what(), - ERROR_INVALID_ETHEREUM_ADDRESS, - ProxyCpp(proxy.ethereumAddress, proxy.websocketUrl))}, - {}); - return 0; - } - } - - auto [connectionErrors, successfullyConnected] = - proxyClient->getProxyClient()->setProxies( - proxyPeerDescriptors, - ProxyDirection::PUBLISH, - proxyClient->getProxyClient()->getLocalEthereumAddress()); - - std::vector errorsCpp; - std::vector successfullyConnectedCpp; - - for (const auto& error : connectionErrors) { - try { - if (error.getOriginalException()) { - std::rethrow_exception(error.getOriginalException()); - } else { - throw std::runtime_error("No original exception"); - } - } catch (const std::exception& e) { - errorsCpp.emplace_back( - std::move(ErrorCpp( - e.what(), - ERROR_PROXY_CONNECTION_FAILED, - ProxyCpp( - "0x" + - Identifiers::getNodeIdFromPeerDescriptor( - error.getPeerDescriptor()), - Connectivity::connectivityMethodToWebsocketUrl( - error.getPeerDescriptor().websocket()))))); - } - } - - for (const auto& proxy : successfullyConnected) { - successfullyConnectedCpp.emplace_back( - std::move(ProxyCpp( - "0x" + Identifiers::getNodeIdFromPeerDescriptor(proxy), - Connectivity::connectivityMethodToWebsocketUrl( - proxy.websocket())))); - } - - *proxyResult = addResult(errorsCpp, successfullyConnectedCpp); - return successfullyConnected.size(); - } - - static StreamMessage createStreamMessage( - const std::shared_ptr& proxyClient, - const char* content, - uint64_t contentLength, - const char* ethereumPrivateKey) { - StreamMessage message; - std::string contentString(content, contentLength); - auto* contentMessage = message.mutable_contentmessage(); - contentMessage->set_content(contentString); - contentMessage->set_contenttype(ContentType::BINARY); - contentMessage->set_encryptiontype(EncryptionType::NONE); - - MessageID messageId; - std::string publisherIdHex = - proxyClient->getProxyClient()->getLocalEthereumAddress(); - - if (!publisherIdHex.starts_with("0x")) { - publisherIdHex = "0x" + publisherIdHex; - } - messageId.set_publisherid( - BinaryUtils::hexToBinaryString(publisherIdHex)); - messageId.set_messagechainid("1"); - messageId.set_timestamp( - std::chrono::duration_cast( - std::chrono::system_clock::now().time_since_epoch()) - .count()); - messageId.set_sequencenumber(proxyClient->getNextSequenceNumber()); - - auto streamPartID = proxyClient->getProxyClient()->getStreamPartID(); - messageId.set_streampartition( - static_cast( - StreamPartIDUtils::getStreamPartition(streamPartID).value())); - messageId.set_streamid(StreamPartIDUtils::getStreamID(streamPartID)); - - message.mutable_messageid()->CopyFrom(messageId); - - if (ethereumPrivateKey) { - std::string signaturePayload = messageId.streamid() + - std::to_string(messageId.streampartition()) + - std::to_string(messageId.timestamp()) + - std::to_string(messageId.sequencenumber()) + publisherIdHex + - messageId.messagechainid() + contentString; - - SLogger::trace("Signature payload: "); - SLogger::trace(""); - SLogger::trace( - "messageId.streamid(): \"" + messageId.streamid() + "\""); - SLogger::trace( - "messageId.streampartition(): \"" + - std::to_string(messageId.streampartition()) + "\""); - SLogger::trace( - "messageId.timestamp(): \"" + - std::to_string(messageId.timestamp()) + "\""); - SLogger::trace( - "messageId.sequencenumber(): \"" + - std::to_string(messageId.sequencenumber()) + "\""); - SLogger::trace("publisheridHex(): \"" + publisherIdHex + "\""); - SLogger::trace( - "messageId.messagechainid(): \"" + messageId.messagechainid() + - "\""); - SLogger::trace("contentString: \"" + contentString + "\""); - SLogger::trace(""); - - SLogger::trace( - "Signing payload as hex string: \"" + - BinaryUtils::binaryStringToHex(signaturePayload) + "\""); - auto signature = SigningUtils::createSignature( - signaturePayload, ethereumPrivateKey); - - message.set_signature(signature); - message.set_signaturetype(SignatureType::SECP256K1); - SLogger::trace( - "Signature in hex: \"" + - BinaryUtils::binaryStringToHex(signature) + "\""); - } - return message; - } - uint64_t proxyClientPublish( - const ProxyResult** proxyResult, - uint64_t clientHandle, - const char* content, - uint64_t contentLength, - const char* ethereumPrivateKey) { - std::shared_ptr proxyClient; - { - std::scoped_lock lock(this->proxyClientsMutex); - auto proxyClientIterator = this->proxyClients.find(clientHandle); - if (proxyClientIterator == this->proxyClients.end()) { - *proxyResult = addResult( - {ErrorCpp( - "Proxy client not found", - ERROR_PROXY_CLIENT_NOT_FOUND, - std::nullopt)}, - {}); - return 0; - } - proxyClient = proxyClientIterator->second; - } - std::vector errorsCpp; - std::vector successfullySentCpp; - try { - auto message = createStreamMessage( - proxyClient, content, contentLength, ethereumPrivateKey); - auto result = proxyClient->getProxyClient()->broadcast(message); - for (const auto& failedPeer : result.first) { - errorsCpp.emplace_back( - std::move(ErrorCpp( - "Failed to send message to proxy", - ERROR_PROXY_BROADCAST_FAILED, - ProxyCpp( - "0x" + - Identifiers::getNodeIdFromPeerDescriptor( - failedPeer), - Connectivity::connectivityMethodToWebsocketUrl( - failedPeer.websocket()))))); - } - for (const auto& proxy : result.second) { - successfullySentCpp.emplace_back( - std::move(ProxyCpp( - "0x" + Identifiers::getNodeIdFromPeerDescriptor(proxy), - Connectivity::connectivityMethodToWebsocketUrl( - proxy.websocket())))); - } - *proxyResult = addResult(errorsCpp, successfullySentCpp); - return successfullySentCpp.size(); - } catch (const std::exception& e) { - SLogger::error( - "Exception in proxyClientPublish: " + std::string(e.what())); - errorsCpp.emplace_back( - std::move(ErrorCpp( - e.what(), ERROR_PROXY_BROADCAST_FAILED, std::nullopt))); - *proxyResult = addResult(errorsCpp, {}); - return 0; - } - } -}; - -} // namespace streamr::libstreamrproxyclient - -#endif \ No newline at end of file diff --git a/packages/streamr-libstreamrproxyclient/src/streamrproxyclient.cpp b/packages/streamr-libstreamrproxyclient/src/streamrproxyclient.cpp index 06c178ac..f0b0f8c7 100644 --- a/packages/streamr-libstreamrproxyclient/src/streamrproxyclient.cpp +++ b/packages/streamr-libstreamrproxyclient/src/streamrproxyclient.cpp @@ -1,6 +1,680 @@ +// Implementation of the public C API (streamrproxyclient.h). +// +// CONSOLIDATED (MODERNIZATION.md Phase 2.6 consolidation): the former +// internal header LibProxyClientApi.hpp was merged into this file and +// the sibling streamr packages are consumed as C++ modules (import) +// instead of textual includes. Only third-party libraries, the standard +// library and the public C API header remain textual includes. #include "streamrproxyclient.h" +#include +#include // NOLINT +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include #include -#include "LibProxyClientApi.hpp" + +import streamr.dht; +import streamr.logger; +import streamr.trackerlessnetwork; +import streamr.utils; + +namespace streamr::libstreamrproxyclient { + +using ::dht::ConnectivityMethod; +using ::dht::ConnectivityResponse; +using ::dht::NodeType; +using ::dht::PeerDescriptor; +using streamr::dht::Identifiers; +using streamr::dht::connection::ConnectionManager; +using streamr::dht::connection::ConnectionManagerOptions; +using streamr::dht::connection::DefaultConnectorFacade; +using streamr::dht::connection::DefaultConnectorFacadeOptions; +using streamr::dht::helpers::Connectivity; +using streamr::dht::transport::FakeEnvironment; +using streamr::dht::transport::FakeTransport; +using streamr::logger::SLogger; +using streamr::trackerlessnetwork::proxy::ProxyClient; +using streamr::trackerlessnetwork::proxy::ProxyClientOptions; +using streamr::utils::BinaryUtils; +using streamr::utils::EthereumAddress; +using streamr::utils::SigningUtils; +using streamr::utils::StreamPartID; +using streamr::utils::StreamPartIDUtils; +using streamr::utils::toEthereumAddress; + +class ProxyCpp { +private: + std::string ethereumAddressString; + std::string websocketUrlString; + Proxy proxy; + +public: + ProxyCpp( + const std::string& ethereumAddress, const std::string& websocketUrl) { + this->ethereumAddressString = ethereumAddress; + this->websocketUrlString = websocketUrl; + this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); + this->proxy.websocketUrl = this->websocketUrlString.c_str(); + } + ProxyCpp(const ProxyCpp& other) { + this->ethereumAddressString = other.ethereumAddressString; + this->websocketUrlString = other.websocketUrlString; + this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); + this->proxy.websocketUrl = this->websocketUrlString.c_str(); + } + + ProxyCpp(ProxyCpp&& other) noexcept { + this->ethereumAddressString = std::move(other.ethereumAddressString); + this->websocketUrlString = std::move(other.websocketUrlString); + this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); + this->proxy.websocketUrl = this->websocketUrlString.c_str(); + } + + ProxyCpp& operator=(const ProxyCpp& other) { + if (this == &other) { + return *this; + } + this->ethereumAddressString = other.ethereumAddressString; + this->websocketUrlString = other.websocketUrlString; + this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); + this->proxy.websocketUrl = this->websocketUrlString.c_str(); + return *this; + } + + ProxyCpp& operator=(ProxyCpp&& other) noexcept { + if (this == &other) { + return *this; + } + this->ethereumAddressString = std::move(other.ethereumAddressString); + this->websocketUrlString = std::move(other.websocketUrlString); + this->proxy.ethereumAddress = this->ethereumAddressString.c_str(); + this->proxy.websocketUrl = this->websocketUrlString.c_str(); + return *this; + } + [[nodiscard]] const Proxy* getProxy() const { return &this->proxy; } +}; + +class ErrorCpp { +private: + std::string messageString; + std::string codeString; + std::optional proxyCpp; + + Error error; + +public: + ErrorCpp( + const std::string& message, // NOLINT + const std::string& code, + const std::optional& proxyCpp) { + this->proxyCpp = proxyCpp; + + this->messageString = std::string(message); + this->codeString = std::string(code); + this->error.message = this->messageString.c_str(); + this->error.code = this->codeString.c_str(); + this->error.proxy = this->proxyCpp.has_value() + ? this->proxyCpp.value().getProxy() + : nullptr; + } + ErrorCpp(const ErrorCpp& other) { + this->messageString = other.messageString; + this->codeString = other.codeString; + this->proxyCpp = other.proxyCpp; + this->error.message = this->messageString.c_str(); + this->error.code = this->codeString.c_str(); + this->error.proxy = + this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; + } + ErrorCpp(ErrorCpp&& other) noexcept { + this->messageString = std::move(other.messageString); + this->codeString = std::move(other.codeString); + this->proxyCpp = std::move(other.proxyCpp); + this->error.message = this->messageString.c_str(); + this->error.code = this->codeString.c_str(); + this->error.proxy = + this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; + } + ErrorCpp& operator=(const ErrorCpp& other) { + if (this == &other) { + return *this; + } + this->messageString = other.messageString; + this->codeString = other.codeString; + this->proxyCpp = other.proxyCpp; + this->error.message = this->messageString.c_str(); + this->error.code = this->codeString.c_str(); + this->error.proxy = + this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; + return *this; + } + + ErrorCpp& operator=(ErrorCpp&& other) noexcept { + if (this == &other) { + return *this; + } + this->messageString = std::move(other.messageString); + this->codeString = std::move(other.codeString); + this->proxyCpp = std::move(other.proxyCpp); + this->error.message = this->messageString.c_str(); + this->error.code = this->codeString.c_str(); + this->error.proxy = + this->proxyCpp ? this->proxyCpp->getProxy() : nullptr; + return *this; + } + [[nodiscard]] const Error* getError() const { return &this->error; } +}; + +class ProxyResultCpp { +private: + std::vector errorsCppVector; + std::vector successfulCppVector; + std::vector errorsVector; + std::vector successfulVector; + ProxyResult proxyResult; + +public: + ProxyResultCpp( + const std::vector& errorsCpp, + const std::vector& successfulCpp) { + this->errorsCppVector = errorsCpp; + this->successfulCppVector = successfulCpp; + + for (const auto& errorCpp : this->errorsCppVector) { + this->errorsVector.push_back(*errorCpp.getError()); + } + for (const auto& proxyCpp : this->successfulCppVector) { + this->successfulVector.push_back(*proxyCpp.getProxy()); + } + + this->proxyResult.errors = this->errorsVector.data(); + this->proxyResult.successful = this->successfulVector.data(); + this->proxyResult.numErrors = this->errorsVector.size(); + this->proxyResult.numSuccessful = this->successfulVector.size(); + } + [[nodiscard]] const ProxyResult* getProxyResult() const { + return &this->proxyResult; + } +}; + +class LibProxyClientApi { +private: + class ProxyClientWrapper { + private: + uint64_t handle; + std::shared_ptr + proxyClient; + std::shared_ptr fakeTransport; + std::shared_ptr connectionManager; + std::atomic sequenceNumber = 1; + + public: + ProxyClientWrapper( + uint64_t handle, + std::shared_ptr + proxyClient, + std::shared_ptr fakeTransport, + std::shared_ptr connectionManager) + : handle(handle), + proxyClient(std::move(proxyClient)), + fakeTransport(std::move(fakeTransport)), + connectionManager(std::move(connectionManager)) {} + + // NOLINTNEXTLINE(bugprone-exception-escape) + ~ProxyClientWrapper() { + // bugprone-exception-escape cannot see through the imported + // module interfaces and assumes the stop() calls may throw + // (same known pattern as in the import-using test files). + this->proxyClient->stop(); + this->connectionManager->stop(); + this->fakeTransport->stop(); + } + + std::shared_ptr& + getProxyClient() { + return this->proxyClient; + } + + std::shared_ptr& getConnectionManager() { + return this->connectionManager; + } + + int32_t getNextSequenceNumber() { + return this->sequenceNumber.fetch_add(1); + } + }; + + FakeEnvironment fakeEnvironment; + + std::map> proxyClients; + std::recursive_mutex proxyClientsMutex; + std::map> results; + std::recursive_mutex resultsMutex; + + class InvalidUrlException : public std::runtime_error { + public: + explicit InvalidUrlException(const std::string& message) + : std::runtime_error(message) {} + }; + + static uint64_t createRandomHandle() { return rand(); } + + static PeerDescriptor createLocalPeerDescriptor( + const std::string& ownEthereumAddress) { + PeerDescriptor peerDescriptor; + peerDescriptor.set_nodeid( + BinaryUtils::hexToBinaryString(ownEthereumAddress)); + peerDescriptor.set_type(NodeType::NODEJS); + peerDescriptor.set_publickey(""); + peerDescriptor.set_signature(""); + peerDescriptor.set_region(1); + peerDescriptor.set_ipaddress(0); + return peerDescriptor; + } + + static PeerDescriptor createProxyPeerDescriptor( + std::string_view proxyEthereumAddress, + const std::string& websocketUrl) { + PeerDescriptor proxyPeerDescriptor; + + // this will throw if the address is not valid + auto ethereumAddress = toEthereumAddress(proxyEthereumAddress); + + proxyPeerDescriptor.set_nodeid( + BinaryUtils::hexToBinaryString(ethereumAddress)); + proxyPeerDescriptor.set_type(NodeType::NODEJS); + + const auto parsedUrl = ada::parse(websocketUrl); + + if (!parsedUrl) { + throw InvalidUrlException( + "Invalid websocket URL for proxy server " + + std::string(websocketUrl)); + } + + ConnectivityMethod connectivityMethod; + connectivityMethod.set_host(parsedUrl->get_hostname()); + + if (parsedUrl->has_port()) { + connectivityMethod.set_port( + std::stoul(std::string(parsedUrl->get_port()))); + } else { + if (parsedUrl->get_protocol() == "wss" || + parsedUrl->get_protocol() == "https") { + connectivityMethod.set_port(443); // NOLINT + } else { + connectivityMethod.set_port(80); // NOLINT + } + } + if (parsedUrl->get_protocol() == "wss" || + parsedUrl->get_protocol() == "https") { + connectivityMethod.set_tls(true); + } else { + connectivityMethod.set_tls(false); + } + + proxyPeerDescriptor.mutable_websocket()->CopyFrom(connectivityMethod); + + SLogger::trace( + "Proxy peer descriptor created: " + + proxyPeerDescriptor.DebugString()); + return proxyPeerDescriptor; + } + + static std::shared_ptr createConnectionManager( + const DefaultConnectorFacadeOptions& opts) { + SLogger::trace("Calling connection manager constructor"); + + ConnectionManagerOptions connectionManagerOptions{ + .createConnectorFacade = + // NOLINTNEXTLINE(bugprone-exception-escape) + [opts]() -> std::shared_ptr { + return std::make_shared(opts); + }}; + return std::make_shared( + std::move(connectionManagerOptions)); + } + + const ProxyResult* addResult( + const std::vector& errorsCpp, + const std::vector& successfulCpp) { + std::shared_ptr result = + std::make_shared(errorsCpp, successfulCpp); + std::scoped_lock lock(this->resultsMutex); + this->results[result->getProxyResult()] = result; + return this->results[result->getProxyResult()]->getProxyResult(); + } + +public: + void proxyClientResultDelete(const ProxyResult* proxyResult) { + std::scoped_lock lock(this->resultsMutex); + this->results.erase(proxyResult); + } + + uint64_t proxyClientNew( + const ProxyResult** proxyResult, + const char* ownEthereumAddress, // NOLINT + const char* streamPartId) { + std::shared_ptr parsedEthereumAddress; + try { + parsedEthereumAddress = std::make_shared( + toEthereumAddress(ownEthereumAddress)); + } catch (const std::runtime_error& e) { + SLogger::error("Error in proxyClientNew: " + std::string(e.what())); + + const auto* result = addResult( + {ErrorCpp( + e.what(), ERROR_INVALID_ETHEREUM_ADDRESS, std::nullopt)}, + {}); + *proxyResult = result; + return 0; + } + std::shared_ptr parsedStreamPartID; + try { + parsedStreamPartID = std::make_shared( + StreamPartIDUtils::parse(streamPartId)); + } catch (const std::invalid_argument& e) { + SLogger::error("Error in proxyClientNew: " + std::string(e.what())); + std::string message = + "Error in proxyClientNew: " + std::string(e.what()); + *proxyResult = addResult( + {ErrorCpp(message, ERROR_INVALID_STREAM_PART_ID, std::nullopt)}, + {}); + return 0; + } + + auto localPeerDescriptor = + createLocalPeerDescriptor(*parsedEthereumAddress); + + auto fakeTransport = + this->fakeEnvironment.createTransport(localPeerDescriptor); + + auto connectionManager = createConnectionManager( + DefaultConnectorFacadeOptions{ + .transport = *fakeTransport, + .createLocalPeerDescriptor = + [localPeerDescriptor]( + const ConnectivityResponse& /* response */) + -> PeerDescriptor { return localPeerDescriptor; }}); + SLogger::trace("Connection manager created, starting it"); + connectionManager->start(); + uint64_t handle = createRandomHandle(); + SLogger::trace("Creating proxy client"); + auto proxyClient = std::make_shared( + handle, + std::make_shared(ProxyClientOptions{ + .transport = *connectionManager, + .localPeerDescriptor = + createLocalPeerDescriptor(*parsedEthereumAddress), + .streamPartId = *parsedStreamPartID, + .connectionLocker = *connectionManager}), + fakeTransport, + connectionManager); + SLogger::trace("Proxy client created, starting it"); + proxyClient->getProxyClient()->start(); + SLogger::trace("Proxy client started"); + + std::scoped_lock lock(this->proxyClientsMutex); + this->proxyClients[handle] = proxyClient; + *proxyResult = addResult({}, {}); + return handle; + } + + void proxyClientDelete( + const ProxyResult** proxyResult, uint64_t clientHandle) { + std::scoped_lock lock(this->proxyClientsMutex); + this->proxyClients.erase(clientHandle); + SLogger::trace("Proxy client erased"); + *proxyResult = addResult({}, {}); + } + + uint64_t proxyClientConnect( + const ProxyResult** proxyResult, + uint64_t clientHandle, + const Proxy* proxies, + size_t numProxies) { + if (numProxies <= 0) { + SLogger::error("No proxies defined, returning error"); + *proxyResult = addResult( + {ErrorCpp( + "No proxies defined", + ERROR_NO_PROXIES_DEFINED, + std::nullopt)}, + {}); + return 0; + } + + std::shared_ptr proxyClient; + { + std::scoped_lock lock(this->proxyClientsMutex); + auto proxyClientIterator = this->proxyClients.find(clientHandle); + + if (proxyClientIterator == this->proxyClients.end()) { + *proxyResult = addResult( + {ErrorCpp( + "Proxy client not found with handle " + + std::to_string(clientHandle), + ERROR_PROXY_CLIENT_NOT_FOUND, + std::nullopt)}, + {}); + return 0; + } + proxyClient = proxyClientIterator->second; + } + + std::vector proxyPeerDescriptors; + for (size_t i = 0; i < numProxies; i++) { + const auto& proxy = proxies[i]; + try { + proxyPeerDescriptors.push_back(createProxyPeerDescriptor( + proxy.ethereumAddress, proxy.websocketUrl)); + } catch (const InvalidUrlException& e) { + *proxyResult = addResult( + {ErrorCpp( + e.what(), + ERROR_INVALID_PROXY_URL, + ProxyCpp(proxy.ethereumAddress, proxy.websocketUrl))}, + {}); + return 0; + } catch (const std::runtime_error& e) { + *proxyResult = addResult( + {ErrorCpp( + e.what(), + ERROR_INVALID_ETHEREUM_ADDRESS, + ProxyCpp(proxy.ethereumAddress, proxy.websocketUrl))}, + {}); + return 0; + } + } + + auto [connectionErrors, successfullyConnected] = + proxyClient->getProxyClient()->setProxies( + proxyPeerDescriptors, + ProxyDirection::PUBLISH, + proxyClient->getProxyClient()->getLocalEthereumAddress()); + + std::vector errorsCpp; + std::vector successfullyConnectedCpp; + + for (const auto& error : connectionErrors) { + try { + if (error.getOriginalException()) { + std::rethrow_exception(error.getOriginalException()); + } else { + throw std::runtime_error("No original exception"); + } + } catch (const std::exception& e) { + errorsCpp.emplace_back( + std::move(ErrorCpp( + e.what(), + ERROR_PROXY_CONNECTION_FAILED, + ProxyCpp( + "0x" + + Identifiers::getNodeIdFromPeerDescriptor( + error.getPeerDescriptor()), + Connectivity::connectivityMethodToWebsocketUrl( + error.getPeerDescriptor().websocket()))))); + } + } + + for (const auto& proxy : successfullyConnected) { + successfullyConnectedCpp.emplace_back( + std::move(ProxyCpp( + "0x" + Identifiers::getNodeIdFromPeerDescriptor(proxy), + Connectivity::connectivityMethodToWebsocketUrl( + proxy.websocket())))); + } + + *proxyResult = addResult(errorsCpp, successfullyConnectedCpp); + return successfullyConnected.size(); + } + + static StreamMessage createStreamMessage( + const std::shared_ptr& proxyClient, + const char* content, + uint64_t contentLength, + const char* ethereumPrivateKey) { + StreamMessage message; + std::string contentString(content, contentLength); + auto* contentMessage = message.mutable_contentmessage(); + contentMessage->set_content(contentString); + contentMessage->set_contenttype(ContentType::BINARY); + contentMessage->set_encryptiontype(EncryptionType::NONE); + + MessageID messageId; + std::string publisherIdHex = + proxyClient->getProxyClient()->getLocalEthereumAddress(); + + if (!publisherIdHex.starts_with("0x")) { + publisherIdHex = "0x" + publisherIdHex; + } + messageId.set_publisherid( + BinaryUtils::hexToBinaryString(publisherIdHex)); + messageId.set_messagechainid("1"); + messageId.set_timestamp( + std::chrono::duration_cast( + std::chrono::system_clock::now().time_since_epoch()) + .count()); + messageId.set_sequencenumber(proxyClient->getNextSequenceNumber()); + + auto streamPartID = proxyClient->getProxyClient()->getStreamPartID(); + messageId.set_streampartition( + static_cast( + StreamPartIDUtils::getStreamPartition(streamPartID).value())); + messageId.set_streamid(StreamPartIDUtils::getStreamID(streamPartID)); + + message.mutable_messageid()->CopyFrom(messageId); + + if (ethereumPrivateKey) { + std::string signaturePayload = messageId.streamid() + + std::to_string(messageId.streampartition()) + + std::to_string(messageId.timestamp()) + + std::to_string(messageId.sequencenumber()) + publisherIdHex + + messageId.messagechainid() + contentString; + + SLogger::trace("Signature payload: "); + SLogger::trace(""); + SLogger::trace( + "messageId.streamid(): \"" + messageId.streamid() + "\""); + SLogger::trace( + "messageId.streampartition(): \"" + + std::to_string(messageId.streampartition()) + "\""); + SLogger::trace( + "messageId.timestamp(): \"" + + std::to_string(messageId.timestamp()) + "\""); + SLogger::trace( + "messageId.sequencenumber(): \"" + + std::to_string(messageId.sequencenumber()) + "\""); + SLogger::trace("publisheridHex(): \"" + publisherIdHex + "\""); + SLogger::trace( + "messageId.messagechainid(): \"" + messageId.messagechainid() + + "\""); + SLogger::trace("contentString: \"" + contentString + "\""); + SLogger::trace(""); + + SLogger::trace( + "Signing payload as hex string: \"" + + BinaryUtils::binaryStringToHex(signaturePayload) + "\""); + auto signature = SigningUtils::createSignature( + signaturePayload, ethereumPrivateKey); + + message.set_signature(signature); + message.set_signaturetype(SignatureType::SECP256K1); + SLogger::trace( + "Signature in hex: \"" + + BinaryUtils::binaryStringToHex(signature) + "\""); + } + return message; + } + uint64_t proxyClientPublish( + const ProxyResult** proxyResult, + uint64_t clientHandle, + const char* content, + uint64_t contentLength, + const char* ethereumPrivateKey) { + std::shared_ptr proxyClient; + { + std::scoped_lock lock(this->proxyClientsMutex); + auto proxyClientIterator = this->proxyClients.find(clientHandle); + if (proxyClientIterator == this->proxyClients.end()) { + *proxyResult = addResult( + {ErrorCpp( + "Proxy client not found", + ERROR_PROXY_CLIENT_NOT_FOUND, + std::nullopt)}, + {}); + return 0; + } + proxyClient = proxyClientIterator->second; + } + std::vector errorsCpp; + std::vector successfullySentCpp; + try { + auto message = createStreamMessage( + proxyClient, content, contentLength, ethereumPrivateKey); + auto result = proxyClient->getProxyClient()->broadcast(message); + for (const auto& failedPeer : result.first) { + errorsCpp.emplace_back( + std::move(ErrorCpp( + "Failed to send message to proxy", + ERROR_PROXY_BROADCAST_FAILED, + ProxyCpp( + "0x" + + Identifiers::getNodeIdFromPeerDescriptor( + failedPeer), + Connectivity::connectivityMethodToWebsocketUrl( + failedPeer.websocket()))))); + } + for (const auto& proxy : result.second) { + successfullySentCpp.emplace_back( + std::move(ProxyCpp( + "0x" + Identifiers::getNodeIdFromPeerDescriptor(proxy), + Connectivity::connectivityMethodToWebsocketUrl( + proxy.websocket())))); + } + *proxyResult = addResult(errorsCpp, successfullySentCpp); + return successfullySentCpp.size(); + } catch (const std::exception& e) { + SLogger::error( + "Exception in proxyClientPublish: " + std::string(e.what())); + errorsCpp.emplace_back( + std::move(ErrorCpp( + e.what(), ERROR_PROXY_BROADCAST_FAILED, std::nullopt))); + *proxyResult = addResult(errorsCpp, {}); + return 0; + } + } +}; + +} // namespace streamr::libstreamrproxyclient using streamr::libstreamrproxyclient::LibProxyClientApi; diff --git a/packages/streamr-logger/StreamrModules.cmake b/packages/streamr-logger/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-logger/StreamrModules.cmake +++ b/packages/streamr-logger/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-proto-rpc/StreamrModules.cmake b/packages/streamr-proto-rpc/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-proto-rpc/StreamrModules.cmake +++ b/packages/streamr-proto-rpc/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-trackerless-network/StreamrModules.cmake b/packages/streamr-trackerless-network/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-trackerless-network/StreamrModules.cmake +++ b/packages/streamr-trackerless-network/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) diff --git a/packages/streamr-utils/StreamrModules.cmake b/packages/streamr-utils/StreamrModules.cmake index 3c6eb29e..2bb5f1e6 100644 --- a/packages/streamr-utils/StreamrModules.cmake +++ b/packages/streamr-utils/StreamrModules.cmake @@ -10,22 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang. The façade modules build correctly -# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# Android builds use the NDK's clang. The modules build correctly with +# NDK r27+ (clang 18) — the failure previously attributed to compiler # immaturity was a -pthread BMI configuration mismatch, fixed in the -# helpers below. Keep a version floor at the oldest verified NDK clang -# (18 = r27): with an older NDK, Android consumes the ordinary headers -# instead (they remain the source of truth during the façade stage), the -# module units are skipped, and import-using test/example targets are not -# built (STREAMR_MODULES_SUPPORTED guards them). +# helpers below. Older NDKs are a hard error: the codebase is being +# consolidated into C++ modules (MODERNIZATION.md Phase 2.6), so the +# former fall-back of building textually from the internal headers no +# longer exists. if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() - set(STREAMR_MODULES_SUPPORTED OFF) - message(STATUS - "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 18 (use NDK r27+ for modules on Android)") + message(FATAL_ERROR + "NDK clang ${CMAKE_CXX_COMPILER_VERSION} is too old: building " + "this codebase requires C++ modules support (NDK r27+, " + "clang >= 18). The pre-consolidation textual fall-back no " + "longer exists (see MODERNIZATION.md Phase 2.6).") endif() else() set(STREAMR_MODULES_SUPPORTED ON) From b0ad349f19951869c30ba6cbe63e03ef379cd04f Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 11:32:53 +0300 Subject: [PATCH 2/2] Fix protoc plugin guard for iOS package builds The NOT CMAKE_CROSSCOMPILING guard (added when Android modules were enabled) is not enough on iOS: the iOS package builds keep the host CMAKE_SYSTEM_NAME by design (Homebrew compiler, only the SDK/sysroot is swapped), so CMake does not consider them cross-compiling and the host-only protoc plugin was configured against the arm64-ios vcpkg tree, which provides no protobuf::libprotoc. Restore the explicit IOS condition (set by toolchains/ios.toolchain.cmake) alongside the cross-compiling check. First iOS run since that commit caught it. Co-Authored-By: Claude Fable 5 --- packages/streamr-proto-rpc/CMakeLists.txt | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/packages/streamr-proto-rpc/CMakeLists.txt b/packages/streamr-proto-rpc/CMakeLists.txt index 9abe80be..92bcd33c 100644 --- a/packages/streamr-proto-rpc/CMakeLists.txt +++ b/packages/streamr-proto-rpc/CMakeLists.txt @@ -96,7 +96,11 @@ file(WRITE "${CMAKE_BINARY_DIR}/streamr-proto-rpc-config.cmake" # and the plugin runs on the developer machine anyway. (Previously this # sat inside the modules block below and was skipped on Android only as # a side effect of the modules gate being OFF there.) -if(NOT CMAKE_CROSSCOMPILING) +# NOTE the explicit IOS check: iOS package builds keep the host +# CMAKE_SYSTEM_NAME (they use the Homebrew compiler and only swap the +# SDK/sysroot — see homebrewClang.cmake), so CMAKE_CROSSCOMPILING is +# FALSE there; IOS is set by toolchains/ios.toolchain.cmake. +if(NOT IOS AND NOT CMAKE_CROSSCOMPILING) add_executable(protobuf-streamr-plugin src/PluginCodeGeneratorMain.cpp include/streamr-proto-rpc/PluginCodeGenerator.hpp) target_include_directories(