Skip to content

Remote py apis#2675

Draft
shivamka1 wants to merge 22 commits into
db_v4from
remote-py-apis
Draft

Remote py apis#2675
shivamka1 wants to merge 22 commits into
db_v4from
remote-py-apis

Conversation

@shivamka1

@shivamka1 shivamka1 commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Transport abstraction for the RemoteGraph client

Motivation

The existing Python RemoteGraph client is write-only, with each of ~15
mutation methods inlining its own Jinja template and JSON-parsing chain.
This PR consolidates that plumbing into a single Transport abstraction
that every operation flows through, and adds a minimal composable read
surface on top of the same seam so the client can now do lazy view
composition like rg.window(0, 10).node("ben").degree() end-to-end.

What shipped

  • New Transport trait in
    raphtory-graphql/src/client/transport.rs
    async fn execute(&self, op: &Op) -> Result<Option<Prop>, ClientError>.
  • Op { Read(ReadExpr) | Write(WriteOp) } in
    op.rs — read side is a
    recursive expression tree, write side is a flat enum of arg structs.
  • GraphqlTransport in
    graphql_transport.rs
    renders write ops via the existing Jinja templates (moved from client
    wrappers), walks read expressions into nested GraphQL queries, parses
    responses.
  • All 15 individual writes migrated through the transport, byte-identical
    wire behavior: 7 on RemoteGraph, 4 on RemoteNode, 4 on RemoteEdge.
    Each method body shrinks from ~30–50 lines to ~5–15 lines.
  • Minimal read surface: Root, Window, Node, Degree — enough for
    rg.window(s, e).node(id).degree() end-to-end. Only the terminal fires
    an RPC.
  • Renames: GraphQLRemote{Graph,Node,Edge}Remote*,
    RaphtoryGraphQLClientRemoteClient, raphtory_client.rs
    remote_client.rs. Python-facing names unchanged.
  • Cleanup: client field removed from RemoteNode/RemoteEdge — they
    now hold only transport and view state. Kept on RemoteGraph pending
    batch-method migration (see below).
  • Bug fix: PyRemoteGraph::node/edge now delegate to the Rust methods,
    so the accumulated view chain (window(...)) is correctly propagated
    when descending to a node — previously silently dropped.
  • Python bindings added for window() and degree().

Testing

  • 46 pre-existing raphtory-graphql tests still green.
  • 3 new Rust tests in graphql_transport.rs (unit tests for the read
    render/parse; one integration test spawning a real server).
  • 3 new Python tests in
    test_remote_graph_transport.py:
    test_add_and_degree, test_windowed_degree, test_view_chain_propagation.

TODO

  • Two Python-side batch mutations (add_nodes, add_edges) still call
    client.query(...) directly (existing TODOs). Migrating them removes
    the last client field.
  • Read surface expansion (layer, at, rolling, nodes, ids, name, count,
    properties, ...).
  • One hard-shape algorithm end-to-end (e.g. WCC) to lock in the
    return-type conversion pattern.
  • Iteration + paging for for n in g.nodes: ....
  • Small LLM eval to retire the product bet.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite Current: d51f152 Previous: 9823ef7 Ratio
lotr_graph/num_edges 4 ns/iter (± 0) 0 ns/iter (± 0) +∞
lotr_graph/num_nodes 4 ns/iter (± 0) 1 ns/iter (± 0) 4
lotr_graph/graph_latest 3 ns/iter (± 0) 0 ns/iter (± 0) +∞
lotr_graph_materialise/materialize 7786999 ns/iter (± 37710) 1564816 ns/iter (± 35303) 4.98
lotr_graph_window_100/num_nodes 13 ns/iter (± 0) 5 ns/iter (± 0) 2.60
lotr_graph_window_100_materialise/materialize 7913789 ns/iter (± 43976) 1669150 ns/iter (± 10700) 4.74
lotr_graph_window_10/has_node_existing 137 ns/iter (± 10) 62 ns/iter (± 11) 2.21
lotr_graph_window_10_materialise/materialize 3320773 ns/iter (± 18455) 971980 ns/iter (± 4278) 3.42
lotr_graph_subgraph_10pc_materialise/materialize 2122228 ns/iter (± 29626) 334634 ns/iter (± 1287) 6.34
lotr_graph_subgraph_10pc_windowed/has_node_existing 140 ns/iter (± 9) 62 ns/iter (± 14) 2.26
lotr_graph_subgraph_10pc_windowed_materialise/materialize 1244948 ns/iter (± 9850) 230399 ns/iter (± 2617) 5.40
lotr_graph_window_50_layered/num_edges_temporal 142219 ns/iter (± 2969) 70121 ns/iter (± 7586) 2.03
lotr_graph_window_50_layered/has_node_existing 372 ns/iter (± 20) 129 ns/iter (± 12) 2.88
lotr_graph_window_50_layered/has_node_nonexisting 5 ns/iter (± 0) 2 ns/iter (± 0) 2.50
lotr_graph_window_50_layered/graph_latest 84737 ns/iter (± 950) 36649 ns/iter (± 916) 2.31
lotr_graph_window_50_layered_materialise/materialize 28753916 ns/iter (± 274745) 3488825 ns/iter (± 24948) 8.24
lotr_graph_persistent_window_50_layered/num_edges_temporal 577936 ns/iter (± 3570) 192686 ns/iter (± 1569) 3.00
lotr_graph_persistent_window_50_layered/has_node_existing 411 ns/iter (± 342) 174 ns/iter (± 83) 2.36
lotr_graph_persistent_window_50_layered/has_node_nonexisting 5 ns/iter (± 0) 2 ns/iter (± 0) 2.50
lotr_graph_persistent_window_50_layered/graph_latest 122921 ns/iter (± 2680) 57549 ns/iter (± 4809) 2.14
lotr_graph_persistent_window_50_layered_materialise/materialize 50798025 ns/iter (± 191574) 5298035 ns/iter (± 147912) 9.59

This comment was automatically generated by workflow using github-action-benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant