Remote py apis#2675
Draft
shivamka1 wants to merge 22 commits into
Draft
Conversation
…ree read wired end-to-end
…ata, update_metadata, delete_edge) through Transport
…date_metadata) through Transport
…tadata) through Transport
…aph.node/edge to Rust methods (fixes view-chain preservation in Python)
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
| Benchmark suite | Current: d51f152 | Previous: 9823ef7 | Ratio |
|---|---|---|---|
lotr_graph/num_edges |
4 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph/num_nodes |
4 ns/iter (± 0) |
1 ns/iter (± 0) |
4 |
lotr_graph/graph_latest |
3 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph_materialise/materialize |
7786999 ns/iter (± 37710) |
1564816 ns/iter (± 35303) |
4.98 |
lotr_graph_window_100/num_nodes |
13 ns/iter (± 0) |
5 ns/iter (± 0) |
2.60 |
lotr_graph_window_100_materialise/materialize |
7913789 ns/iter (± 43976) |
1669150 ns/iter (± 10700) |
4.74 |
lotr_graph_window_10/has_node_existing |
137 ns/iter (± 10) |
62 ns/iter (± 11) |
2.21 |
lotr_graph_window_10_materialise/materialize |
3320773 ns/iter (± 18455) |
971980 ns/iter (± 4278) |
3.42 |
lotr_graph_subgraph_10pc_materialise/materialize |
2122228 ns/iter (± 29626) |
334634 ns/iter (± 1287) |
6.34 |
lotr_graph_subgraph_10pc_windowed/has_node_existing |
140 ns/iter (± 9) |
62 ns/iter (± 14) |
2.26 |
lotr_graph_subgraph_10pc_windowed_materialise/materialize |
1244948 ns/iter (± 9850) |
230399 ns/iter (± 2617) |
5.40 |
lotr_graph_window_50_layered/num_edges_temporal |
142219 ns/iter (± 2969) |
70121 ns/iter (± 7586) |
2.03 |
lotr_graph_window_50_layered/has_node_existing |
372 ns/iter (± 20) |
129 ns/iter (± 12) |
2.88 |
lotr_graph_window_50_layered/has_node_nonexisting |
5 ns/iter (± 0) |
2 ns/iter (± 0) |
2.50 |
lotr_graph_window_50_layered/graph_latest |
84737 ns/iter (± 950) |
36649 ns/iter (± 916) |
2.31 |
lotr_graph_window_50_layered_materialise/materialize |
28753916 ns/iter (± 274745) |
3488825 ns/iter (± 24948) |
8.24 |
lotr_graph_persistent_window_50_layered/num_edges_temporal |
577936 ns/iter (± 3570) |
192686 ns/iter (± 1569) |
3.00 |
lotr_graph_persistent_window_50_layered/has_node_existing |
411 ns/iter (± 342) |
174 ns/iter (± 83) |
2.36 |
lotr_graph_persistent_window_50_layered/has_node_nonexisting |
5 ns/iter (± 0) |
2 ns/iter (± 0) |
2.50 |
lotr_graph_persistent_window_50_layered/graph_latest |
122921 ns/iter (± 2680) |
57549 ns/iter (± 4809) |
2.14 |
lotr_graph_persistent_window_50_layered_materialise/materialize |
50798025 ns/iter (± 191574) |
5298035 ns/iter (± 147912) |
9.59 |
This comment was automatically generated by workflow using github-action-benchmark.
…eld from RemoteGraph
…e impls from Py* types to op types
…Degree/OutDegree/Name) with Python bindings + tests
…)/.dst() navigation
…optional-string machinery
…ypes, ExcludeNodes) + Valid, DefaultLayer, and graph Path/Namespace/Name terminals
…ighbour accessors
… wrong answers under view chains
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transport abstraction for the RemoteGraph client
Motivation
The existing Python
RemoteGraphclient is write-only, with each of ~15mutation methods inlining its own Jinja template and JSON-parsing chain.
This PR consolidates that plumbing into a single
Transportabstractionthat every operation flows through, and adds a minimal composable read
surface on top of the same seam so the client can now do lazy view
composition like
rg.window(0, 10).node("ben").degree()end-to-end.What shipped
Transporttrait inraphtory-graphql/src/client/transport.rs—async fn execute(&self, op: &Op) -> Result<Option<Prop>, ClientError>.Op { Read(ReadExpr) | Write(WriteOp) }inop.rs— read side is arecursive expression tree, write side is a flat enum of arg structs.
GraphqlTransportingraphql_transport.rs—renders write ops via the existing Jinja templates (moved from client
wrappers), walks read expressions into nested GraphQL queries, parses
responses.
wire behavior: 7 on
RemoteGraph, 4 onRemoteNode, 4 onRemoteEdge.Each method body shrinks from ~30–50 lines to ~5–15 lines.
Root,Window,Node,Degree— enough forrg.window(s, e).node(id).degree()end-to-end. Only the terminal firesan RPC.
GraphQLRemote{Graph,Node,Edge}→Remote*,RaphtoryGraphQLClient→RemoteClient,raphtory_client.rs→remote_client.rs. Python-facing names unchanged.clientfield removed fromRemoteNode/RemoteEdge— theynow hold only
transportand view state. Kept onRemoteGraphpendingbatch-method migration (see below).
PyRemoteGraph::node/edgenow delegate to the Rust methods,so the accumulated view chain (
window(...)) is correctly propagatedwhen descending to a node — previously silently dropped.
window()anddegree().Testing
raphtory-graphqltests still green.graphql_transport.rs(unit tests for the readrender/parse; one integration test spawning a real server).
test_remote_graph_transport.py:test_add_and_degree,test_windowed_degree,test_view_chain_propagation.TODO
add_nodes,add_edges) still callclient.query(...)directly (existingTODOs). Migrating them removesthe last
clientfield.properties, ...).
return-type conversion pattern.
for n in g.nodes: ....