Skip to content

refactor!: remove push_to/pull_from and the source/sink concept family #368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mvandeberg merged 1 commit into from
Jul 23, 2026
Merged

refactor!: remove push_to/pull_from and the source/sink concept family #368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions doc/modules/ROOT/nav.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,10 @@
* xref:6.streams/6.intro.adoc[Stream Concepts]
** xref:6.streams/6a.overview.adoc[Overview]
** xref:6.streams/6b.streams.adoc[Streams (Partial I/O)]
** xref:6.streams/6c.sources-sinks.adoc[Sources and Sinks (Complete I/O)]
** xref:6.streams/6d.buffer-concepts.adoc[Buffer Sources and Sinks]
** xref:6.streams/6e.algorithms.adoc[Transfer Algorithms]
** xref:6.streams/6f.isolation.adoc[Physical Isolation]
* xref:7.testing/7.intro.adoc[Testing]
** xref:7.testing/7a.drivers.adoc[Driving Tests]
** xref:7.testing/7b.mock-streams.adoc[Mock Streams]
** xref:7.testing/7c.mock-sources-sinks.adoc[Mock Sources and Sinks]
** xref:7.testing/7d.mock-buffer-concepts.adoc[Mock Buffer Sources and Sinks]
** xref:7.testing/7e.buffer-inspection.adoc[Buffer Inspection]
* xref:8.examples/8.intro.adoc[Example Programs]
** xref:8.examples/8a.hello-task.adoc[Hello Task]
Expand All @@ -48,7 +43,6 @@
** xref:8.examples/8f.timeout-cancellation.adoc[Timeout with Cancellation]
** xref:8.examples/8g.parallel-fetch.adoc[Parallel Fetch]
** xref:8.examples/8i.echo-server-corosio.adoc[Echo Server with Corosio]
** xref:8.examples/8j.stream-pipeline.adoc[Stream Pipeline]
** xref:8.examples/8k.strand-serialization.adoc[Strand Serialization]
** xref:8.examples/8l.async-mutex.adoc[Async Mutex]
** xref:8.examples/8m.parallel-tasks.adoc[Parallel Tasks]
Expand All @@ -59,13 +53,8 @@
** xref:9.design/9a.CapyLayering.adoc[Layered Abstractions]
** xref:9.design/9b.Separation.adoc[Why Capy Is Separate]
** xref:9.design/9c.ReadStream.adoc[ReadStream]
** xref:9.design/9d.ReadSource.adoc[ReadSource]
** xref:9.design/9e.BufferSource.adoc[BufferSource]
** xref:9.design/9f.WriteStream.adoc[WriteStream]
** xref:9.design/9g.WriteSink.adoc[WriteSink]
** xref:9.design/9h.BufferSink.adoc[BufferSink]
** xref:9.design/9i.TypeEraseAwaitable.adoc[Type-Erasing Awaitables]
** xref:9.design/9j.any_buffer_sink.adoc[AnyBufferSink]
** xref:9.design/9k.Executor.adoc[Executor]
** xref:9.design/9l.RunApi.adoc[Run API]
** xref:9.design/9m.WhyNotCobalt.adoc[Why Not Cobalt?]
Expand Down
91 changes: 6 additions & 85 deletions doc/modules/ROOT/pages/6.streams/6a.overview.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ This section introduces Capy's stream concepts—the abstractions that enable da

* Understanding of buffer sequences

== Six Concepts for Data Flow
== Three Concepts for Data Flow

Capy defines six concepts for I/O operations, organized in three pairs:
Capy defines three concepts for I/O operations:

[cols="1,1,2"]
|===
Expand All @@ -22,26 +22,12 @@ Capy defines six concepts for I/O operations, organized in three pairs:
| Write
| Partial writes—writes as much as possible

| `ReadSource`
| Read
| Complete reads—fills buffer or signals EOF

| `WriteSink`
| Write
| Complete writes with explicit EOF signaling

| `BufferSource`
| Read
| Callee-owns-buffers read pattern

| `BufferSink`
| Write
| Callee-owns-buffers write pattern
| `Stream`
| Read + Write
| A connected pair—satisfies both `ReadStream` and `WriteStream`
|===

== Streams vs Sources/Sinks

=== Streams: Partial I/O
== Streams: Partial I/O

Stream operations transfer *some* data and return. They do not guarantee a specific amount:

Expand All @@ -58,47 +44,6 @@ auto [ec, n] = co_await stream.write_some(buffers);

This matches raw OS behavior—syscalls return when data is available, not when buffers are full.

=== Sources/Sinks: Complete I/O

Source/sink operations complete fully or signal completion:

[source,cpp]
----
// ReadSource: fills buffer completely, or returns EOF/error with partial
auto [ec, n] = co_await source.read(buffer);
// n == buffer_size(buffer), or ec indicates why not

// WriteSink: writes all data, with explicit EOF
co_await sink.write_eof(buffers); // atomic write + EOF signal
----

These are higher-level abstractions built on streams.

== Buffer Sources/Sinks: Callee-Owns-Buffers

The third pair inverts buffer ownership:

* With streams/sources/sinks, the caller provides buffers
* With buffer sources/sinks, the callee provides buffers

[source,cpp]
----
// BufferSource: callee provides read-only buffers
const_buffer arr[8];
auto [ec, bufs] = co_await source.pull(arr);
// bufs is a span pointing into source's internal data;
// ec == cond::eof with an empty span signals exhaustion
source.consume(n); // mark n bytes consumed before the next pull

// BufferSink: callee provides writable buffers
mutable_buffer arr[8];
auto bufs = sink.prepare(arr); // returns a span of writable buffers
// Write into bufs, then commit
co_await sink.commit(bytes_written);
----

This pattern enables zero-copy I/O: data never moves through intermediate buffers.

== Type-Erasing Wrappers

Each concept has a corresponding type-erasing wrapper:
Expand All @@ -115,18 +60,6 @@ Each concept has a corresponding type-erasing wrapper:

| (Both)
| `any_stream`

| `ReadSource`
| `any_read_source`

| `WriteSink`
| `any_write_sink`

| `BufferSource`
| `any_buffer_source`

| `BufferSink`
| `any_buffer_sink`
|===

These wrappers enable:
Expand All @@ -143,18 +76,6 @@ These wrappers enable:
* You're implementing a protocol that processes data incrementally
* Performance is critical and you want minimal abstraction

=== Use Sources/Sinks When:

* You need complete data units (messages, records, frames)
* EOF signaling is part of your protocol
* You're composing transformations (compression, encryption)

=== Use Buffer Sources/Sinks When:

* Zero-copy is essential
* The source/sink owns the memory (memory-mapped files, hardware buffers)
* You're implementing a processing pipeline

== The Value Proposition

Type-erased wrappers let you write transport-agnostic code:
Expand Down
2 changes: 1 addition & 1 deletion doc/modules/ROOT/pages/6.streams/6b.streams.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,4 +256,4 @@ The implementation doesn't know the concrete stream type. It compiles once and w
| Type-erased bidirectional stream wrapper
|===

You have now learned the stream concepts for partial I/O. Continue to xref:6.streams/6c.sources-sinks.adoc[Sources and Sinks] to learn about complete I/O with EOF signaling.
You have now learned the stream concepts for partial I/O. Continue to xref:6.streams/6f.isolation.adoc[Physical Isolation].
Loading
Loading