Summary
Buffers allocated by pool-backed memory resources are freed directly by their C++ DevicePtrHandle deleter. Their associated memory resource's deallocate() method is never called.
This makes deallocation behavior inconsistent and prevents subclasses of DeviceMemoryResource, PinnedMemoryResource, and ManagedMemoryResource from observing or customizing teardown.
Reproducer
from cuda.core import Device, DeviceMemoryResource
calls = []
class RecordingMemoryResource(DeviceMemoryResource):
def deallocate(self, ptr, size, *, stream):
calls.append((ptr, size, stream))
return super().deallocate(ptr, size, stream=stream)
device = Device()
device.set_current()
mr = RecordingMemoryResource(device)
buf = mr.allocate(1024, stream=device.default_stream)
assert buf.memory_resource is mr
buf.close()
assert len(calls) == 1 # Fails: calls is emptyThe allocation is freed, but directly through the C++ handle rather than through RecordingMemoryResource.deallocate().
Inconsistent behavior
The behavior depends on how the Buffer was constructed:
mr.allocate(...) uses a directly owning C++ handle and bypasses mr.deallocate().Buffer.from_handle(..., mr=mr) records the MR as the owner and dispatches teardown through mr.deallocate().
Consequently, the same memory resource subclass has two different deallocation models.
Callback-backed resources such as LegacyPinnedMemoryResource also dispatch through mr.deallocate(), making the behavior inconsistent between built-in resource implementations.
Expected behavior
If concrete memory resources are supported as subclass bases, an overridden deallocate() should be honored for buffers allocated by the subclass.
This is useful for accounting, tracing, validation, failure injection, and custom resource policies—not only test instrumentation.
If concrete memory resources are intentionally not subclassable, that restriction should instead be explicit and enforced.
Implementation considerations
The current direct C++ path has important properties that should be preserved:
- It retains the memory-pool handle for the allocation lifetime.
- It can free without executing Python during interpreter shutdown.
- It stores the stream/context recipe needed for safe deferred teardown.
A fix may therefore require a callback-capable pool allocation handle that retains the pool while dispatching through the Python MR, or a fast path for exact built-in types with callback dispatch for subclasses.
GraphMemoryResource, which also creates directly owning handles, should be audited for the same behavior.
Related
Summary
Buffers allocated by pool-backed memory resources are freed directly by their C++
DevicePtrHandledeleter. Their associated memory resource'sdeallocate()method is never called.This makes deallocation behavior inconsistent and prevents subclasses of
DeviceMemoryResource,PinnedMemoryResource, andManagedMemoryResourcefrom observing or customizing teardown.Reproducer
The allocation is freed, but directly through the C++ handle rather than through
RecordingMemoryResource.deallocate().Inconsistent behavior
The behavior depends on how the Buffer was constructed:
mr.allocate(...)uses a directly owning C++ handle and bypassesmr.deallocate().Buffer.from_handle(..., mr=mr)records the MR as the owner and dispatches teardown throughmr.deallocate().Consequently, the same memory resource subclass has two different deallocation models.
Callback-backed resources such as
LegacyPinnedMemoryResourcealso dispatch throughmr.deallocate(), making the behavior inconsistent between built-in resource implementations.Expected behavior
If concrete memory resources are supported as subclass bases, an overridden
deallocate()should be honored for buffers allocated by the subclass.This is useful for accounting, tracing, validation, failure injection, and custom resource policies—not only test instrumentation.
If concrete memory resources are intentionally not subclassable, that restriction should instead be explicit and enforced.
Implementation considerations
The current direct C++ path has important properties that should be preserved:
A fix may therefore require a callback-capable pool allocation handle that retains the pool while dispatching through the Python MR, or a fast path for exact built-in types with callback dispatch for subclasses.
GraphMemoryResource, which also creates directly owning handles, should be audited for the same behavior.Related
from_handle/from_*APIs #1989 — inconsistent lifetime management acrossfrom_handle/from_*APIs