Skip to content

[FEA]: Add support for CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW in LaunchConfig #2626

Description

@lijinf2

Is this a duplicate?

Area

cuda.core

Is your feature request related to a problem? Please describe.

LaunchConfig does not expose CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW, which sets an L2 access-policy window for a launch (Ampere+).

Parent tracking: #496.

Describe the solution you'd like

  • Users can set CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW via LaunchConfig.
  • Mapping to the native launch attribute is tested.

Describe alternatives you've considered

Use cuda.bindings.driver (CUlaunchAttribute / CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW) and cuLaunchKernelEx directly.

Downside: leaves cuda.core users without a first-class LaunchConfig API; they must drop to low-level bindings, manage the attribute array themselves, and lose consistency with other LaunchConfig launch attributes (is_cooperative, programmatic_stream_serialization, etc.).

Additional context

Incremental LaunchConfig attribute coverage under #496 (same approach as #1334 for programmatic_stream_serialization).

Example (C API) — agent-generated; reviewed but not executed:

// 1. Reserve L2 cache space
cudaDeviceProp prop;
cudaGetDeviceProperties(&prop, 0);
size_t l2_setaside = min((size_t)(prop.l2CacheSize * 0.75),
                         (size_t)prop.persistingL2CacheMaxSize);
cudaDeviceSetLimit(cudaLimitPersistingL2CacheSize, l2_setaside);

// 2. Configure access policy window
size_t window_size = min((size_t)prop.accessPolicyMaxWindowSize,
                         num_bytes);

CUlaunchAttribute attr;
attr.id = CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW;
attr.value.accessPolicyWindow.base_ptr = (void*)data_persistent;
attr.value.accessPolicyWindow.num_bytes = window_size;
attr.value.accessPolicyWindow.hitRatio  = 0.6;
attr.value.accessPolicyWindow.hitProp   = CU_ACCESS_PROPERTY_PERSISTING;
attr.value.accessPolicyWindow.missProp  = CU_ACCESS_PROPERTY_STREAMING;

// 3. Launch with cuLaunchKernelEx
CUlaunchConfig config = {
    .gridDimX = grid, .gridDimY = 1, .gridDimZ = 1,
    .blockDimX = 256, .blockDimY = 1, .blockDimZ = 1,
    .sharedMemBytes = 0,
    .hStream = stream,
    .attrs = &attr,
    .numAttrs = 1
};

for (int i = 0; i < 10; i++) {
    cuLaunchKernelEx(&config, kernelA, NULL, NULL);
}

// 4. Disable window (release L2 for other kernels)
attr.value.accessPolicyWindow.num_bytes = 0;
cuLaunchKernelEx(&config, kernelB, NULL, NULL);

// 5. Clear persisting L2 lines
cuCtxResetPersistingL2Cache();

Metadata

Metadata

Assignees

No one assigned

    Labels

    cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions