Python: allow registering custom types for checkpoint serialization and deserialization - #7449
Python: allow registering custom types for checkpoint serialization and deserialization#7449Mahajan-Sachin wants to merge 10 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a process-wide registry to extend the restricted checkpoint unpickler’s allowlist so applications in hosted environments can permit custom workflow state types without modifying checkpoint storage configuration.
Changes:
- Introduces a global
_CUSTOM_ALLOWED_TYPESregistry and a publicregister_checkpoint_type(...)API in checkpoint encoding. - Extends restricted unpickling checks to permit globally registered types.
- Exposes
register_checkpoint_typefrom the package root and adds unit tests covering registration and validation.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| python/packages/core/agent_framework/_workflows/_checkpoint_encoding.py | Adds global allowlist + registration API and integrates it into restricted unpickling. |
| python/packages/core/agent_framework/init.py | Re-exports register_checkpoint_type via the lazy export mechanism and __all__. |
| python/packages/core/tests/workflow/test_checkpoint_unrestricted_pickle.py | Adds tests validating that global registration allows restricted deserialization. |
Suppressed comments (1)
python/packages/core/tests/workflow/test_checkpoint_unrestricted_pickle.py:552
- This test also mutates the process-wide checkpoint type registry and should restore it to avoid leaking global state to other tests.
def test_register_checkpoint_type_allows_string_key():
"""Custom types registered globally by string key are allowed during restricted deserialization."""
from agent_framework import register_checkpoint_type
original = _GloballyAllowedStringState(value="globally_allowed")
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
@copilot please review |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
python/packages/core/agent_framework/_workflows/_checkpoint_encoding.py:201
- The fallback
UnpicklingErrorguidance is now incomplete: with global registration supported, users should be told they can also allow a type viaagent_framework.register_checkpoint_type(...)(in addition toallowed_types/allowed_checkpoint_types).
if type_key in self._allowed_types or type_key in _CUSTOM_ALLOWED_TYPES:
resolved = super().find_class(module, name) # nosec
if isinstance(resolved, type):
return resolved
raise pickle.UnpicklingError(f"Checkpoint deserialization blocked for non-type global '{type_key}'.")
…ion register_checkpoint_type
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
python/packages/core/agent_framework/_workflows/_checkpoint_encoding.py:418
register_checkpoint_typeaccepts strings with multiple ':' characters (e.g."pkg.mod:Type:Extra") because it usespartition(":")and doesn't validate the remainder. This will silently register a key that can never match themodule:namekeys produced by pickle, making registration ineffective and hard to debug. Consider rejecting keys that don't contain exactly one ':' before stripping/normalizing.
if isinstance(cls_or_key, str):
module, sep, qualname = cls_or_key.partition(":")
module = module.strip()
qualname = qualname.strip()
if not sep or not module or not qualname:
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (2)
python/packages/core/agent_framework/_workflows/_checkpoint_encoding.py:218
- The unblock guidance mentions
FileCheckpointStorage.allowed_checkpoint_types, butFileCheckpointStoragedoesn’t expose anallowed_checkpoint_typesattribute (it’s a constructor parameter stored internally as_allowed_types). This example is misleading; suggest pointing to passingallowed_checkpoint_typesto the constructor instead.
f"To allow this type, register it globally via 'agent_framework.register_checkpoint_type', "
f"include its 'module:qualname' key in the 'allowed_types' set passed to 'decode_checkpoint_value', "
f"or add it to 'allowed_checkpoint_types' on your checkpoint storage "
f"(for example, 'FileCheckpointStorage.allowed_checkpoint_types')."
python/packages/core/agent_framework/_workflows/_checkpoint_encoding.py:411
register_checkpoint_typemutates a process-wide global allowlist, but the docstring doesn’t call out the global/process-wide scope or the security implications (expands what the restricted unpickler will instantiate). Clarifying this helps callers understand when to call it and the risk tradeoff.
"""Register a custom type to be allowed during checkpoint deserialization.
Each registered type should be either a type class (e.g., custom models or
dataclasses) or a ``"module:qualname"`` string.
Args:
cls_or_key: The type class or module-qualified string to register.
"""
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
python/packages/core/tests/workflow/test_checkpoint_unrestricted_pickle.py:38
- Importing the private symbol
_CUSTOM_ALLOWED_TYPESfromagent_framework._workflows._checkpoint_encodingwill likely trigger pyright'sreportPrivateUsage(this file already suppresses that check for_base64_to_unpickle). Add an inline pyright ignore to avoid type-check failures in CI.
from agent_framework._workflows._checkpoint_encoding import _CUSTOM_ALLOWED_TYPES
…TYPES import in tests
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
python/packages/core/agent_framework/_workflows/_checkpoint_encoding.py:138
- The
_RestrictedUnpicklersummary docstring no longer reflects behavior now that a process-wide custom-type registry is part of the allowlist. Consider mentioning the global registration mechanism in the docstring summary so readers don’t miss that types can be permitted outsideallowed_types.
"""Unpickler that restricts which classes may be instantiated.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
python/packages/core/tests/workflow/test_checkpoint_unrestricted_pickle.py:43
- This fixture restores the global registry after the test, but it never clears it before the test runs. That makes the new tests depend on
_CUSTOM_ALLOWED_TYPESalready being empty (or at least not containing these types) when the test starts, which can become order-dependent if any other test registers checkpoint types globally.
backup = set(_CUSTOM_ALLOWED_TYPES)
yield
_CUSTOM_ALLOWED_TYPES.clear()
_CUSTOM_ALLOWED_TYPES.update(backup)
|
Hi @moonbox3 and @TaoChenOSU, I've updated the PR to address all review feedback and edge cases:
All unit tests and type-checks are passing cleanly. Ready for your review! |
Motivation & Context
In hosting environments like Foundry, developers don't manage the checkpoint store directly. If custom types are used in their workflows, checkpoint serialization/deserialization will fail because the restricted unpickler rejects any unknown type for security reasons. Currently, there is no way to extend
allowed_checkpoint_typeswithout direct access to the checkpoint storage configuration.This PR introduces a process-wide global registry for checkpoint serialization, similar to the existing
register_state_typeutility used in agent sessions, allowing developers to register custom classes or string type keys.Description & Review Guide
What are the major changes?
_CUSTOM_ALLOWED_TYPESregistry set inpackages/core/agent_framework/_workflows/_checkpoint_encoding.py.register_checkpoint_type(cls_or_key: type[Any] | str)function to allow developers to register classes or string keys._RestrictedUnpickler.find_classand_is_allowed_typeto allow types registered in_CUSTOM_ALLOWED_TYPES.register_checkpoint_typeunder the core package root__init__.py.packages/core/tests/workflow/test_checkpoint_unrestricted_pickle.pyverifying registration behavior, string key registration, and input validation.What is the impact of these changes?
Developers can now call
register_checkpoint_type(MyCustomClass)at module import time. This allows their custom types to pass through checkpoint serialization in hosting environments without needing direct access to storage initializers.What do you want reviewers to focus on?
_CUSTOM_ALLOWED_TYPESinto bothfind_classand_is_allowed_typein the restricted unpickler.__init__.py.Related Issue
Fixes #7413
Contribution Checklist