Skip to content

refactor!: Adapt to apify-client v3#719

Draft
vdusek wants to merge 3 commits into
masterfrom
adapt-to-apify-client-v3
Draft

refactor!: Adapt to apify-client v3#719
vdusek wants to merge 3 commits into
masterfrom
adapt-to-apify-client-v3

Conversation

@vdusek
Copy link
Copy Markdown
Contributor

@vdusek vdusek commented Dec 23, 2025

Description

Issues

Testing

  • The existing SDK tests pass with apify-python-client v3.

@vdusek vdusek self-assigned this Dec 23, 2025
@github-actions github-actions Bot added this to the 130th sprint - Tooling team milestone Dec 23, 2025
@github-actions github-actions Bot added t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics. labels Dec 23, 2025
@vdusek vdusek added the adhoc Ad-hoc unplanned task added during the sprint. label Dec 23, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 23, 2025

Codecov Report

❌ Patch coverage is 94.46640% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.66%. Comparing base (94bdd37) to head (127d10e).

Files with missing lines Patch % Lines
src/apify/_actor.py 69.69% 10 Missing ⚠️
...ify/storage_clients/_apify/_api_client_creation.py 66.66% 1 Missing ⚠️
...rc/apify/storage_clients/_apify/_dataset_client.py 66.66% 1 Missing ⚠️
.../storage_clients/_apify/_key_value_store_client.py 80.00% 1 Missing ⚠️
...age_clients/_apify/_request_queue_single_client.py 94.73% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #719      +/-   ##
==========================================
- Coverage   87.00%   86.66%   -0.35%     
==========================================
  Files          48       48              
  Lines        2956     2918      -38     
==========================================
- Hits         2572     2529      -43     
- Misses        384      389       +5     
Flag Coverage Δ
e2e 35.64% <61.66%> (-1.95%) ⬇️
integration 57.74% <81.81%> (-1.16%) ⬇️
unit 74.81% <77.47%> (-1.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from a45f20c to 4270cb9 Compare January 2, 2026 12:45
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from c26dd9f to 7d6dbb1 Compare January 9, 2026 14:27
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch 3 times, most recently from 3e14e50 to 12a29ae Compare January 23, 2026 14:25
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from ba4d354 to e094d6a Compare January 27, 2026 14:09
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from 4528205 to 9c44f3b Compare February 23, 2026 11:00
vdusek added a commit to apify/apify-client-python that referenced this pull request Feb 23, 2026
### Summary

This is a major refactoring that introduces fully typed Pydantic models
throughout the client library. The models are generated from the OpenAPI
specifications. All API responses now return typed objects instead of
raw dictionaries.

This follows up on apify/apify-docs#2182.

### Issues

- Closes: #21
- Closes: #481

### Packages

- Add direct dependency on `Pydantic`.
- Removes the dependency on `apify-shared`.
- Add dev dependency
[datamodel-code-generator](https://koxudaxi.github.io/datamodel-code-generator/)
for model generation.

### Key changes

- Uses
[datamodel-code-generator](https://koxudaxi.github.io/datamodel-code-generator/)
tool configured via `pyproject.toml` to generate Pydantic models based
on the [OpenAPI specs](https://docs.apify.com/api/openapi.json).
- Refactors the whole codebase to adopt the new generated models.
- All resource clients now return typed Pydantic models (`Actor`,
`Task`, `Run`, etc.).
- Adds response wrappers for validating and extracting API response
data.
- Updates list methods to return typed pagination models.
- Documentation examples now use typed attribute access.
- Updates the SDK to use the new typed client.
- See the corresponding PR in `apify/apify-sdk-python` for details -
apify/apify-sdk-python#719.
  - It will be merged later.

### Architecture

- Get rid of 3/4/5 levels of inheritance.
- Get rid of inline imports because of circular dependencies.
- I had to utilize `ClientRegistry` to be able to achieve that (because
of resource clients-siblings imports).

### Breaking changes

- Client methods now return Pydantic models instead of dicts.
- Access patterns change from dict-style (`result['key']`) to
attribute-style (`result.key`).

### Test plan

- Updated test concurrency to 16 workers.
- A lot of new tests were implemented - coverage ~95%.
- Unit tests - do not call production API, only for testing utils or
other functionality using mocks.
  - Integration tests - call production API.
- Thanks to the new tests, I was able to do a lot of fixes in the
OpenAPI specs.

### Next steps

- Explore the generation of resource clients using
[openapi-python-client](https://github.com/openapi-generators/openapi-python-client).
- Fully automate model updates based on changes in
[apify-api/openapi](https://github.com/apify/apify-docs/tree/master/apify-api/openapi).
- This will be released as part of the Apify client v3.0.
@vdusek vdusek changed the title chore: Adapt to apify-client v3 [WIP] refactor!: Adapt to apify-client v3 [WIP] Mar 10, 2026
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch 2 times, most recently from 8057e7c to 05a8aba Compare April 16, 2026 13:07
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from a5fb9c8 to fc2dde8 Compare April 28, 2026 14:34
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from fc2dde8 to e3439b4 Compare May 11, 2026 07:53
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch 2 times, most recently from 7ac5f59 to 2019646 Compare May 22, 2026 15:35
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch 3 times, most recently from 4a98a06 to cb9ed09 Compare May 27, 2026 10:59
@vdusek vdusek changed the base branch from master to drop-python-3.10 May 27, 2026 10:59
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from cb9ed09 to 24a6fa8 Compare May 27, 2026 11:01
@vdusek vdusek marked this pull request as ready for review May 27, 2026 13:26
@vdusek vdusek changed the title refactor!: Adapt to apify-client v3 [WIP] refactor!: Adapt to apify-client v3 May 27, 2026
@vdusek vdusek marked this pull request as draft May 27, 2026 13:45
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from 1a1043e to a23a8b5 Compare May 28, 2026 08:57
Base automatically changed from drop-python-3.10 to master May 28, 2026 14:56
@vdusek vdusek force-pushed the adapt-to-apify-client-v3 branch from 102881c to dd58d25 Compare May 29, 2026 13:59
vdusek added 2 commits May 29, 2026 16:48
…lasses

The SDK kept full standalone pydantic copies of apify-client's four Actor
pricing-info models (plus ActorChargeEvent / PricingPerEvent) and a
_from_client_pricing_info converter. The copies existed only because the
platform's APIFY_ACTOR_PRICING_INFO env var omits fields apify-client v3
marks required (apifyMarginPercentage, createdAt, startedAt, per-event
eventDescription, per-variant trialMinutes / pricePerUnitUsd / unitName).

Replace the copies with thin subclasses of the apify-client models that
relax only those omitted fields to optional. Since each subclass is-a
apify-client model, the charging manager can isinstance-check against the
client base, so Run.pricing_info from the API flows through unchanged and
the converter is removed.

Configuration.actor_pricing_info keeps its exact discriminated-union shape,
so there is no public API change. Also fixes a latent bug: event_price_usd
is now correctly optional (inherited), so tier-priced pay-per-event Actors
no longer fail env-var validation; such events are skipped in the
flat-price charging path.
Restructure the apify-client section of the v4 upgrading guide to mirror the
client's "Upgrading to v3" guide, and document the changes that reach SDK
users through it:

- Split into "Built on apify-client v3", "Typed responses", and "Literal
  string aliases instead of StrEnum classes"; fix the typed-responses method
  list (start / abort / call / call_task; metamorph returns None).
- Add "Actor pricing info models" - the pricing models are now thin
  apify-client subclasses; shape unchanged, fuller field set, event_price_usd
  optional.
- Add "Using the client from Actor.new_client" - 404 raising NotFoundError on
  ambiguous endpoints, keyword-only arguments, and async iterate_* returning
  AsyncIterator.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

2 participants