feat(ai,ai-fal): per-model typed durations for video generation#641
Draft
tombeckenham wants to merge 10 commits into
Draft
feat(ai,ai-fal): per-model typed durations for video generation#641tombeckenham wants to merge 10 commits into
tombeckenham wants to merge 10 commits into
Conversation
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* feat(ai-openai): add gpt-image-2 to image model meta Adds `gpt-image-2` to OPENAI_IMAGE_MODELS so it can be used through openaiImage adapters. Reuses the gpt-image-1 provider-options/size shape (quality, background, output_format, output_compression, moderation, partial_images; sizes 1024x1024 / 1536x1024 / 1024x1536 / auto) and extends size + prompt-length validators. Also updates the media-generation skill and image-generation doc page to list the new model. * fix(ai-openrouter): restore web_fetch in tool capabilities map The model-metadata sync in #623 regenerated `OpenRouterChatModelToolCapabilitiesByName` with `['web_search']` only, which made `webFetchTool()` (added in #611) unassignable to any OpenRouter text adapter and broke the per-model type-safety test. Add `'web_fetch'` back so the existing tests compile.
* docs: refresh README discoverability * docs: address README review feedback
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: complete tool calls with server results * fix: hydrate server tool outputs from history * test: cover server tool history hydration * ci: apply automated fixes * test: add issue 176 manual repro page * ci: apply automated fixes * test: add live issue 176 repro flow * test: hide issue 176 repro from sidebar --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Adds a fifth generic `TModelDurationByName` to `VideoAdapter` plus two
introspection methods on the base class:
- `availableDurations()` returns a `DurationOptions` tagged union
(`discrete | range | mixed | none`) describing the durations the
current model accepts.
- `snapDuration(seconds)` coerces raw seconds to the closest valid
duration for the current model.
`generateVideo({ duration })` is now typed via
`VideoDurationForAdapter<TAdapter>`. The FAL adapter derives its
per-model duration type from the SDK's `EndpointTypeMap`, so e.g.
`falVideo('fal-ai/kling-video/v1.6/standard/text-to-video')` types
`duration` as `'5' | '10'`; `falVideo('fal-ai/veo3')` types it as
`'4s' | '6s' | '8s'`; `falVideo('fal-ai/minimax/video-01')` rejects
the field entirely.
Adapters that have not yet declared their per-model duration map get
sensible defaults (`{ kind: 'none' }`, `undefined`) so existing video
adapters keep working without changes.
Built on top of #622 (`@tanstack/ai-schemas`); once that PR's FAL
pipeline syncs runtime constraint data, the hand-curated map in
`packages/typescript/ai-fal/src/video/video-provider-options.ts` can
be replaced with schema-derived lookups. Follow-up issue #634 covers
building the Gemini Veo adapter directly on this contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version Preview3 package(s) bumped directly, 28 bumped as dependents. 🟥 Major bumps
🟨 Minor bumps
🟩 Patch bumps
|
|
View your CI Pipeline Execution ↗ for commit a4ed1b6 ☁️ Nx Cloud last updated this comment at |
|
View your CI Pipeline Execution ↗ for commit a4ed1b6
☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-schemas
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #534. Built on top of #622 (
@tanstack/ai-schemas) — targets that branch as the base. When #622 merges into main, this PR will rebase + retarget to main.Summary
VideoAdapter(TModelDurationByName) + two introspection methods on the base class —availableDurations()andsnapDuration(seconds). Default implementations return{ kind: 'none' }/undefined, so video adapters that haven't been migrated keep working unchanged.generateVideo({ duration })is now per-model typed viaVideoDurationForAdapter<TAdapter>.durationper model from@fal-ai/client'sEndpointTypeMap. Runtime constraint data for popular models lives in a hand-curated map inpackages/typescript/ai-fal/src/video/video-provider-options.ts(to be replaced with schema-derived lookups once feat(schemas): @tanstack/ai-schemas with nightly OpenAPI sync (closes #619) #622's FAL pipeline syncs).snapToDurationOptionutil handles discrete enums, ranges, mixed, keyword-with-unit forms ('8s'), andkind: 'none'.Per-model behaviour after this PR
durationtypeavailableDurations()fal-ai/kling-video/v1.6/{standard,pro}/text-to-video'5' | '10'discretefal-ai/pika/v2.2/text-to-video'5' | '10'discretefal-ai/luma-dream-machine/ray-2'5s' | '9s'discretefal-ai/veo3andfal-ai/veo3/image-to-video'4s' | '6s' | '8s'discretefal-ai/wan-25-preview/text-to-video'2' … '15'discretefal-ai/minimax/video-01{ kind: 'none' }fal-ai/hunyuan-video-v1.5/text-to-videonum_frames){ kind: 'none' }Scope decisions
{ kind: 'none' }) until someone migrates it.packages/typescript/ai-gemini/src/model-meta.ts:827-940). Filed as feat(gemini): add Google Veo video adapter on the typed-duration contract #634 to build directly on this contract.Breaking change
Callers passing
duration: <number>to FAL video models must either:'5','8s', etc.) directly, oradapter.snapDuration(seconds)and let it coerce.Existing video adapters that haven't opted into the typed-duration map (OpenAI Sora today) are not breaking.
Test plan
pnpm test:types— repo-wide type check passespnpm test:lib— all 32 packages, 904 core tests + 86 FAL tests pass (5 new snap.test.ts cases + 4 new FAL durations cases)pnpm test:eslint— cleanpnpm build— all 32 projects build cleanFAL_KEYis available; the E2E suite'svideo-genmatrix currently doesn't include'fal'so this is also follow-up infra work.falVideo('fal-ai/veo3')autocompletes'4s' | '6s' | '8s'onduration;falVideo('fal-ai/minimax/video-01')rejectsduration.🤖 Generated with Claude Code