LCORE-2917 Adding image attachment capabilities to /query and /streaming_query#2222
LCORE-2917 Adding image attachment capabilities to /query and /streaming_query#2222JslYoon wants to merge 1 commit into
Conversation
WalkthroughThe PR replaces the query endpoint stub with a complete synchronous and streaming API specification, documents attachments, interruption, responses, errors, and examples, and adds image signature validation with unit and end-to-end coverage. ChangesQuery API documentation and attachment validation
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 7✅ Passed checks (7 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
JslYoon
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/devel_doc/query_endpoint.md`:
- Line 181: Update the fenced code block containing the event payload example to
specify an explicit language, such as text, on its opening fence while
preserving the example content.
In `@tests/e2e/features/streaming_query.feature`:
- Around line 149-166: Update the streaming scenarios in
tests/e2e/features/streaming_query.feature at lines 149-166 and 206-228 to wait
for the SSE stream to complete after asserting HTTP 200, then verify a
successful terminal/result event or explicitly confirm no error event is
emitted. Apply the same completion assertion to both the valid image-query
scenario and the mixed-attachment scenario.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: df59433a-aeb1-4ed4-8701-9ccf8d2c3396
📒 Files selected for processing (4)
docs/devel_doc/query_endpoint.mddocs/devel_doc/streaming_query_endpoint.mdtests/e2e/features/query.featuretests/e2e/features/streaming_query.feature
📜 Review details
⏰ Context from checks skipped due to timeout. (15)
- GitHub Check: E2E: server mode / ci / group 3
- GitHub Check: E2E: library mode / ci / group 2
- GitHub Check: E2E: library mode / ci / group 1
- GitHub Check: mypy
- GitHub Check: E2E: library mode / ci / group 3
- GitHub Check: E2E: server mode / ci / group 2
- GitHub Check: E2E: server mode / ci / group 1
- GitHub Check: unit_tests (3.12)
- GitHub Check: unit_tests (3.13)
- GitHub Check: integration_tests (3.12)
- GitHub Check: integration_tests (3.13)
- GitHub Check: E2E Tests for Lightspeed Evaluation job
- GitHub Check: spectral
- GitHub Check: Pylinter
- GitHub Check: build-pr
🧰 Additional context used
📓 Path-based instructions (2)
**/*
📄 CodeRabbit inference engine (Custom checks)
**/*: Flag meaningful O(n^2)+ algorithms on non-trivial inputs, including handlers and Kubernetes list operations.
Flag N+1 patterns that list items and then query once per item, including Kubernetes API and database access.
Flag expensive work inside loops, including API calls, JSON parsing, and regex compilation.
Flag unbounded growth in caches, watchers, or buffers when eviction or limits are missing.
Flag missing pagination or limits on list operations and API endpoints.
Flag secrets or tokens logged in plaintext or hardcoded in source.
Flag API endpoints missing authentication or authorization.
Flag injection vulnerabilities, including SQL injection, command injection, and path traversal.
Flag sensitive data leaked in API responses, WebSocket messages, or logs.
Flag Kubernetes Secrets and Red Hat secrets missing OwnerReferences.Never commit secrets or keys; use environment variables for sensitive data.
Files:
docs/devel_doc/streaming_query_endpoint.mdtests/e2e/features/streaming_query.featuretests/e2e/features/query.featuredocs/devel_doc/query_endpoint.md
tests/e2e/**/*.feature
📄 CodeRabbit inference engine (AGENTS.md)
Write end-to-end tests using behave and Gherkin feature files.
Files:
tests/e2e/features/streaming_query.featuretests/e2e/features/query.feature
🪛 markdownlint-cli2 (0.23.0)
docs/devel_doc/query_endpoint.md
[warning] 181-181: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔇 Additional comments (5)
tests/e2e/features/query.feature (1)
160-241: LGTM!tests/e2e/features/streaming_query.feature (1)
168-205: LGTM!docs/devel_doc/query_endpoint.md (2)
1-180: LGTM!Also applies to: 182-247, 258-500, 504-525
248-257: 🗄️ Data Integrity & IntegrationNo change needed. The SSE
endevent contract includesavailable_quotasalongsidedata;EndStreamPayloadalso documents this by-name exception.docs/devel_doc/streaming_query_endpoint.md (1)
1-5: LGTM!
radofuchs
left a comment
radofuchs
left a comment
There was a problem hiding this comment.
LGTM in overall, just need a few updates
| "system_prompt": "You are a helpful assistant" | ||
| } | ||
| """ | ||
| Then The status code of the response is 200 |
There was a problem hiding this comment.
testing only for 200 is not enough here, add also the check for some more details, e.g. for some specific descriptive words in the response
| "system_prompt": "You are a helpful assistant" | ||
| } | ||
| """ | ||
| Then The status code of the response is 200 |
There was a problem hiding this comment.
same as above, but make sure that both parts are processed properly
| "system_prompt": "You are a helpful assistant" | ||
| } | ||
| """ | ||
| Then The status code of the response is 200 |
There was a problem hiding this comment.
agreed with coderabbit here, also apply the same comments from query
| Then The status code of the response is 200 | ||
|
|
||
| Scenario: Check if query with shields returns 413 when question is too long for model context | ||
| Scenario: Check if LLM responds properly when a valid image attachment is sent |
There was a problem hiding this comment.
completely missing a test here for a case where you send a random base64 string. I tested this locally, lightspeed-stack returns generic 500 for this which needs a fix
Signed-off-by: Lucas <lyoon@redhat.com>
9adc98a to
56eb60b
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/devel_doc/query_endpoint.md`:
- Around line 100-105: Update the image attachment documentation near the
existing base64 and MIME-type rules to state that decoded bytes must match the
declared image/jpeg or image/png file signature, as validated by
_validate_image_magic_bytes. Specify that valid base64 with a mismatched file
format is rejected with HTTP 422.
- Around line 92-95: Align the attachment documentation with actual runtime
behavior: either add explicit validation for the documented attachment_type and
content_type allowlists in the Attachment model and preserve the HTTP 422 claim,
or remove the allowlist and invalid-value 422 claims if no such validation is
intended. Ensure the documentation does not promise constraints that the model
does not enforce.
- Around line 179-183: Make the streaming_query SSE envelope consistent across
the generic schema and both end-event examples: either place available_quotas
inside the end event’s data object or update the schema description to
explicitly allow it at the top level, and apply the same shape to the additional
referenced examples.
In `@src/models/common/query.py`:
- Around line 18-20: Update the PNG entry in _IMAGE_SIGNATURES to use the
complete eight-byte PNG signature b"\x89PNG\r\n\x1a\n" instead of the four-byte
prefix, and adjust the existing PNG test fixture to include the complete
signature while preserving the JPEG behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 718e1ada-e0cb-453b-a77d-efbf8405aa75
📒 Files selected for processing (6)
docs/devel_doc/query_endpoint.mddocs/devel_doc/streaming_query_endpoint.mdsrc/models/common/query.pytests/e2e/features/query.featuretests/e2e/features/streaming_query.featuretests/unit/models/requests/test_attachment.py
📜 Review details
⏰ Context from checks skipped due to timeout. (15)
- GitHub Check: list_outdated_dependencies
- GitHub Check: build-pr
- GitHub Check: bandit
- GitHub Check: integration_tests (3.12)
- GitHub Check: integration_tests (3.13)
- GitHub Check: mypy
- GitHub Check: unit_tests (3.13)
- GitHub Check: unit_tests (3.12)
- GitHub Check: E2E: library mode / ci / group 3
- GitHub Check: E2E: library mode / ci / group 2
- GitHub Check: E2E: server mode / ci / group 2
- GitHub Check: E2E: server mode / ci / group 1
- GitHub Check: E2E: library mode / ci / group 1
- GitHub Check: E2E: server mode / ci / group 3
- GitHub Check: E2E Tests for Lightspeed Evaluation job
🧰 Additional context used
📓 Path-based instructions (5)
**/*
📄 CodeRabbit inference engine (Custom checks)
**/*: Flag meaningful O(n^2)+ algorithms on non-trivial inputs, including handlers and Kubernetes list operations.
Flag N+1 patterns that list items and then query once per item, including Kubernetes API and database access.
Flag expensive work inside loops, including API calls, JSON parsing, and regex compilation.
Flag unbounded growth in caches, watchers, or buffers when eviction or limits are missing.
Flag missing pagination or limits on list operations and API endpoints.
Flag secrets or tokens logged in plaintext or hardcoded in source.
Flag API endpoints missing authentication or authorization.
Flag injection vulnerabilities, including SQL injection, command injection, and path traversal.
Flag sensitive data leaked in API responses, WebSocket messages, or logs.
Flag Kubernetes Secrets and Red Hat secrets missing OwnerReferences.
Files:
docs/devel_doc/streaming_query_endpoint.mdsrc/models/common/query.pytests/unit/models/requests/test_attachment.pydocs/devel_doc/query_endpoint.mdtests/e2e/features/query.featuretests/e2e/features/streaming_query.feature
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Use absolute imports for internal Python modules.
All modules must begin with descriptive docstrings explaining their purpose.
Uselogger = get_logger(__name__)fromlog.pyfor module logging.
Use complete type annotations for function parameters and return types.
Use modern union syntax such asstr | int; useOptional[Type]for optional values.
Usetyping_extensions.Selffor model validators.
Functions must use descriptive, action-oriented snake_case names such asget_,validate_, andcheck_.
Avoid modifying mutable parameters in place; return a new data structure instead.
Useasync deffor I/O operations and external API calls.
HandleAPIConnectionErrorfrom Llama Stack.
All classes must have descriptive docstrings and complete type annotations for class attributes; use specific types instead ofAny.
Use PascalCase for classes and descriptive standard suffixes such asConfiguration,Error/Exception,Resolver, andInterface.
Use ABC and@abstractmethodfor abstract interfaces.
Follow Google Python docstring conventions; document all modules, classes, and functions, includingParameters,Returns,Raises, andAttributessections as applicable.
Never commit secrets or keys; use environment variables for sensitive data.
Files:
src/models/common/query.pytests/unit/models/requests/test_attachment.py
src/models/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/models/**/*.py: ExtendConfigurationBasefor configuration models andBaseModelfor data models.
Use@model_validatorand@field_validatorfor Pydantic model validation.
Files:
src/models/common/query.py
tests/unit/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
tests/unit/**/*.py: Use pytest for all unit tests; do not use unittest.
Useconftest.pyfor shared fixtures,pytest-mockforAsyncMockobjects, andpytest.mark.asynciofor asynchronous tests.
Files:
tests/unit/models/requests/test_attachment.py
tests/e2e/features/**/*.feature
📄 CodeRabbit inference engine (AGENTS.md)
End-to-end scenarios must use behave’s Gherkin feature-file format.
Files:
tests/e2e/features/query.featuretests/e2e/features/streaming_query.feature
🧠 Learnings (5)
📚 Learning: 2026-01-12T10:58:40.230Z
Learnt from: blublinsky
Repo: lightspeed-core/lightspeed-stack PR: 972
File: src/models/config.py:459-513
Timestamp: 2026-01-12T10:58:40.230Z
Learning: In lightspeed-core/lightspeed-stack, for Python files under src/models, when a user claims a fix is done but the issue persists, verify the current code state before accepting the fix. Steps: review the diff, fetch the latest changes, run relevant tests, reproduce the issue, search the codebase for lingering references to the original problem, confirm the fix is applied and not undone by subsequent commits, and validate with local checks to ensure the issue is resolved.
Applied to files:
src/models/common/query.py
📚 Learning: 2026-02-25T07:46:33.545Z
Learnt from: asimurka
Repo: lightspeed-core/lightspeed-stack PR: 1211
File: src/models/responses.py:8-16
Timestamp: 2026-02-25T07:46:33.545Z
Learning: In the Python codebase, requests.py should use OpenAIResponseInputTool as Tool while responses.py uses OpenAIResponseTool as Tool. This difference is intentional due to differing schemas for input vs output tools in llama-stack-api. Apply this distinction consistently to other models under src/models (e.g., ensure request-related tools use the InputTool variant and response-related tools use the ResponseTool variant). If adding new tools, choose the corresponding InputTool or Tool class based on whether the tool represents input or output, and document the rationale in code comments.
Applied to files:
src/models/common/query.py
📚 Learning: 2026-06-24T13:45:37.249Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 1971
File: src/utils/markdown_repair.py:31-36
Timestamp: 2026-06-24T13:45:37.249Z
Learning: In the lightspeed-stack repository, docstrings must use the section header name "Parameters:" (not "Args:") for function arguments, even if the project references Google Python docstring conventions. Ensure docstrings follow the project’s established "Parameters:" header format for any documented function parameters.
Applied to files:
src/models/common/query.pytests/unit/models/requests/test_attachment.py
📚 Learning: 2026-07-06T15:26:18.398Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 2071
File: src/models/config.py:2416-2422
Timestamp: 2026-07-06T15:26:18.398Z
Learning: In this repo’s Python code under src/**, don’t treat differences in string concatenation style as a style inconsistency when Black has effectively forced (or made clearer) use of explicit `+` string concatenation in multi-line logger/string expressions. If adjacent-literal implicit concatenation is avoided/changed specifically to accommodate Black’s formatting in these call sites, accept the `+` usage and don’t recommend converting it solely for consistency with nearby blocks that use implicit concatenation.
Applied to files:
src/models/common/query.py
📚 Learning: 2026-07-17T19:25:05.325Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 2166
File: src/utils/saved_prompts.py:129-157
Timestamp: 2026-07-17T19:25:05.325Z
Learning: For any endpoint that handles saved prompts and calls `src/utils/saved_prompts.py::create_saved_prompt`, treat the endpoint as the validation boundary. Before calling `create_saved_prompt`, validate the incoming saved-prompt name and content, specifically using `validate_saved_prompt_name` and then persist (store) the normalized value it returns. Do not call `create_saved_prompt` with unvalidated/raw name/content.
Applied to files:
src/models/common/query.py
🪛 markdownlint-cli2 (0.23.0)
docs/devel_doc/query_endpoint.md
[warning] 181-181: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔇 Additional comments (8)
docs/devel_doc/query_endpoint.md (1)
181-181: Specify the fenced code-block language.This is the same unresolved markdown-lint issue from the previous review; add
text(or another suitable language) to the opening fence.Source: Linters/SAST tools
docs/devel_doc/streaming_query_endpoint.md (1)
1-5: LGTM!tests/e2e/features/query.feature (2)
160-182: Assert that the attachments were actually processed.
imageanderrorare generic prompt/attachment terms; these assertions can pass when vision processing is unavailable or ignored. Assert a deterministic, attachment-specific result instead.Also applies to: 240-268
183-239: LGTM!Also applies to: 269-269
tests/e2e/features/streaming_query.feature (2)
149-172: Assert that the attachments were actually processed.Stream completion is now awaited, but generic
imageanderrorfragments still allow a passing response that did not consume the supplied image.Also applies to: 230-259
173-229: LGTM!src/models/common/query.py (1)
100-100: LGTM!tests/unit/models/requests/test_attachment.py (1)
109-128: LGTM!
| **Allowed `attachment_type` values:** `"alert"`, `"api object"`, `"configuration"`, `"error message"`, `"event"`, `"image"`, `"log"`, `"stack trace"` | ||
|
|
||
| **Allowed `content_type` values:** `"text/plain"`, `"application/json"`, `"application/yaml"`, `"application/xml"`, `"image/jpeg"`, `"image/png"` | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Align the documented attachment allowlists with runtime validation.
The supplied Attachment model declares attachment_type and content_type as plain str fields; its validator only checks image consistency, base64 validity, size, and magic bytes. Unless another validator enforces these lists, unsupported values are accepted despite this documentation claiming they are allowed values and return HTTP 422 when invalid. Add explicit validation or remove the allowlist/422 claims.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/devel_doc/query_endpoint.md` around lines 92 - 95, Align the attachment
documentation with actual runtime behavior: either add explicit validation for
the documented attachment_type and content_type allowlists in the Attachment
model and preserve the HTTP 422 claim, or remove the allowlist and invalid-value
422 claims if no such validation is intended. Ensure the documentation does not
promise constraints that the model does not enforce.
| Image attachments (`content_type` of `"image/jpeg"` or `"image/png"`) are handled differently from text attachments: | ||
|
|
||
| - `attachment_type` must be `"image"` when `content_type` is an image MIME type, and vice versa. Mismatches are rejected with HTTP 422. | ||
| - `content` must be valid base64-encoded image data. Invalid base64 is rejected with HTTP 422. | ||
| - Decoded image size must not exceed **100 MB** (104,857,600 bytes). | ||
| - Image attachments are **not** appended to the query text. Instead, they are passed as structured multimodal input to the LLM, each converted to a data URL (`data:{content_type};base64,{content}`). |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Document the required image signature validation.
Image content is also checked against its declared MIME type via _validate_image_magic_bytes(...) in src/models/common/query.py (Lines 42-102). State that decoded bytes must match the declared image/jpeg or image/png signature; valid base64 with the wrong file format is rejected.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/devel_doc/query_endpoint.md` around lines 100 - 105, Update the image
attachment documentation near the existing base64 and MIME-type rules to state
that decoded bytes must match the declared image/jpeg or image/png file
signature, as validated by _validate_image_magic_bytes. Specify that valid
base64 with a mismatched file format is rejected with HTTP 422.
| `POST /v1/streaming_query` returns an SSE stream. Each event is a line of the form: | ||
|
|
||
| ``` | ||
| data: {"event": "<event_type>", "data": {<payload>}} | ||
| ``` |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Make the SSE envelope consistent with the end event examples.
The generic format says all payload is inside data, but both end examples place available_quotas at the top level. Update the schema description or move available_quotas into data so clients have one unambiguous event shape.
Also applies to: 243-258, 503-503
🧰 Tools
🪛 markdownlint-cli2 (0.23.0)
[warning] 181-181: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/devel_doc/query_endpoint.md` around lines 179 - 183, Make the
streaming_query SSE envelope consistent across the generic schema and both
end-event examples: either place available_quotas inside the end event’s data
object or update the schema description to explicitly allow it at the top level,
and apply the same shape to the additional referenced examples.
| _IMAGE_SIGNATURES: dict[str, bytes] = { | ||
| "image/png": b"\x89PNG", | ||
| "image/jpeg": b"\xff\xd8\xff", |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Use PNG’s complete eight-byte signature.
b"\x89PNG" accepts payloads such as b"\x89PNGnot-an-image". Require b"\x89PNG\r\n\x1a\n" and update the existing PNG test fixture accordingly.
Proposed fix
- "image/png": b"\x89PNG",
+ "image/png": b"\x89PNG\r\n\x1a\n",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| _IMAGE_SIGNATURES: dict[str, bytes] = { | |
| "image/png": b"\x89PNG", | |
| "image/jpeg": b"\xff\xd8\xff", | |
| _IMAGE_SIGNATURES: dict[str, bytes] = { | |
| "image/png": b"\x89PNG\r\n\x1a\n", | |
| "image/jpeg": b"\xff\xd8\xff", |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/models/common/query.py` around lines 18 - 20, Update the PNG entry in
_IMAGE_SIGNATURES to use the complete eight-byte PNG signature
b"\x89PNG\r\n\x1a\n" instead of the four-byte prefix, and adjust the existing
PNG test fixture to include the complete signature while preserving the JPEG
behavior.
3 participants
Description
This is the documentation and e2e updates for a previous merged PR #2108
Type of change
Tools used to create PR
Identify any AI code assistants used in this PR (for transparency and review context)
Related Tickets & Documents
Related Issue # https://redhat.atlassian.net/browse/LCORE-2917
Closes # https://redhat.atlassian.net/browse/LCORE-2917
Checklist before requesting a review
Testing
Summary by CodeRabbit
Documentation
Bug Fixes
Tests