feat(lapis): add sequence position fields syntax to aggregated endpoint

[fhennig]

Summary

• Adds SequenceName[position] syntax (e.g. S[501]) to the fields parameter of /aggregated, allowing users to request specific sequence positions as group-by columns
• Shorthand [N] is supported for single-segmented genomes and resolves to the single nucleotide sequence
• Sequence names are validated case-insensitively against both genes and nucleotide segments from the reference genome
• /details, phylo tree, and MRCA endpoints reject position fields at the type level (they only ever construct a PlainField), with a clear error message
• GET /aggregated's fields query parameter now documents the position-field syntax too, matching what the POST body schema already described
• Generates a .map({"S[501]":="S".at(501)}) SaneQL step before .groupBy(...) when position fields are present — the user-facing name is used directly as the SaneQL alias, so no post-processing is needed
• Response column keys mirror the input exactly: S[501] in → S[501] out, [501] in → [501] out
• Internal cleanup: split the field/orderBy converters and cleaners out of Field.kt/OrderByField.kt into a dedicated request.converter package, renamed the sealed field hierarchy for clarity (RequestField → Field, Field → PlainField), and renamed SequenceFiltersRequestWithFields → AggregatedFiltersRequest to match its /aggregated-only sibling naming (DetailsFiltersRequest, PhyloTreeSequenceFiltersRequest, MRCASequenceFiltersRequest)

Example

POST /sample/aggregated
{"fields": ["country", "S[501]"]}

Generates:

default.filter(true).map({"S[501]":="S".at(501)}).groupBy({"count":=count()}, {"country", "S[501]"})

Test plan

• Unit tests for SequenceName[position] parsing in AggregatedFiltersRequestTest
• Error cases: unknown sequence name, position ≤ 0, shorthand on multi-segmented genome
• SaneQL generation tests in SiloQueryToSaneQlTest for position-only and mixed fields
• e2e coverage for a gene (amino acid) position, a plain nucleotide position, and the single-segment shorthand syntax
• Full test suite passes

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
lapis	Ready	Preview, Comment	Jul 27, 2026 2:36pm

[Copilot]

Pull request overview

Adds support for requesting specific sequence positions (e.g. S[501], and [501] for single-segment genomes) as group-by columns in the /aggregated endpoint by parsing them from the fields request parameter and generating the appropriate SaneQL .map(...).groupBy(...) pipeline.

Changes:

• Introduces RequestField with a new SequencePositionField type and updates request parsing to recognize Name[N] / [N] syntax.
• Extends aggregated SaneQL generation to add a .map(...) step that computes requested positions and includes them in .groupBy(...).
• Updates SaneQL AST rendering/tests to quote assignment keys (e.g. {"count":=count()}) and adds unit coverage for new parsing/generation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
lapis/src/main/kotlin/org/genspectrum/lapis/request/Field.kt	Adds RequestField + SequencePositionField, and extends field conversion to parse/validate position syntax against the reference genome schema.
lapis/src/main/kotlin/org/genspectrum/lapis/request/SequenceFiltersRequestWithFields.kt	Updates request model to store fields as List<RequestField> and keeps deserialization via the field converter.
lapis/src/main/kotlin/org/genspectrum/lapis/silo/SiloQuery.kt	Adds sequencePositionFields to aggregated actions and emits .map(...) before .groupBy(...) when present.
lapis/src/main/kotlin/org/genspectrum/lapis/silo/SaneQlAst.kt	Changes assignment rendering to quote/escape keys (e.g. "name":=value) to align with SaneQL expectations and harden output.
lapis/src/main/kotlin/org/genspectrum/lapis/model/SiloQueryModel.kt	Splits RequestField values into metadata fields vs. sequence-position fields for aggregated query construction.
lapis/src/main/kotlin/org/genspectrum/lapis/controller/LapisController.kt	Ensures response headers/field lists use outputColumnName (aggregated) and filters non-metadata fields out of details fields handling.
lapis/src/main/kotlin/org/genspectrum/lapis/controller/ControllerDescriptions.kt	Updates aggregated endpoint parameter description to document the new position-field syntax.
lapis/src/main/kotlin/org/genspectrum/lapis/request/PhyloTreeSequenceFiltersRequest.kt	Adjusts phylo-tree parsing to work with the generalized FieldConverter<RequestField> while enforcing a Field result.
lapis/src/test/kotlin/org/genspectrum/lapis/request/SequenceFiltersRequestWithFieldsTest.kt	Adds unit tests for parsing valid/invalid Name[N] / [N] field forms.
lapis/src/test/kotlin/org/genspectrum/lapis/silo/SiloQueryToSaneQlTest.kt	Updates expected SaneQL strings and adds coverage for aggregated SaneQL with sequence position fields.
lapis/src/test/kotlin/org/genspectrum/lapis/silo/SaneQlAstTest.kt	Updates AST rendering assertions to match quoted assignment output.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[fhennig]

Ok, something weird with the test? But still reviewable I think. I'll look into it.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 41 out of 41 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

lapis/src/main/kotlin/org/genspectrum/lapis/silo/SaneQlAst.kt:74

• The SaneQlList KDoc above still uses the pre-change assignment example ({count:=count()}), but SaneQlAssignment now renders as {"count":=count()}. Update the example to match the new rendered syntax to avoid misleading documentation.

    override fun render() = "{" + items.joinToString(", ") { it.render() } + "}"
}

/** A `"name":=value` assignment, used inside [SaneQlList]s that act as records, e.g. `{"count":=count()}`. */
data class SaneQlAssignment(

lapis/src/main/kotlin/org/genspectrum/lapis/request/converter/SequencePositionFieldConverter.kt:8

• SEQUENCE_POSITION_REGEX is more restrictive than other sequence-name parsers (e.g., mutations allow - via [a-zA-Z0-9_-]+). This prevents using position fields for valid segment/gene names that contain hyphens (or start with digits), even though the reference genome schema does not restrict those names.

private val SEQUENCE_POSITION_REGEX = Regex("""^([A-Za-z][A-Za-z0-9_]*)?\[(\d+)]$""")

lapis/src/main/kotlin/org/genspectrum/lapis/request/converter/PlainFieldConverter.kt:21

• PlainFieldConverter aims to reject sequence position fields with a dedicated error, but tryConvert() can throw (e.g., unknown sequence or shorthand on multi-segment genomes), which bypasses this message and surfaces a different one. Catch those cases and consistently rethrow the “not supported here” error for any input that matches position-field syntax.

    override fun convert(source: String): PlainField {
        sequencePositionFieldConverter.tryConvert(source)?.let {
            throw BadRequestException(
                "Sequence position fields are not supported here: ${it.outputColumnName}",
            )
        }
        return metadataFieldConverter.convert(source)
    }

[fengelniederhammer]

👍

[fengelniederhammer]

Then let's maybe change the original description in the annotation in the controller? Either remove it or make it say "I'm just a placeholder that will be overwritten later"?

[fengelniederhammer]

I just noticed that we have a SampleEndpointsGetParameterCustomizer, maybe we should put it there?

Also the SwaggerUI of the multi segmented local instance doesn't load. Generating the API docs fails:

2026-07-28 08:53:57,290 WARN [http-nio-8094-exec-4] [4bcc64a3-e491-4a89-8278-d33272b0fb65] org.genspectrum.lapis.controller.ExceptionHandler: Caught unexpected exception: getContent(...) must not be null
java.lang.NullPointerException: getContent(...) must not be null
        at org.genspectrum.lapis.openApi.OperationsSorter.customize$lambda$0(OperationsSorter.kt:18)
        at org.genspectrum.lapis.openApi.OperationsSorter.customize$lambda$1(OperationsSorter.kt:16)
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:986)
        at org.genspectrum.lapis.openApi.OperationsSorter.customize(OperationsSorter.kt:16)