fix: DSL2 optional-output syntax for Nextflow 25.x + clamp version#10
Merged
Conversation
Nextflow 25.x parses the bareword process-output form
`path "<glob>" optional true` as `path("<glob>").optional(true)`,
where the inner call returns null, throwing at script-load/include
time:
java.lang.NullPointerException: Cannot invoke method optional() on null object
at BaseScript.process(BaseScript.groovy:136)
Because this fires while parsing an include, one offending module
aborts the whole pipeline before any workflow runs — even modules
whose paths are never executed.
Convert every bareword occurrence to the modern comma+colon form
(`, optional: true`) and clamp the required Nextflow version to the
runner (25.04+).
Files:
- modules/local/msstats/main.nf:18
- modules/local/msstatstmt/main.nf:18
- modules/local/openms/proteomicslfq/main.nf:20-27 (8 outputs)
- nextflow.config:446 nextflowVersion '!>=23.04.0' -> '!>=25.04.0'
Verified with `nextflow inspect main.nf -profile test,docker` on
Nextflow 25.10.4: full include graph parses clean (41 processes
resolved, no optional() NPE).
Python linting (
|
Collaborator
Author
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
Our AWS Batch runner image was bumped to
nextflow/nextflow:25.10.4(required soenv()works in the SCM config). On 25.x, Nextflow parses the bareword DSL2 process-output formas
path("<glob>").optional(true). The inner call now returnsnull, so.optional(true)throws at script-load / include time:Because this fires while parsing an include (not at runtime), a single offending module aborts the entire pipeline before any workflow runs — even modules for paths we never execute. Confirmed originally on rev
de14f523: first failure atmodules/local/msstatstmt/main.nf(path "*.pdf" optional true), which killed a DIA/LFQ job that never intended to touch TMT.It worked before only because the old runner ran an older Nextflow.
Fix
Convert every bareword occurrence to the modern comma+colon form (
, optional: true). Lines already using, optional: true(multiqc, pmultiqc, proteinquantifier) were left alone.modules/local/msstats/main.nfpath "*.pdf" optional true→path "*.pdf", optional: truemodules/local/msstatstmt/main.nfpath "*.pdf" optional true→path "*.pdf", optional: truemodules/local/openms/proteomicslfq/main.nf... emit: X optional true→... emit: X, optional: truenextflow.confignextflowVersion = '!>=23.04.0'→nextflowVersion = '!>=25.04.0'The version clamp matches the runner;
!enforces it.Verification
Under Nextflow 25.10.4,
nextflow inspect main.nf -profile test,dockerloads the full include graph and resolves cleanly — nooptional()NPE. 41 processes resolved, including the previously-brokenMSSTATS,MSSTATSTMT,PROTEOMICSLFQ, plus all TMT/DIA-NN modules.Direct before/after on the offending module (25.10.4):
nextflow inspectoutput (head):{ "processes": [ { "name": "SILICOLIBRARYGENERATION", "container": "docker.io/biocontainers/diann:v1.8.1_cv1" }, { "name": "SEARCHENGINESAGE", "container": "quay.io/biocontainers/openms-thirdparty:3.1.0--h9ee0642_1" }, ... { "name": "MSSTATS", ... }, { "name": "PROTEOMICSLFQ", ... }, { "name": "MSSTATSTMT", ... }, { "name": "DIANNSUMMARY", "container": "docker.io/biocontainers/diann:v1.8.1_cv1" } ] }exit=0, zero errors/exceptions in stderr.
Scope
Fork-only — not pushed upstream (nf-core / bigbio). This fork carries DIA-NN in-workflow, which upstream dropped.