Skip to content

TraceFileObserver NPE on every task when trace file already exists #6890

@robsyme

Description

@robsyme

Bug report

Expected behavior and actual behavior

Expected: When a trace file already exists (e.g. on S3) and trace.overwrite is not enabled, Nextflow should log a clear warning that the trace file could not be created and continue the pipeline run gracefully.

Actual: TraceFileObserver.onFlowCreate() throws AbortOperationException (via TraceHelper.newFileWriter()), which Session.notifyEvent() catches and logs at debug level only. Because the exception occurs before the writer Agent is initialized, writer remains null. Every subsequent call to onTaskComplete(), onTaskCached(), and onFlowComplete() then throws a NullPointerException, producing a stack trace for every task in the run:

Cannot invoke "groovyx.gpars.agent.Agent.send(Object)" because "this.writer" is null
java.lang.NullPointerException: Cannot invoke "groovyx.gpars.agent.Agent.send(Object)" because "this.writer" is null
    at nextflow.trace.TraceFileObserver.onTaskComplete(TraceFileObserver.groovy:219)

Steps to reproduce the problem

  1. Run a Nextflow pipeline with trace.enabled = true and trace.file pointing to S3 (or any path where the file already exists).
  2. Do not set trace.overwrite = true.
  3. Run the pipeline a second time without removing the trace file.
// nextflow.config
trace {
    enabled = true
    file = 's3://my-bucket/results/trace.tsv'
    // overwrite = true  // intentionally omitted
}
// main.nf
process FOO {
    input: val x
    output: stdout
    script: "echo $x"
}

workflow {
    Channel.of(1, 2, 3) | FOO
}

Run once to create the trace file, then run again to trigger the bug.

Program output

On the second run, the .nextflow.log contains an NPE for every task:

Cannot invoke "groovyx.gpars.agent.Agent.send(Object)" because "this.writer" is null
java.lang.NullPointerException: Cannot invoke "groovyx.gpars.agent.Agent.send(Object)" because "this.writer" is null
    at nextflow.trace.TraceFileObserver.onTaskComplete(TraceFileObserver.groovy:219)
    ...
Cannot invoke "groovyx.gpars.agent.Agent.send(Object)" because "this.writer" is null
java.lang.NullPointerException: Cannot invoke "groovyx.gpars.agent.Agent.send(Object)" because "this.writer" is null
    at nextflow.trace.TraceFileObserver.onFlowComplete(TraceFileObserver.groovy:189)
    ...

The original AbortOperationException with the helpful message ("Trace file already exists ... enable the 'trace.overwrite' option") is only logged at debug level by Session.notifyEvent(), so users never see it with default log settings.

Environment

  • Nextflow version: 25.10.2 (also present on current master)
  • Java version: 21
  • Operating system: Any (observed on Linux with S3 trace path)

Additional context

Root cause: In TraceFileObserver.onFlowCreate(), TraceHelper.newFileWriter() throws when the file exists. The exception is caught by Session.notifyEvent() and logged at debug level. But writer (an Agent<PrintWriter>) was never assigned, so every subsequent method that calls writer.send { ... } or writer.await() throws an NPE.

The four affected call sites in TraceFileObserver:

  • onFlowCreate line 178: writer.send { ... } (header write)
  • onFlowComplete line 189: writer.await()
  • onTaskComplete line 219: writer.send { ... }
  • onTaskCached line 233: writer.send { ... }

Fix: Catch the exception in onFlowCreate and log at warn level so the user sees it, return early, and add null-safe (?.) guards on writer at the remaining call sites so the observer degrades gracefully.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions