Skip to content

Retrieving the value of skipped steps (using withParam) causes the entire workflow to hang, but it doesn't throw an error #16793

Description

@yajun-L

Pre-requisites

  • I have double-checked my configuration
  • I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
  • I have searched existing issues and could not find a match for this bug
  • I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

I'm using withParam to start a parallel task and when to skip a certain step. When I try to get the merged result of this skipped step, the entire workflow gets stuck.
Image
I tried using output.valueFrom.default to set a default value for the output, but it didn't work. In version 3.7.1, I could merge values ​​from different steps like this way, but it's causing problems in version 4.1.1.

arguments:
  parameters:
    - name: in
      value: >-
        {{=concat(
        string(tasks['test-producer'].outputs.parameters) == '[]' ? [] : fromJSON(tasks['test-producer'].outputs.parameters.result),
        string(tasks['test-producer2'].outputs.parameters) == '[]' ? [] : fromJSON(tasks['test-producer2'].outputs.parameters.result),
        )}}

Version(s)

v4.1.1

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: producer-output-default-
spec:
  entrypoint: main
  arguments:
    parameters:
      - name: inputs
        value: >-
          [
            {"result": "123fds", "classify": "fungi"},
            {"result": "235dsf", "classify": "fungi"},
            {"result": "654cxz", "classify": "bacteria"}
          ]
  templates:
    - name: main
      inputs:
        parameters:
          - name: inputs
      dag:
        tasks:
          - name: test-producer
            template: produce
            arguments:
              parameters:
                - name: query
                  value: "{{item}}"
            when: "{{=item.classify}} == fungi"
            withParam: "{{inputs.parameters.inputs}}"
            # when: "false"
          - name: test-producer2
            template: produce
            arguments:
              parameters:
                - name: query
                  value: "{{item}}"
            when: "{{=item.classify}} == euk"
            withParam: "{{inputs.parameters.inputs}}"
            # when: "true"
          - name: consumer
            template: consume
            depends: "test-producer || test-producer2"
            arguments:
              parameters:
                - name: in
                  value: "{{tasks.test-producer2.outputs.parameters.msg}}"

    - name: produce
      inputs:
        parameters:
          - name: query
      outputs:
        parameters:
          - name: msg
            valueFrom:
              path: /tmp/out.txt
              default: "hello-from-producer"   # <- used when the node is skipped/omitted
      container:
        image: alpine:3.22.1
        command:
          - sh
          - -c
        args: ["echo", "real-value: [{{inputs.parameters.query}}]", "/tmp/out.txt"]

    - name: consume
      inputs:
        parameters:
          - name: in
      container:
        image: alpine:3.22.1
        command:
          - sh
          - -c
        args: ["echo", "consumer got: [{{inputs.parameters.in}}]"]

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}


time=2026-08-21T08:57:15.116Z level=INFO msg="Processing workflow" component=workflow_worker namespace=liangyj workflow=producer-output-default-6tww5 phase=Running resourceVersion=126141663
time=2026-08-21T08:57:15.119Z level=INFO msg="Task-result reconciliation" component=workflow_worker namespace=liangyj workflow=producer-output-default-6tww5 numObjs=2
time=2026-08-21T08:57:15.120Z level=WARN msg="was unable to obtain the node" namespace=liangyj workflow=producer-output-default-6tww5 component=workflow_worker nodeID=producer-output-default-6tww5-2134133241 taskName=consumer
time=2026-08-21T08:57:15.120Z level=WARN msg="was unable to obtain the node" taskName=consumer component=workflow_worker namespace=liangyj workflow=producer-output-default-6tww5 nodeID=producer-output-default-6tww5-2134133241
time=2026-08-21T08:57:15.121Z level=WARN msg="was unable to obtain the node" nodeID=producer-output-default-6tww5-2134133241 taskName=consumer component=workflow_worker namespace=liangyj workflow=producer-output-default-6tww5
time=2026-08-21T08:57:15.122Z level=WARN msg="was unable to find variable" component=workflow_worker namespace=liangyj workflow=producer-output-default-6tww5 error="failed to resolve {{tasks.test-producer2.outputs.parameters.msg}}"
time=2026-08-21T08:57:15.122Z level=WARN msg="was unable to obtain the node" taskName=consumer workflow=producer-output-default-6tww5 component=workflow_worker namespace=liangyj nodeID=producer-output-default-6tww5-2134133241
time=2026-08-21T08:57:15.122Z level=INFO msg="TaskSet Reconciliation" namespace=liangyj workflow=producer-output-default-6tww5 component=workflow_worker
time=2026-08-21T08:57:15.122Z level=INFO msg=reconcileAgentPod component=workflow_worker namespace=liangyj workflow=producer-output-default-6tww5

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions