Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dag Template does not work properly when executing Workflow level Retry settings and Hook settings. #14058

Open
4 tasks done
wjdwogjs999 opened this issue Jan 8, 2025 · 2 comments · May be fixed by #14060
Open
4 tasks done

Dag Template does not work properly when executing Workflow level Retry settings and Hook settings. #14058

wjdwogjs999 opened this issue Jan 8, 2025 · 2 comments · May be fixed by #14060
Labels
area/controller Controller issues, panics area/hooks area/retryStrategy Template-level retryStrategy type/bug

Comments

@wjdwogjs999
Copy link

Pre-requisites

  • I have double-checked my configuration
  • I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
  • I have searched existing issues and could not find a match for this bug
  • I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

We created a workflow by setting Retry and Hook at the Workflow level in the DAG Task example provided by Argo Workflow. As the DAG template did not operate properly, the first DAG template attempted did not fail, but a new DAG template was executed as a Running Hook child node. Additionally, although the Workflow status has become Success, the status of the DAG template that was initially started and B and C continue to remain in the Running status and do not change.

image

Version(s)

latest, 3.6.2

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-workflow-retry-and-hook
spec:
  entrypoint: diamond
  hooks:
    exit:
      template: notify
    running:
      expression: workflow.status == "Running"
      template: notify
  retryStrategy:
    limit: 1
  templates:
    - name: diamond
      dag:
        tasks:
        - name: A
          template: echo
          arguments:
            parameters: [{name: message, value: A}]
        - name: B
          dependencies: [A]
          template: echo
          arguments:
            parameters: [{name: message, value: B}]
        - name: C
          dependencies: [A]
          template: echo
          arguments:
            parameters: [{name: message, value: C}]
        - name: D
          dependencies: [B, C]
          template: main
          arguments:
            parameters: [{name: message, value: D}]
    - name: echo
      inputs:
        parameters:
        - name: message
      container:
        image: alpine:3.7
        command: [echo, "{{inputs.parameters.message}}"]
    - name: notify
      script:
        image: alpine
        command: [sh]
        source: |
          echo "notifying workflow is {{workflow.status}}..."
    - name: main
      script:
        image: alpine
        command: [sh]
        source: |
          sleep 5
          echo "raising an error..."

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

{"Phase":"","ResourceVersion":"1565081387","level":"info","msg":"Processing workflow","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.810Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Task-result reconciliation","namespace":"flops-wf-dev","numObjs":0,"time":"2025-01-08T08:18:02.818Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Updated phase  -\u003e Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.818Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"Node was nil, will be initialized as type Skipped","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.818Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"was unable to obtain node for , letting display name to be nodeName","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.820Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Retry node dag-workflow-retry-and-hook4lxph initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.820Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"was unable to obtain node for , letting display name to be nodeName","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.820Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"DAG node dag-workflow-retry-and-hook4lxph-4071813683 initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.821Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1090616921, taskName D","time":"2025-01-08T08:18:02.821Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1124172159, taskName B","time":"2025-01-08T08:18:02.821Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1140949778, taskName A","time":"2025-01-08T08:18:02.821Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1140949778, taskName A","time":"2025-01-08T08:18:02.821Z"}
{"level":"info","msg":"All of node dag-workflow-retry-and-hook4lxph(0).A dependencies [] completed","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.821Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"Node was nil, will be initialized as type Skipped","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.821Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Retry node dag-workflow-retry-and-hook4lxph-1140949778 initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.823Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Pod node dag-workflow-retry-and-hook4lxph-650517257 initialized Pending","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.823Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Created pod: dag-workflow-retry-and-hook4lxph(0).A(0) (dag-workflow-retry-and-hook4lxph-echo-650517257)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.933Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1124172159, taskName B","time":"2025-01-08T08:18:02.933Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1107394540, taskName C","time":"2025-01-08T08:18:02.933Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1107394540, taskName C","time":"2025-01-08T08:18:02.933Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1090616921, taskName D","time":"2025-01-08T08:18:02.933Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1124172159, taskName B","time":"2025-01-08T08:18:02.933Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-1090616921, taskName D","time":"2025-01-08T08:18:02.933Z"}
{"level":"info","lifeCycleHook":"running","msg":"Running workflow level hooks","namespace":"flops-wf-dev","node":"dag-workflow-retry-and-hook4lxph.hooks.running","time":"2025-01-08T08:18:02.934Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"Node was nil, will be initialized as type Skipped","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.934Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"was unable to obtain node for , letting display name to be nodeName","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.935Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Retry node dag-workflow-retry-and-hook4lxph-310273811 initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.935Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"was unable to obtain node for , letting display name to be nodeName","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.935Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Pod node dag-workflow-retry-and-hook4lxph-2635142686 initialized Pending","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.935Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"couldn't get boundaryTemplate through nodeName dag-workflow-retry-and-hook4lxph.hooks.running(0)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:02.936Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Created pod: dag-workflow-retry-and-hook4lxph.hooks.running(0) (dag-workflow-retry-and-hook4lxph-notify-2635142686)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:03.042Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"TaskSet Reconciliation","namespace":"flops-wf-dev","time":"2025-01-08T08:18:03.042Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"reconcileAgentPod","namespace":"flops-wf-dev","time":"2025-01-08T08:18:03.042Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Workflow update successful","namespace":"flops-wf-dev","phase":"Running","resourceVersion":"1565081400","time":"2025-01-08T08:18:03.073Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"Phase":"Running","ResourceVersion":"1565081400","level":"info","msg":"Processing workflow","namespace":"flops-wf-dev","time":"2025-01-08T08:18:12.943Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Task-result reconciliation","namespace":"flops-wf-dev","numObjs":2,"time":"2025-01-08T08:18:12.947Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"task-result changed","namespace":"flops-wf-dev","nodeID":"dag-workflow-retry-and-hook4lxph-2635142686","time":"2025-01-08T08:18:12.947Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"task-result changed","namespace":"flops-wf-dev","nodeID":"dag-workflow-retry-and-hook4lxph-650517257","time":"2025-01-08T08:18:12.947Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Running","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-2635142686","old.message":"","old.phase":"Pending","old.progress":"0/1","time":"2025-01-08T08:18:12.947Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Running","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-650517257","old.message":"","old.phase":"Pending","old.progress":"0/1","time":"2025-01-08T08:18:12.948Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3951413369, taskName D","time":"2025-01-08T08:18:12.950Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3984968607, taskName B","time":"2025-01-08T08:18:12.950Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-4001746226, taskName A","time":"2025-01-08T08:18:12.950Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-4001746226, taskName A","time":"2025-01-08T08:18:12.950Z"}
{"level":"info","msg":"All of node dag-workflow-retry-and-hook4lxph.hooks.running.A dependencies [] completed","namespace":"flops-wf-dev","time":"2025-01-08T08:18:12.950Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"Node was nil, will be initialized as type Skipped","namespace":"flops-wf-dev","time":"2025-01-08T08:18:12.950Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Retry node dag-workflow-retry-and-hook4lxph-4001746226 initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:12.951Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"action":"terminateContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-notify-2635142686/terminateContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:12.953Z"}
{"action":"terminateContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-650517257/terminateContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:12.953Z"}
{"level":"info","msg":"Pod node dag-workflow-retry-and-hook4lxph-2043699369 initialized Pending","namespace":"flops-wf-dev","time":"2025-01-08T08:18:12.955Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Created pod: dag-workflow-retry-and-hook4lxph.hooks.running.A(0) (dag-workflow-retry-and-hook4lxph-echo-2043699369)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:13.054Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3984968607, taskName B","time":"2025-01-08T08:18:13.054Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3968190988, taskName C","time":"2025-01-08T08:18:13.054Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3968190988, taskName C","time":"2025-01-08T08:18:13.054Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3951413369, taskName D","time":"2025-01-08T08:18:13.055Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3984968607, taskName B","time":"2025-01-08T08:18:13.055Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3951413369, taskName D","time":"2025-01-08T08:18:13.055Z"}
{"level":"info","lifeCycleHook":"running","msg":"Running workflow level hooks","namespace":"flops-wf-dev","node":"dag-workflow-retry-and-hook4lxph.hooks.running","time":"2025-01-08T08:18:13.055Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"TaskSet Reconciliation","namespace":"flops-wf-dev","time":"2025-01-08T08:18:13.056Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"reconcileAgentPod","namespace":"flops-wf-dev","time":"2025-01-08T08:18:13.056Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Workflow update successful","namespace":"flops-wf-dev","phase":"Running","resourceVersion":"1565081681","time":"2025-01-08T08:18:13.107Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"Phase":"Running","ResourceVersion":"1565081681","level":"info","msg":"Processing workflow","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.061Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Task-result reconciliation","namespace":"flops-wf-dev","numObjs":3,"time":"2025-01-08T08:18:23.066Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"task-result changed","namespace":"flops-wf-dev","nodeID":"dag-workflow-retry-and-hook4lxph-2043699369","time":"2025-01-08T08:18:23.066Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Succeeded","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-2043699369","old.message":"","old.phase":"Pending","old.progress":"0/1","time":"2025-01-08T08:18:23.066Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Succeeded","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-650517257","old.message":"","old.phase":"Running","old.progress":"0/1","time":"2025-01-08T08:18:23.067Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Succeeded","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-2635142686","old.message":"","old.phase":"Running","old.progress":"0/1","time":"2025-01-08T08:18:23.067Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3951413369, taskName D","time":"2025-01-08T08:18:23.069Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3984968607, taskName B","time":"2025-01-08T08:18:23.069Z"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph-4001746226 phase Running -\u003e Succeeded","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.070Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph-4001746226 finished: 2025-01-08 08:18:23.07064124 +0000 UTC","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.070Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3984968607, taskName B","time":"2025-01-08T08:18:23.070Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3984968607, taskName B","time":"2025-01-08T08:18:23.071Z"}
{"level":"info","msg":"All of node dag-workflow-retry-and-hook4lxph.hooks.running.B dependencies [A] completed","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.071Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"Node was nil, will be initialized as type Skipped","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.071Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Retry node dag-workflow-retry-and-hook4lxph-3984968607 initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.072Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Pod node dag-workflow-retry-and-hook4lxph-3006597538 initialized Pending","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.072Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Created pod: dag-workflow-retry-and-hook4lxph.hooks.running.B(0) (dag-workflow-retry-and-hook4lxph-echo-3006597538)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.169Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3968190988, taskName C","time":"2025-01-08T08:18:23.169Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3968190988, taskName C","time":"2025-01-08T08:18:23.169Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3968190988, taskName C","time":"2025-01-08T08:18:23.170Z"}
{"level":"info","msg":"All of node dag-workflow-retry-and-hook4lxph.hooks.running.C dependencies [A] completed","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.170Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"Node was nil, will be initialized as type Skipped","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.170Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Retry node dag-workflow-retry-and-hook4lxph-3968190988 initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.171Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Pod node dag-workflow-retry-and-hook4lxph-4105830023 initialized Pending","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.171Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Created pod: dag-workflow-retry-and-hook4lxph.hooks.running.C(0) (dag-workflow-retry-and-hook4lxph-echo-4105830023)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.288Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3951413369, taskName D","time":"2025-01-08T08:18:23.288Z"}
{"level":"warning","msg":"was unable to obtain the node for dag-workflow-retry-and-hook4lxph-3951413369, taskName D","time":"2025-01-08T08:18:23.288Z"}
{"level":"info","lifeCycleHook":"running","msg":"Running workflow level hooks","namespace":"flops-wf-dev","node":"dag-workflow-retry-and-hook4lxph.hooks.running","time":"2025-01-08T08:18:23.288Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph-310273811 phase Running -\u003e Succeeded","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.288Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph-310273811 finished: 2025-01-08 08:18:23.28893434 +0000 UTC","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.288Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"TaskSet Reconciliation","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.289Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"reconcileAgentPod","namespace":"flops-wf-dev","time":"2025-01-08T08:18:23.289Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Workflow update successful","namespace":"flops-wf-dev","phase":"Running","resourceVersion":"1565081955","time":"2025-01-08T08:18:23.330Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"action":"labelPodCompleted","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-650517257/labelPodCompleted","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:23.338Z"}
{"action":"labelPodCompleted","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-notify-2635142686/labelPodCompleted","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:23.338Z"}
{"action":"labelPodCompleted","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-2043699369/labelPodCompleted","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:23.338Z"}
{"Phase":"Running","ResourceVersion":"1565081955","level":"info","msg":"Processing workflow","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.180Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Task-result reconciliation","namespace":"flops-wf-dev","numObjs":5,"time":"2025-01-08T08:18:33.187Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"task-result changed","namespace":"flops-wf-dev","nodeID":"dag-workflow-retry-and-hook4lxph-4105830023","time":"2025-01-08T08:18:33.187Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"task-result changed","namespace":"flops-wf-dev","nodeID":"dag-workflow-retry-and-hook4lxph-3006597538","time":"2025-01-08T08:18:33.187Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Running","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-4105830023","old.message":"","old.phase":"Pending","old.progress":"0/1","time":"2025-01-08T08:18:33.187Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Succeeded","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-3006597538","old.message":"","old.phase":"Pending","old.progress":"0/1","time":"2025-01-08T08:18:33.187Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph phase Running -\u003e Succeeded","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.189Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph finished: 2025-01-08 08:18:33.18932727 +0000 UTC","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.189Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","lifeCycleHook":"running","msg":"Running workflow level hooks","namespace":"flops-wf-dev","node":"dag-workflow-retry-and-hook4lxph.hooks.running","time":"2025-01-08T08:18:33.189Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"TaskSet Reconciliation","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.190Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"reconcileAgentPod","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.190Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Running OnExit handler: ","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.190Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"Node was nil, will be initialized as type Skipped","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.190Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"was unable to obtain node for , letting display name to be nodeName","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.191Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Retry node dag-workflow-retry-and-hook4lxph-3318121763 initialized Running","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.191Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"was unable to obtain node for , letting display name to be nodeName","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.191Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Pod node dag-workflow-retry-and-hook4lxph-960047726 initialized Pending","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.192Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"warning","msg":"couldn't get boundaryTemplate through nodeName dag-workflow-retry-and-hook4lxph.onExit(0)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.192Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"action":"terminateContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-4105830023/terminateContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:33.193Z"}
{"level":"info","msg":"Created pod: dag-workflow-retry-and-hook4lxph.onExit(0) (dag-workflow-retry-and-hook4lxph-notify-960047726)","namespace":"flops-wf-dev","time":"2025-01-08T08:18:33.307Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Workflow update successful","namespace":"flops-wf-dev","phase":"Running","resourceVersion":"1565082251","time":"2025-01-08T08:18:33.351Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"action":"labelPodCompleted","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-3006597538/labelPodCompleted","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:33.359Z"}
{"action":"killContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-notify-2635142686/killContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:42.954Z"}
{"action":"killContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-650517257/killContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:42.954Z"}
{"Phase":"Running","ResourceVersion":"1565082251","level":"info","msg":"Processing workflow","namespace":"flops-wf-dev","time":"2025-01-08T08:18:43.315Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Task-result reconciliation","namespace":"flops-wf-dev","numObjs":6,"time":"2025-01-08T08:18:43.322Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"task-result changed","namespace":"flops-wf-dev","nodeID":"dag-workflow-retry-and-hook4lxph-960047726","time":"2025-01-08T08:18:43.323Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Running","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-960047726","old.message":"","old.phase":"Pending","old.progress":"0/1","time":"2025-01-08T08:18:43.323Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Succeeded","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-4105830023","old.message":"","old.phase":"Running","old.progress":"0/1","time":"2025-01-08T08:18:43.323Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","lifeCycleHook":"running","msg":"Running workflow level hooks","namespace":"flops-wf-dev","node":"dag-workflow-retry-and-hook4lxph.hooks.running","time":"2025-01-08T08:18:43.325Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"TaskSet Reconciliation","namespace":"flops-wf-dev","time":"2025-01-08T08:18:43.325Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"reconcileAgentPod","namespace":"flops-wf-dev","time":"2025-01-08T08:18:43.325Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Running OnExit handler: ","namespace":"flops-wf-dev","time":"2025-01-08T08:18:43.325Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"action":"terminateContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-notify-960047726/terminateContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:43.328Z"}
{"level":"info","msg":"Workflow update successful","namespace":"flops-wf-dev","phase":"Running","resourceVersion":"1565082516","time":"2025-01-08T08:18:43.368Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"action":"labelPodCompleted","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-4105830023/labelPodCompleted","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:43.380Z"}
{"Phase":"Running","ResourceVersion":"1565082516","level":"info","msg":"Processing workflow","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.359Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Task-result reconciliation","namespace":"flops-wf-dev","numObjs":6,"time":"2025-01-08T08:18:53.366Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node changed","namespace":"flops-wf-dev","new.message":"","new.phase":"Succeeded","new.progress":"0/1","nodeID":"dag-workflow-retry-and-hook4lxph-960047726","old.message":"","old.phase":"Running","old.progress":"0/1","time":"2025-01-08T08:18:53.367Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","lifeCycleHook":"running","msg":"Running workflow level hooks","namespace":"flops-wf-dev","node":"dag-workflow-retry-and-hook4lxph.hooks.running","time":"2025-01-08T08:18:53.369Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"TaskSet Reconciliation","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.369Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"reconcileAgentPod","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.369Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Running OnExit handler: ","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.369Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph-3318121763 phase Running -\u003e Succeeded","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.370Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"node dag-workflow-retry-and-hook4lxph-3318121763 finished: 2025-01-08 08:18:53.370527138 +0000 UTC","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.370Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Updated phase Running -\u003e Succeeded","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.370Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Marking workflow completed","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.370Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Marking workflow as pending archiving","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.370Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"Workflow update successful","namespace":"flops-wf-dev","phase":"Succeeded","resourceVersion":"1565082738","time":"2025-01-08T08:18:53.451Z","workflow":"dag-workflow-retry-and-hook4lxph"}
{"level":"info","msg":"archiving workflow","namespace":"flops-wf-dev","time":"2025-01-08T08:18:53.547Z","uid":"16f96661-67d6-4715-bd2f-10f7201a1648","workflow":"dag-workflow-retry-and-hook4lxph"}
{"action":"labelPodCompleted","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-notify-960047726/labelPodCompleted","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:18:53.550Z"}
{"level":"info","msg":"Queueing Succeeded workflow flops-wf-dev/dag-workflow-retry-and-hook4lxph for delete in 10m0s due to TTL","time":"2025-01-08T08:18:53.638Z"}
{"action":"killContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-echo-4105830023/killContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:19:03.194Z"}
{"action":"killContainers","key":"flops-wf-dev/dag-workflow-retry-and-hook4lxph-notify-960047726/killContainers","level":"info","msg":"cleaning up pod","time":"2025-01-08T08:19:13.330Z"}

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

{"level":"info","msg":"No output parameters","time":"2025-01-08T08:18:30.391Z"}
{"level":"info","msg":"No output artifacts","time":"2025-01-08T08:18:30.391Z"}
{"level":"info","msg":"S3 Save path: /tmp/argo/outputs/logs/main.log, key: dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-4105830023/main.log","time":"2025-01-08T08:18:30.392Z"}
{"endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","level":"info","msg":"Creating minio client using static credentials","time":"2025-01-08T08:18:30.392Z"}
{"bucket":"flops-workflow","endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-4105830023/main.log","level":"info","msg":"Saving file to s3","path":"/tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:30.393Z"}
{"artifactName":"main-logs","duration":190976597,"error":null,"key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-4105830023/main.log","level":"info","msg":"Save artifact","time":"2025-01-08T08:18:30.583Z"}
{"level":"info","localArtPath":"/tmp/argo/outputs/logs/main.log","msg":"not deleting local artifact","time":"2025-01-08T08:18:30.583Z"}
{"level":"info","msg":"Successfully saved file: /tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:30.583Z"}
{"level":"info","msg":"Alloc=19043 TotalAlloc=23079 Sys=36437 NumGC=3 Goroutines=10","time":"2025-01-08T08:18:30.615Z"}
{"level":"info","msg":"Deadline monitor stopped","time":"2025-01-08T08:18:30.636Z"}
{"level":"info","msg":"No output parameters","time":"2025-01-08T08:18:10.634Z"}
{"level":"info","msg":"No output artifacts","time":"2025-01-08T08:18:10.635Z"}
{"level":"info","msg":"S3 Save path: /tmp/argo/outputs/logs/main.log, key: dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-650517257/main.log","time":"2025-01-08T08:18:10.725Z"}
{"endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","level":"info","msg":"Creating minio client using static credentials","time":"2025-01-08T08:18:10.725Z"}
{"bucket":"flops-workflow","endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-650517257/main.log","level":"info","msg":"Saving file to s3","path":"/tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:10.725Z"}
{"artifactName":"main-logs","duration":92188460,"error":null,"key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-650517257/main.log","level":"info","msg":"Save artifact","time":"2025-01-08T08:18:10.817Z"}
{"level":"info","localArtPath":"/tmp/argo/outputs/logs/main.log","msg":"not deleting local artifact","time":"2025-01-08T08:18:10.817Z"}
{"level":"info","msg":"Successfully saved file: /tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:10.817Z"}
{"level":"info","msg":"Alloc=18765 TotalAlloc=23081 Sys=32853 NumGC=3 Goroutines=10","time":"2025-01-08T08:18:10.878Z"}
{"level":"info","msg":"Deadline monitor stopped","time":"2025-01-08T08:18:10.891Z"}
{"level":"info","msg":"No output parameters","time":"2025-01-08T08:18:10.444Z"}
{"level":"info","msg":"No output artifacts","time":"2025-01-08T08:18:10.444Z"}
{"level":"info","msg":"S3 Save path: /tmp/argo/outputs/logs/main.log, key: dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-notify-2635142686/main.log","time":"2025-01-08T08:18:10.444Z"}
{"endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","level":"info","msg":"Creating minio client using static credentials","time":"2025-01-08T08:18:10.444Z"}
{"bucket":"flops-workflow","endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-notify-2635142686/main.log","level":"info","msg":"Saving file to s3","path":"/tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:10.445Z"}
{"artifactName":"main-logs","duration":152823591,"error":null,"key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-notify-2635142686/main.log","level":"info","msg":"Save artifact","time":"2025-01-08T08:18:10.597Z"}
{"level":"info","localArtPath":"/tmp/argo/outputs/logs/main.log","msg":"not deleting local artifact","time":"2025-01-08T08:18:10.597Z"}
{"level":"info","msg":"Successfully saved file: /tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:10.597Z"}
{"level":"info","msg":"Alloc=18363 TotalAlloc=23035 Sys=32341 NumGC=3 Goroutines=10","time":"2025-01-08T08:18:10.633Z"}
{"level":"info","msg":"Deadline monitor stopped","time":"2025-01-08T08:18:10.652Z"}
{"level":"info","msg":"No Script output reference in workflow. Capturing script output ignored","time":"2025-01-08T08:18:39.661Z"}
{"level":"info","msg":"No output parameters","time":"2025-01-08T08:18:39.661Z"}
{"level":"info","msg":"No output artifacts","time":"2025-01-08T08:18:39.661Z"}
{"level":"info","msg":"S3 Save path: /tmp/argo/outputs/logs/main.log, key: dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-notify-960047726/main.log","time":"2025-01-08T08:18:39.679Z"}
{"endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","level":"info","msg":"Creating minio client using static credentials","time":"2025-01-08T08:18:39.679Z"}
{"bucket":"flops-workflow","endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-notify-960047726/main.log","level":"info","msg":"Saving file to s3","path":"/tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:39.679Z"}
{"artifactName":"main-logs","duration":233108020,"error":null,"key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-notify-960047726/main.log","level":"info","msg":"Save artifact","time":"2025-01-08T08:18:39.912Z"}
{"level":"info","localArtPath":"/tmp/argo/outputs/logs/main.log","msg":"not deleting local artifact","time":"2025-01-08T08:18:39.912Z"}
{"level":"info","msg":"Successfully saved file: /tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:39.912Z"}
{"level":"info","msg":"Alloc=18772 TotalAlloc=23078 Sys=32597 NumGC=3 Goroutines=10","time":"2025-01-08T08:18:39.955Z"}
{"level":"info","msg":"No output parameters","time":"2025-01-08T08:18:19.239Z"}
{"level":"info","msg":"No output artifacts","time":"2025-01-08T08:18:19.239Z"}
{"level":"info","msg":"S3 Save path: /tmp/argo/outputs/logs/main.log, key: dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-2043699369/main.log","time":"2025-01-08T08:18:19.269Z"}
{"endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","level":"info","msg":"Creating minio client using static credentials","time":"2025-01-08T08:18:19.269Z"}
{"bucket":"flops-workflow","endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-2043699369/main.log","level":"info","msg":"Saving file to s3","path":"/tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:19.269Z"}
{"artifactName":"main-logs","duration":159149403,"error":null,"key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-2043699369/main.log","level":"info","msg":"Save artifact","time":"2025-01-08T08:18:19.428Z"}
{"level":"info","localArtPath":"/tmp/argo/outputs/logs/main.log","msg":"not deleting local artifact","time":"2025-01-08T08:18:19.428Z"}
{"level":"info","msg":"Successfully saved file: /tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:19.429Z"}
{"level":"info","msg":"Alloc=18785 TotalAlloc=23079 Sys=32853 NumGC=3 Goroutines=10","time":"2025-01-08T08:18:19.463Z"}
{"error":"context canceled","level":"info","msg":"stopping progress monitor (context done)","time":"2025-01-08T08:18:19.476Z"}
{"level":"info","msg":"No Script output reference in workflow. Capturing script output ignored","time":"2025-01-08T08:18:29.528Z"}
{"level":"info","msg":"No output parameters","time":"2025-01-08T08:18:29.528Z"}
{"level":"info","msg":"No output artifacts","time":"2025-01-08T08:18:29.528Z"}
{"level":"info","msg":"S3 Save path: /tmp/argo/outputs/logs/main.log, key: dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-3006597538/main.log","time":"2025-01-08T08:18:29.536Z"}
{"endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","level":"info","msg":"Creating minio client using static credentials","time":"2025-01-08T08:18:29.537Z"}
{"bucket":"flops-workflow","endpoint":"flops-minio-hl.flops-dev.svc.cluster.local:9000","key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-3006597538/main.log","level":"info","msg":"Saving file to s3","path":"/tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:29.537Z"}
{"artifactName":"main-logs","duration":107129688,"error":null,"key":"dag-workflow-retry-and-hook4lxph/dag-workflow-retry-and-hook4lxph-echo-3006597538/main.log","level":"info","msg":"Save artifact","time":"2025-01-08T08:18:29.644Z"}
{"level":"info","localArtPath":"/tmp/argo/outputs/logs/main.log","msg":"not deleting local artifact","time":"2025-01-08T08:18:29.644Z"}
{"level":"info","msg":"Successfully saved file: /tmp/argo/outputs/logs/main.log","time":"2025-01-08T08:18:29.644Z"}
{"level":"info","msg":"Alloc=18497 TotalAlloc=23068 Sys=32597 NumGC=3 Goroutines=10","time":"2025-01-08T08:18:29.679Z"}
wjdwogjs999 added a commit to devos-asgard/argo-workflows that referenced this issue Jan 8, 2025
@wjdwogjs999
fix(controller): Fixed so that Hook is not retried when Workflow leve…
72367c2 
…l Retry and Hook exist. Fixes argoproj#14058

Signed-off-by: jaeheon.jung <[email protected]>
@wjdwogjs999
Copy link
Author

latest version
image

3.6.2 version
image

wjdwogjs999 added a commit to devos-asgard/argo-workflows that referenced this issue Jan 8, 2025
@wjdwogjs999
fix(controller): Fixed so that Hook is not retried when Workflow leve…
fbcf140 
…l Retry and Hook exist. Fixes argoproj#14058

Signed-off-by: jaeheon.jung <[email protected]>
wjdwogjs999 added a commit to devos-asgard/argo-workflows that referenced this issue Jan 8, 2025
@wjdwogjs999
fix(controller): Fixed so that Hook is not retried when Workflow leve…
d94a805 
…l Retry and Hook exist. Fixes argoproj#14058

Signed-off-by: jaeheon.jung <[email protected]>
wjdwogjs999 added a commit to devos-asgard/argo-workflows that referenced this issue Jan 8, 2025
@wjdwogjs999
fix(controller): Fixed so that Hook is not retried when Workflow leve…
0984ed7 
…l Retry and Hook exist. Fixes argoproj#14058

Signed-off-by: jaeheon.jung <[email protected]>
@wjdwogjs999 wjdwogjs999 linked a pull request Jan 8, 2025 that will close this issue
Open
@jswxstw jswxstw added area/controller Controller issues, panics area/hooks area/retryStrategy Template-level retryStrategy labels Jan 8, 2025
@jswxstw
Copy link
Member

jswxstw commented Jan 8, 2025

This is also an issue caused by using the global retryStrategy, which has been mentioned by others before: #13239 (comment).

arthas-choi added a commit to devos-asgard/argo-workflows that referenced this issue Jan 14, 2025
@arthas-choi
Merge branch 'main' into argoproj#14058
bd20c01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/controller Controller issues, panics area/hooks area/retryStrategy Template-level retryStrategy type/bug
Projects
None yet
Milestone
No milestone
2 participants
@wjdwogjs999 @jswxstw