argo-otlp-exporter

A small daemon that watches Argo Workflow custom resources via the Kubernetes API and emits an OTLP trace per completed workflow. Each workflow becomes a span tree (root = workflow, children = each step's pod), shipped to an OpenTelemetry collector (e.g. Grafana Alloy / Tempo). Lets you view per-workflow timing as a waterfall in Tempo without modifying any WorkflowTemplate.

Written in Go — small static binary, stdlib net/http for both the Kubernetes API watch and the OTLP HTTP/JSON POST. No protobuf, no gRPC, no client-go dependency tree.

Why

Argo Workflows' built-in metrics tell you aggregate controller perf (queue depth, operation duration). The per-workflow step timing already lives in Workflow.status.nodes[].startedAt/finishedAt — the Argo UI renders it as a graph per-workflow. There's no built-in way to ship that data to a trace store, which is the natural shape for "how long did each step take across this run".

This exporter fills that gap: it turns Argo's existing node-timing data into OTLP spans without instrumenting individual templates.

How it works

1. Watch Workflow CRs in a namespace via the Kubernetes Watch API (/apis/argoproj.io/v1alpha1/namespaces/<ns>/workflows?watch=true) using the pod's serviceaccount token + CA.
2. On a MODIFIED event where status.phase ∈ {Succeeded, Failed, Error}, parse status.nodes and build a span tree.
3. POST the OTLP/JSON payload to $OTLP_HTTP_ENDPOINT (default http://alloy.alloy.svc.cluster.local:4318/v1/traces).

Span tree shape:

workflow:<name>                       <- root span (workflow lifetime)
├── <template-name>                   <- one span per Pod node in .status.nodes
├── <template-name>
└── …

DAG / Steps parent relationships use Argo's boundaryID so the tree mirrors the workflow's own structure.

Build

CGO_ENABLED=0 go build -ldflags="-s -w" -trimpath -o argo-otlp-exporter ./
./argo-otlp-exporter

Configuration

Env vars (all optional, sensible defaults):

Var	Default	Meaning
WATCH_NAMESPACE	workflows-dev	Argo workflows namespace to watch
OTLP_HTTP_ENDPOINT	http://alloy.alloy.svc.cluster.local:4318/v1/traces	OTLP collector endpoint
SERVICE_NAME	argo-workflow	service.name resource attribute
TOKEN_PATH	/var/run/secrets/kubernetes.io/serviceaccount/token	Bearer-token file
CA_PATH	/var/run/secrets/kubernetes.io/serviceaccount/ca.crt	API server CA
KUBE_API	https://kubernetes.default.svc	API server base URL

Deployment

See deploy/manifest.yaml for a minimal Kubernetes Deployment + RBAC.

License

MIT (see LICENSE).