Skip to content

Simplify approach for creating dag run and task spans#62554

Open
dstandish wants to merge 38 commits intoapache:mainfrom
astronomer:minimize-otel-code
Open

Simplify approach for creating dag run and task spans#62554
dstandish wants to merge 38 commits intoapache:mainfrom
astronomer:minimize-otel-code

Conversation

@dstandish
Copy link
Contributor

@dstandish dstandish commented Feb 27, 2026

Replace otel spans tracking implementation with something much simpler

  • rip out existing spans
  • add back the dag run span
  • add back a task span (this is handled in the runner)
    • the detailed task spans will be done in a followup
  • rip out Trace, _TraceMeta, DebugTrace "classes" --> later
  • remove the "Tracer" protocol and EmptyTrace --> later
    • we don't need to keep them for backcompat because they were never really used~
    • EmptyTrace was returned when otel not enabled. but it should be fine to lean on the normal behavior of otel to create non-recording spans when otel not enabled, and the updates to OtelTrace should accomplish this
      deprecate the OtelTrace class
    • update its methods so that it does as little as possible but doesn't break dags for any unfortunate souls that decided to use it
  • re-implement the configuration bit which looks at our deprecated otel config vars and the env vars supported by otel

@boring-cyborg boring-cyborg bot added area:DAG-processing area:Executors-core LocalExecutor & SequentialExecutor area:Scheduler including HA (high availability) scheduler area:task-sdk area:Triggerer labels Feb 27, 2026
@xBis7 xBis7 mentioned this pull request Feb 27, 2026
2 tasks
@dstandish dstandish force-pushed the minimize-otel-code branch 3 times, most recently from 74b3942 to 236e5ee Compare March 3, 2026 16:40
@dstandish dstandish changed the title WIP - Minimize otel code Simplify approach for creating dag run and task spans Mar 3, 2026
@dstandish dstandish force-pushed the minimize-otel-code branch 2 times, most recently from a1cb165 to a85b5eb Compare March 5, 2026 03:29
Copy link
Contributor

@xBis7 xBis7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dstandish Great job! I left some comments but generally it looks good and I think it can be converted to an actual PR.

Additionally, I tested it manually and I ran a dag with long timeouts. I didn't see any issues.

@dstandish dstandish force-pushed the minimize-otel-code branch from d138440 to a5edfd2 Compare March 6, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:DAG-processing area:Executors-core LocalExecutor & SequentialExecutor area:Scheduler including HA (high availability) scheduler area:task-sdk area:Triggerer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants