[SL-2831] Switch target info logs to debug #1490
Labels
High priority
Created by Linear-GitHub Sync
Medium
Created by Linear-GitHub Sync
Metricflow
Created by Linear-GitHub Sync
starter task
Created by Linear-GitHub Sync
Milestone
We have a lot of
logger.info()
calls inside of MetricFlow that flood our datadog logs. There are a couple of places where these are essentially printing object processing info, but are doing so inside of a method that's being invoked within a loop or recursive process. This means complex semantic manifests or queries against large numbers of metrics could end up spitting out thousands of these log lines, which are mainly useful for tracking progress during development.This task is to find and update a small number of high volume logs of this type. You can find them by doing datadog searches for high volume logs (simply scrolling through a reasonable subset of the last day's worth of MT logs for MFS should give you some ideas), or just debugging some query and being like "whoa why are there 9 pages of this same message here?!". You can also find them by looking carefully at MetricFlow's codebase and realizing that they're invoked inside of some nested loop someplace. Things to look out for are nondescript info messages that are called inside of deep for loops in the DataflowPlanBuilder, or that are called in any
visit_*
method in a visitor pattern object.Examples:
More broadly, if we get datadog tracing set up we can probably switch all timing logs in MetricFlow to debug, since the function span views should cover most of what we need there.
From SyncLinear.com | SL-2831
The text was updated successfully, but these errors were encountered: