Skip to content

Add tokio runtime metrics#1791

Open
mzacho wants to merge 3 commits intoistio:masterfrom
mzacho:tokio-metrics
Open

Add tokio runtime metrics#1791
mzacho wants to merge 3 commits intoistio:masterfrom
mzacho:tokio-metrics

Conversation

@mzacho
Copy link
Copy Markdown

@mzacho mzacho commented Mar 17, 2026

Self-explanatory.

Tested locally in kind, the new metrics as reported from localhost:15020/metrics:

# HELP tokio_num_workers the number of worker threads used by the runtime
# TYPE tokio_num_workers gauge
tokio_num_workers 2
# HELP tokio_global_queue_depth the number of tasks currently scheduled in the runtime's global queue
# TYPE tokio_global_queue_depth gauge
tokio_global_queue_depth 0
# HELP tokio_num_alive_tasks the number of alive tasks in the runtime
# TYPE tokio_num_alive_tasks gauge
tokio_num_alive_tasks 4
# HELP tokio_worker_total_busy_duration_seconds the amount of time worker threads have been busy
# TYPE tokio_worker_total_busy_duration_seconds counter
tokio_worker_total_busy_duration_seconds_total{worker="0"} 0.000133915
tokio_worker_total_busy_duration_seconds_total{worker="1"} 0.002096876
# HELP tokio_worker_park_count the total number of times the given worker thread has parked
# TYPE tokio_worker_park_count counter
tokio_worker_park_count{worker="0"} 9
tokio_worker_park_count{worker="1"} 11
# HELP tokio_worker_park_unpark_count the total number of times the given worker thread has parked and unparked
# TYPE tokio_worker_park_unpark_count counter
tokio_worker_park_unpark_count{worker="0"} 17
tokio_worker_park_unpark_count{worker="1"} 21

@mzacho mzacho requested review from a team as code owners March 17, 2026 23:57
@istio-testing istio-testing added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 17, 2026
@istio-testing
Copy link
Copy Markdown
Contributor

istio-testing commented Mar 17, 2026

Hi @mzacho. Thanks for your PR.

I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@keithmattix
Copy link
Copy Markdown
Contributor

keithmattix commented Mar 18, 2026

/ok-to-test

@istio-testing istio-testing added ok-to-test Set this label allow normal testing to take place for a PR not submitted by an Istio org member. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed needs-ok-to-test size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 18, 2026
@mzacho
Copy link
Copy Markdown
Author

mzacho commented Mar 18, 2026

make check-features failed when doing cd fuzz; RUSTFLAGS="--cfg fuzzing" cargo check since it's compiling ztunnel with tokio version 1.44.2 where the metrics worker_park_count, worker_total_busy_duration and worker_park_unpark_count weren't stabilized yet.

Ran cd fuzz && cargo update to update fuzz/Cargo.lock which fixes this.

@mzacho
Copy link
Copy Markdown
Author

mzacho commented Mar 18, 2026

/retest

@istio-testing istio-testing added the needs-rebase Indicates a PR needs to be rebased before being merged label Mar 28, 2026
@istio-testing istio-testing added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR needs to be rebased before being merged size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 8, 2026
Stevenjin8
Stevenjin8 reviewed Apr 8, 2026
View reviewed changes
src/metrics/tokio_runtime.rs
Comment on lines +46 to +53
macro_rules! encode_gauge {
($self:expr, $encoder:expr, $name:tt, $help:expr) => {{
let metric = ConstGauge::new($self.metrics.$name() as i64);
let metric_encoder =
$encoder.encode_descriptor(stringify!($name), $help, None, metric.metric_type())?;
metric.encode(metric_encoder)?;
}};
}
Copy link
Copy Markdown
Contributor

@Stevenjin8 Stevenjin8 Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance we could avoid these macros?

Copy link
Copy Markdown
Contributor

@Stevenjin8 Stevenjin8 Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh inline it would also not be the worst thing in the world.

Stevenjin8
Stevenjin8 requested changes Apr 8, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@Stevenjin8 Stevenjin8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the macros, lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Stevenjin8 Stevenjin8 Stevenjin8 requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

ok-to-test Set this label allow normal testing to take place for a PR not submitted by an Istio org member. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mzacho @istio-testing @keithmattix @Stevenjin8