Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recording multiple values for the same attribute #13332

Closed
davisjose19 opened this issue Feb 17, 2025 · 6 comments · Fixed by #13337
Closed

Recording multiple values for the same attribute #13332

davisjose19 opened this issue Feb 17, 2025 · 6 comments · Fixed by #13337
Assignees
Labels
bug Something isn't working

Comments

@davisjose19
Copy link

davisjose19 commented Feb 17, 2025

Describe the bug

I was using the otel java agent version 2.12.0 - and since today morning my application started generated WARN logs every minute :

[otel.javaagent 2025-02-17 15:56:31:842 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
[otel.javaagent 2025-02-17 15:56:31:842 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
[otel.javaagent 2025-02-17 15:56:31:843 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
[otel.javaagent 2025-02-17 15:56:31:843 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
[otel.javaagent 2025-02-17 15:56:31:843 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Eden Space", jvm.memory.type="heap"}
[otel.javaagent 2025-02-17 15:56:31:843 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Survivor Space", jvm.memory.type="heap"}
[otel.javaagent 2025-02-17 15:56:31:843 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="waiting"}

I was using a gradle script to pull the latest version of the javaagent and run it - noticed that you guys released a new version - 2.13
So to test I ran it again with 2.12 version and I could not see these WARN messages.

Steps to reproduce

Run java agent locally with a spring boot application and every minute logs will be pushed saying metrics are being duplicated.

Expected behavior

There should not be WARN logs for the metrics being sent to otel

Actual behavior

I am seeing WARN logs -

[otel.javaagent 2025-02-17 15:59:31:843 +0000] [PeriodicMetricReader-1] WARN io.opentelemetry.sdk.metrics.internal.state.AsynchronousMetricStorage - Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="waiting"}

Javaagent or library instrumentation version

2.13

Environment

JDK:
OS:

[otel.javaagent 2025-02-17 15:54:47:545 +0000] [main] DEBUG io.opentelemetry.javaagent.tooling.VersionLogger - Running on Java 17.0.7. JVM Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 17.0.7+8-LTS-224

Additional context

No response

@davisjose19 davisjose19 added bug Something isn't working needs triage New issue that requires triage labels Feb 17, 2025
@stevesea
Copy link

stevesea commented Feb 17, 2025

seeing the similar behavior in our apps on upgrade from 2.12.0 to 2.13.0

Disabling runtime-telemetry via otel.instrumentation.runtime-telemetry.enabled=false removes the warnings

@trask
Copy link
Member

trask commented Feb 18, 2025

thanks for reporting this quickly!

if it turns out to be a regression, we'll release a 2.13.1 to patch it

#13173 seems the most likely culprit (but just purely a guess at this point)

cc @zeitlinger in case you have some time to look at it

@zeitlinger
Copy link
Member

I'll take a look.

@zeitlinger
Copy link
Member

It's a regression - this line was accidentally deleted: 9101f03#diff-fe9278430749297fddb0e617d628a596653d29bcb73458045036ecac145d6ed2L36

@zeitlinger zeitlinger self-assigned this Feb 18, 2025
@zeitlinger zeitlinger removed the needs triage New issue that requires triage label Feb 18, 2025
@trask
Copy link
Member

trask commented Feb 20, 2025

v2.13.1 patch is released, thanks again @davisjose19 and @stevesea for reporting it!

@davisjose19
Copy link
Author

Thank you for the quick fix @trask & @zeitlinger

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants