AgentTestingExporterAccess.getExportedMetrics() doesn't get any metrics

[chlos]

Describe the bug

I'm trying to test the traces/metrics my Java app emits with the help of io.opentelemetry.javaagent.testing.
It allowed me to check spans, but it didn't work with the metrics.

AgentTestingExporterAccess.getExportedMetrics() seems to always return an empty list.

Steps to reproduce

import io.opentelemetry.javaagent.testing.common.AgentTestingExporterAccess;

@BeforeEach
void setUp() {
    AgentTestingExporterAccess.reset();
}


@Test
void otelTest() {
// tracing
GlobalOpenTelemetry.getTracer("test").spanBuilder("test").startSpan().end();
List<SpanData> spans = AgentTestingExporterAccess.getExportedSpans();
assertEquals(1, spans.size(), "a span expected");

// metrics
GlobalOpenTelemetry.getMeter("test").upDownCounterBuilder("test").build().add(1);
List<MetricData> metricsTmp = AgentTestingExporterAccess.getExportedMetrics();
assertEquals(1, metricsTmp.size(), "a counter expected");
}

Expected behavior

Test passes:

• the tracing assertion passes
• the metrics assertion passes

Actual behavior

Test fails:

• the tracing assertion passes
• the metrics assertion fails

org.opentest4j.AssertionFailedError: a counter expected ==> expected: <1> but was: <0>

Javaagent or library instrumentation version

2.13.1

Environment

JDK:
OS:

Additional context

build.gradle:

dependencies: {
    // ...
    implementation platform("io.opentelemetry:opentelemetry-bom:1.21.0")
    implementation("io.opentelemetry:opentelemetry-api")

    testImplementation("io.opentelemetry:opentelemetry-sdk")
    testImplementation("io.opentelemetry:opentelemetry-sdk-metrics")
    testImplementation("io.opentelemetry:opentelemetry-sdk-testing")
    testImplementation("io.opentelemetry.javaagent:opentelemetry-testing-common:2.13.1-alpha")
    testImplementation("io.opentelemetry.javaagent:opentelemetry-agent-for-testing:2.13.1-alpha")
}

configurations {
    otelAgent
}

dependencies {
    otelAgent "io.opentelemetry.javaagent:opentelemetry-javaagent:2.13.1"
}

test {
    useJUnitPlatform()
    jvmArgs "-javaagent:${configurations.otelAgent.singleFile}"
}

[laurit]

Could you provide a minimal sample application that reproduces the issue so it would be easier for us to debug this.

[chlos]

Well the test I posted is a sample application. Basically it's an extended version of AgentForTestingTest.exportAndRetrieve() -- I just added a part with testing AgentTestingExporterAccess.getExportedMetrics()

[chlos]

Hopefully this PR is helpful for reproducing and debugging this issue: #13414

[chlos]

And here is the example (full log) of the test failure:

2025-02-27T18:51:41.7139820Z AgentForTestingTest > exportAndRetrieveMetrics() STANDARD_ERROR
2025-02-27T18:51:41.7141273Z     [otel.javaagent 2025-02-27 18:51:41:311 +0000] [Test worker] DEBUG io.opentelemetry.sdk.metrics.export.PeriodicMetricReader - No metric data to export - skipping export.
2025-02-27T18:51:41.7143521Z     [otel.javaagent 2025-02-27 18:51:41:379 +0000] [Test worker] DEBUG io.opentelemetry.sdk.metrics.export.PeriodicMetricReader - No metric data to export - skipping export.
2025-02-27T18:51:41.7144506Z 
2025-02-27T18:51:41.7144900Z AgentForTestingTest > exportAndRetrieveMetrics() FAILED
2025-02-27T18:51:41.7145661Z     org.opentest4j.AssertionFailedError: expected: <1> but was: <0>
2025-02-27T18:51:41.7146897Z         at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
2025-02-27T18:51:41.7148837Z         at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
2025-02-27T18:51:41.7150044Z         at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
2025-02-27T18:51:41.7150990Z         at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
2025-02-27T18:51:41.7151949Z         at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
2025-02-27T18:51:41.7153252Z         at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
2025-02-27T18:51:41.7154691Z         at io.opentelemetry.javaagent.testing.AgentForTestingTest.exportAndRetrieveMetrics(AgentForTestingTest.java:44)

[chlos]

As far as I can see, the agent configures (during AgentTestingExporterAccess.reset();) a normal in-memory tracer, but in case of a meter it falls back to the no-op implementation:

[chlos]

Hey @laurit, just noticed your workaround PR: #13423 I didn't really understood it -- does it fix only certain versions of the test agent?

In my code I was trying to use 2.13.1-alpha/2.13.1, how can I fix it in my case?

[chlos]

Thanks for fixing it in #13423 @laurit !
Could you merge the #13414 then?

[chlos]

Just in case let's cover the logging part with a test: #13459