Enable using JMX Metrics instrumation as a library, not only through javaagent

[adamw]

Is your feature request related to a problem? Please describe.

I'd like to use the Otel <-> JMX integration as a library, without the need to install a Java Agent, as described currently on: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/README.md

I don't think the javaagent code is published, I've been only able to find the library on Maven Central:

<dependency>
    <groupId>io.opentelemetry.instrumentation</groupId>
    <artifactId>opentelemetry-jmx-metrics</artifactId>
    <version>2.8.0-alpha</version>
    <scope>runtime</scope>
</dependency>

Describe the solution you'd like

My goal is to include all observability-related setup as code, in the main method of the application. I've got OTEL working without problems using AutoConfiguredOpenTelemetrySdk.initialize().getOpenTelemetrySdk(), but to get the JMX integration I think I'll have to copy the initialisation code from JmxMetricInsightInstaller.java for now.

Ideally. I'd be able to do sth like JmxMetrics.initialise(openTelemetry), given an instance of OpenTelemetry.

Describe alternatives you've considered

No response

Additional context

No response

[trask]

hi @adamw!

<dependency>
    <groupId>io.opentelemetry.instrumentation</groupId>
    <artifactId>opentelemetry-jmx-metrics</artifactId>
    <version>2.8.0-alpha</version>
    <scope>runtime</scope>
</dependency>

I think this is what you want.

Ideally. I'd be able to do sth like JmxMetrics.initialise(openTelemetry), given an instance of OpenTelemetry.

check out

opentelemetry-java-instrumentation/instrumentation/jmx-metrics/library/src/main/java/io/opentelemetry/instrumentation/jmx/engine/JmxMetricInsight.java

Line 27 in faff2d9

public static JmxMetricInsight createService(OpenTelemetry ot, long discoveryDelay) {

[adamw]

@trask Thanks, I do have that, and I'm calling JmxMetricInsight.createService. However, I had to copy the code which calls .start on the created instance with the proper configuration. The JmxMetricInsightInstaller.java does exactly that: reads configuration and calls JmxMetricInsight.createService. I suppose the configuration parsing/reading class could also be part of a published jar, so that it would be usable in a no-agent setup?

[adamw]

In my case, the proper way to solve my problem (exposing runtime JVM metrics) was to use a dedicated library: opentelemetry-runtime-telemetry-java8, instead of using the JMX metrics (see #12310).

However, I still think that having a way of using the JMX metrics bridge without the java agent would be beneficial, so I'm keeping this open.

[trask]

@trask Thanks, I do have that, and I'm calling JmxMetricInsight.createService. However, I had to copy the code which calls .start on the created instance with the proper configuration. The JmxMetricInsightInstaller.java does exactly that: reads configuration and calls JmxMetricInsight.createService. I suppose the configuration parsing/reading class could also be part of a published jar, so that it would be usable in a no-agent setup?

cc @PeterF778

[PeterF778]

If I remember correctly, JMX Metric Insight uses two different calls for building the configuration (parser) and for starting the collection to make sure that no attempts to report metrics happen before the agent is fully initialized (metric exporter, etc.). I'll have another look to see if this can be made more user friendly.

[PeterF778]

I revisited the code. The opentelemetry-jmx-metrics.jar contains both the yaml parser and instrumentation. However, the default configuration files for supported platforms, as well as the controlling properties otel.jmx.target.system, otel.jmx.config (and others), are Java Agent specific. Hence, JmxMetricInsightInstaller uses two steps: it creates a MetricConfiguration and creates a service instance, passing the configuration as well.
It is easy to introduce a new entry createAndStartService(OpenTelemetry otel, MetricConfiguration conf), but I do not think it will make a big difference.

[breedx-splk]

@adamw Hey in case you haven't seen yet, there's been a new effort to consolidate the jmx stuff in contrib with what's in the agent, and the result is the newish jmx-scraper which is here: https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/jmx-scraper

Does that help with your specific use case? Thanks!

[SylvainJuge]

Also, feel free to provide feedback on how we could/should reuse the JMX metrics definitions once they are defined in a single location here: #13238

[github-actions]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed automatically if there is no response from the author within 7 additional days from this comment.

[SylvainJuge]

FYI the current approach we are taking is to add the yaml files with PRs similar to #13392 that I opened for JVM metrics.

Once this is merged and released we should be able to reuse those definitions as-is with jmx-scraper or in any tool that relies on the jmx-metrics library provided by instrumentation.

[adamw]

Sorry for being late into the discussion :) I'm currently using opentelemetry-runtime-telemetry-java8 and registering the observers as described in that readme. Would the jmx-scraper replace runtime-telemetry-java8?

[SylvainJuge]

No, the runtime-telemetry-java8 (and the -java17 one) would not be replaced by jmx-scraper , even if there is overlap between the captured metrics.

• the jvm.yaml will be included in instrumentation, but not used by the instrumentation agent
• the instrumentation agent continues to capture JVM metrics through instrumentation/runtime-telemetry as it currently does
• the jvm.yaml will be used by jmx-scraper to capture the JVM metrics.

Doing so allows to provide a single place to store all the yaml files and reuse them, the compromise is that it can be a bit confusing to see a yaml file that is not used in the agent but this is fine as long as there is only one for JVM metrics.

Also, another notable difference is that for some metrics we can't get all the recommended metric attributes through JMX, for example:jvm.thread.state is not available through JMX, it is currently only captured by iterating over all the threads which we can't do with yaml. Those differences are documented in #13392 .