How to propagate Context to reactive/coroutine MongoDB commands?

[glebignatieff]

Hi everyone!

I am quite novice in OpenTelemetry, previously worked with the Python packages and now I am trying to configure the same setup, but in Kotlin/JVM. I am using Ktor with MongoDB Kotlin Driver in my project. In order to instrument both of them I use ktor-2.0 and mongo-3.1 (library) respectively.

I have noticed that with such a combination I get different trace IDs:

Nov 14, 2023 4:18:18 PM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'insert test.test' : b220bee54252917b92ee9460c7c0e4b2 f0a5f66867c20fa6 CLIENT [tracer: io.opentelemetry.mongo-3.1:1.31.0-alpha] AttributesMap{data={db.name=test, db.operation=insert, db.mongodb.collection=test, db.statement={"insert": "test", "ordered": "?", "txnNumber": "?", "$db": "?", "lsid": {"id": "?"}, "documents": [{"_id": "?", "test": "?"}]}, db.system=mongodb, net.peer.port=27017, db.connection_string=mongodb://localhost:27017, net.peer.name=localhost}, capacity=128, totalAddedValues=8}
Nov 14, 2023 4:18:18 PM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET /test-tracing' : b6525bc98a4c08fe3aa7204f0b2aadcc 7351637f60027a3b SERVER [tracer: io.opentelemetry.ktor-2.0:1.31.0-alpha] AttributesMap{data={http.method=GET, http.scheme=http, net.protocol.version=1.1, net.protocol.name=http, http.target=/test-tracing, net.host.port=8080, net.sock.peer.addr=127.0.0.1, http.response_content_length=12, http.route=/test-tracing, user_agent.original=curl/8.1.2, net.host.name=localhost, http.status_code=200}, capacity=128, totalAddedValues=12}

In comparison with the sync MongoDB driver the IDs are the same:

Nov 14, 2023 4:16:41 PM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'insert test.test' : 03cdd199593d9c50ade25ec50ec2d838 92c982cad923a38d CLIENT [tracer: io.opentelemetry.mongo-3.1:1.31.0-alpha] AttributesMap{data={db.connection_string=mongodb://localhost:27017, net.peer.name=localhost, db.system=mongodb, net.peer.port=27017, db.statement={"insert": "test", "ordered": "?", "txnNumber": "?", "$db": "?", "lsid": {"id": "?"}, "documents": [{"_id": "?", "test": "?"}]}, db.operation=insert, db.mongodb.collection=test, db.name=test}, capacity=128, totalAddedValues=8}
Nov 14, 2023 4:16:41 PM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET /test-tracing' : 03cdd199593d9c50ade25ec50ec2d838 214878a5ca39d793 SERVER [tracer: io.opentelemetry.ktor-2.0:1.31.0-alpha] AttributesMap{data={http.status_code=200, user_agent.original=curl/8.1.2, net.host.name=localhost, http.response_content_length=12, http.route=/test-tracing, net.host.port=8080, http.target=/test-tracing, net.sock.peer.addr=127.0.0.1, net.protocol.name=http, http.scheme=http, net.protocol.version=1.1, http.method=GET}, capacity=128, totalAddedValues=12}

While debugging I've noticed that Context in TracingCommandListener is empty, therefore a new one is created, which leads to the listed results.

I've tried to wrap the call with asContextElement, but no luck. Does anyone has an idea how can I propagate Context in this case?

Here's an MVP to reproduce the problem: opentelemetry-test.zip (MongoDB on the localhost is required).

[laurit]

You might have better luck with java agent. As you noticed the context is lost before TracingCommandListener to make it work we'd need to figure out how to get the context there. I think the context is lost in https://github.com/mongodb/mongo-java-driver/blob/02a61642394ca707beb30947101afb5b8c6d36b7/driver-core/src/main/com/mongodb/internal/connection/DefaultConnectionPool.java#L1286 in java agent we can wrap the task object and use that to preserve the context but for library instrumentation our options are much more limited. One option to make it work would be to rewrite the instrumentation in a different way. Instead of using TracingCommandListener one could attempt to wrap the mongo reactive client and create spans based on the methods called there.

[glebignatieff]

I see, thanks for the response! I guess I'll try my luck with the agent as rewriting the instrumentation isn't currently an option :)