Add "connectors" to the collector visualization

[svrnm]

The image demonstrating the inner workings of the OpenTelemetry collector (https://opentelemetry.io/docs/collector/img/otel-collector.svg) is not showing connectors, which are now part of the collector. Having them added would be great. In general the image should remain as it is.

[Naggayi-Daphne-Pearl]

hello @svrnm,
Can i work on this issue please

[Beccamak]

@svrnm Hello, please can I work on this?

[svrnm]

@Naggayi-Daphne-Pearl since you raised your hand first, are you still open to contribute to this? If yes, please go ahead and provide a PR. If not, @Beccamak can give it a try.

[Beccamak]

Thank you @svrnm I'll wait for @Naggayi-Daphne-Pearl response. I'll gladly look for another issue to contribute to, if need be.

[svrnm]

@Beccamak since @Naggayi-Daphne-Pearl didn't pick this up, are you still interested?

[Beccamak]

@svrnm Yes please. Could you please guide me on how connectors are to come into the existing image?

[svrnm]

Could you please guide me on how connectors are to come into the existing image?

Please take a look into this image, which visualizes how connectors work:

https://opentelemetry.io/docs/collector/building/connector/otel-collector-after-connector.svg

This means that they are an exporter and receiver at the same time.

To emphasize this: visualizing this is really hard and I have no good idea how to do this. The image you see here is very iconic and used across many places, so part of this task is to come up with a great solution to integrate them, which keeps the image as plain and simple as it is today.

[Beccamak]

@svrnm Alright, thank you. I'll look into it and give you feedback.

[Beccamak]

@svrnm What do you think about this?

[svrnm]

adding the connectors to that longbar makes sense. I am not sure what that "HERE *" would mean/look like? What about a line that goes out from the connector on the exporter site into a "hole" and another "hole" that comes up at the connector at the receiver site and connects that arrow back?

cc @open-telemetry/collector-approvers please add your thoughts as well

[Beccamak]

Thank you so much, @svrnm. The "here" indicator showed where the connectors were added.
Following your further suggestions, I came up with the following:

1. Outer Connection
   

2. Inner connection
   

3. Modified inner connection. I noticed that the connection line overlapped with the lines connecting the receivers to the batch, so I developed another version to improve it.
   

Please let me know your thoughts.

[jpkrohling]

Here's a more verbose drawing: https://link.excalidraw.com/readonly/OGEGwmXiBarCIqe9e02o?darkMode=true

I think it's not easy to depict connectors clearly without having two pipelines.

[Beccamak]

@jpkrohling Thank you so much.
@svrnm Can I go ahead with the implementation now?

[svrnm]

Here's a more verbose drawing: link.excalidraw.com/readonly/OGEGwmXiBarCIqe9e02o?darkMode=true

I think it's not easy to depict connectors clearly without having two pipelines.

But having two pipelines would make the graphic much less appealing, especially as teaser for this page:

https://opentelemetry.io/docs/collector/

So, if there is no other way we all like I would prefer not having it. The "Modified inner connection." is a version I personally like. wdyt?

@svrnm Can I go ahead with the implementation now?

I would prefer to have a clear idea if(!) and how we implement it. Please give it some time

[Beccamak]

@svrnm I totally understand. I'll wait until it's clear.

[jpkrohling]

@djaglowski might be able to give some input as well, but I think it's fine leaving connectors out and explaining them later on. After all, a connector is a receiver and exporter at the same time, only at different pipelines (exporter in one, receiver in another).

[djaglowski]

Personally I think that viewing connectors as some form of loopback into a parallel pipeline makes the overall data flow more difficult to reason about. I would suggest a new diagram (in addition to one without connectors) which shows all data flow as left-to-right. Very roughly, it could look like this:

[svrnm]

We have a visualization for that already:

https://opentelemetry.io/docs/collector/building/connector/otel-collector-after-connector.svg

So, the question is:

• do we want to change the existing diagram?
• do we want to have a second diagram?

[jpkrohling]

It might be me, but when looking at that, my eyes go first to the connector in the metrics pipeline, and I have to follow it "back" to the exporter on the traces pipeline. I'm not sure what's the solution, but perhaps a set of arrows could help? Like, ">>>" instead of "---".

[Beccamak]

@svrnm Is there a resolution for this that we can implement right away?

[svrnm]

@svrnm Is there a resolution for this that we can implement right away?

I am worried that there is none, and that we keep the existing visualization.

The only thing I see is that based on @jpkrohling's comment we should update this visualization:

https://opentelemetry.io/docs/collector/building/connector/otel-collector-after-connector.svg

to have arrows for the "back" arrow