Skip to content

Commit 974505f

Browse files
author
Liudmila Molkova
committed
more cleanups and lint
1 parent 1618512 commit 974505f

File tree

2 files changed

+73
-19
lines changed

2 files changed

+73
-19
lines changed

CHANGELOG.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,9 @@ release.
3535

3636
### OTEPs
3737

38-
- [OTEP-4333](https://github.com/open-telemetry/oteps/blob/main/text/4333-component.md)
38+
- [OTEP-4333](https://github.com/open-telemetry/opentelemetry-specification/pull/4333)
3939
Recording exceptions on logs.
4040

41-
4241
## v1.40.0 (2024-12-12)
4342

4443
### Context

oteps/4333-recording-exceptions-on-logs.md

Lines changed: 72 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,24 @@
11
# Recording exceptions and errors on logs
22

3+
<!-- toc -->
4+
5+
- [Motivation](#motivation)
6+
- [Guidance](#guidance)
7+
* [Details](#details)
8+
- [API changes](#api-changes)
9+
- [Examples](#examples)
10+
* [Logging exception from client library in a user application](#logging-exception-from-client-library-in-a-user-application)
11+
* [Logging error inside the library (native instrumentation)](#logging-error-inside-the-library-native-instrumentation)
12+
* [Logging errors in messaging processor](#logging-errors-in-messaging-processor)
13+
+ [Native instrumentation](#native-instrumentation)
14+
+ [Instrumentation library](#instrumentation-library)
15+
- [Trade-offs and mitigations](#trade-offs-and-mitigations)
16+
- [Prior art and alternatives](#prior-art-and-alternatives)
17+
- [Open questions](#open-questions)
18+
- [Future possibilities](#future-possibilities)
19+
20+
<!-- tocstop -->
21+
322
This OTEP provides guidance on how to record exceptions using OpenTelemetry logs focusing on minimizing duplication and providing context to reduce the noise.
423

524
## Motivation
@@ -32,12 +51,12 @@ starting point, but they are encouraged to adjust it to their needs.
3251
This guidance boils down to the following:
3352

3453
Instrumentations should record exception information (along with other context) on the log record and
35-
use appropriate severity - only unhandled exceptions should be recorded as `Error` or higher. They
54+
use appropriate severity - only unhandled exceptions should be recorded as `Error` or higher. Instrumentations
3655
should strive to report each exception once.
3756

38-
Instrumentation should provide the whole exception instance to the OTel (instead of individual attributes)
39-
and the OTel SDK should, based on user configuration, decide which information to record. As a default,
40-
this OTEP proposes to record exception stack traces on log with `Error` or higher severity.
57+
Instrumentations should provide the whole exception instance to the OTel (instead of individual attributes)
58+
and the OTel SDK should, based on configuration, decide which information to record. As a default,
59+
this OTEP proposes to record exception stack traces on logs with `Error` or higher severity.
4160

4261
### Details
4362

@@ -47,7 +66,16 @@ this OTEP proposes to record exception stack traces on log with `Error` or highe
4766
2. Instrumentations for incoming requests, message processing, background job execution, or others that wrap user code and usually create local root spans, should record logs
4867
for unhandled exceptions with `Error` severity.
4968

50-
Some runtimes and frameworks provide global exception handler that can be used to record exception logs. Priority should be given to the instrumentation point where the operation context is available.
69+
> [!NOTE]
70+
>
71+
> Top-level instrumentations is the only place non-native instrumentations should
72+
> record exceptions.
73+
74+
Some runtimes provide global exception handler that can be used to log exceptions.
75+
Priority should be given to the instrumentation point where the operation context is available.
76+
Language SIGs are encouraged to give runtime-specific guidance. For example, here's
77+
[.NET guidance](https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/trace/reporting-exceptions#unhandled-exception)
78+
for recording exceptions on traces.
5179

5280
3. Native instrumentations should record log describing an error and the context it happened in
5381
when this error is detected (or where the most context is available).
@@ -72,16 +100,12 @@ this OTEP proposes to record exception stack traces on log with `Error` or highe
72100
See [logback exception config](https://logback.qos.ch/manual/layouts.html#ex) for an example of configuration that
73101
records stack trace conditionally.
74102

75-
> [!NOTE]
76-
>
77-
> Based on this guidance non-native instrumentations should record exceptions in top-level instrumentations only (#2 in [Details](#details))
103+
## API changes
78104

79-
> [!Important]
105+
> [!IMPORTANT]
80106
>
81107
> OTel should provide API like `setException` when creating log record that will record only necessary information depending
82-
> on the configuration and log severity. See [API changes](#api-changes) for the details.
83-
84-
## API changes
108+
> on the configuration and log severity.
85109
86110
It should not be an instrumentation library concern to decide whether exception stack trace should be recorded or not.
87111
Library may write logs providing exception instance through a log bridge and not be aware of this guidance.
@@ -96,9 +120,9 @@ OTel SDK should implement such methods and set exception attributes based on con
96120
and in the optimal way. This would also provide a more scalable way to evolve this guidance
97121
and extend configuration options if necessary.
98122

99-
### Examples
123+
## Examples
100124

101-
#### Catching exception from client library in a user application
125+
### Logging exception from client library in a user application
102126

103127
```java
104128
StorageClient client = createClient(endpoint, credential);
@@ -133,7 +157,7 @@ try {
133157
}
134158
```
135159

136-
#### Recording error inside the library (native instrumentation)
160+
### Logging error inside the library (native instrumentation)
137161

138162
It's a common practice to record exceptions using logging libraries. Client libraries that are natively instrumented with OpenTelemetry should
139163
leverage OTel Events/Logs API for their exception logging purposes.
@@ -171,7 +195,7 @@ public class StorageClient {
171195
Network level errors are part of normal life, we should consider using low severity for them
172196

173197
```java
174-
public class Connection {
198+
public class NetworkClient {
175199

176200
private final Logger logger;
177201
...
@@ -193,7 +217,9 @@ public class Connection {
193217
}
194218
```
195219

196-
#### Messaging processor instrumentation
220+
### Logging errors in messaging processor
221+
222+
#### Native instrumentation
197223

198224
In this example, application code provides the callback to the messaging processor to
199225
execute for each message.
@@ -228,6 +254,35 @@ try {
228254
}
229255
```
230256

257+
If native instrumentation supports tracing, it should capture the error in the scope of the processing
258+
span.
259+
260+
#### Instrumentation library
261+
262+
In this example we leverage Spring Kafka `RecordInterceptor` extensibility point that allows to
263+
listen to exceptions that remained unhandled.
264+
265+
```java
266+
import org.springframework.kafka.listener.RecordInterceptor;
267+
final class InstrumentedRecordInterceptor<K, V> implements RecordInterceptor<K, V> {
268+
...
269+
270+
@Override
271+
public void failure(ConsumerRecord<K, V> record, Exception exception, Consumer<K, V> consumer) {
272+
// we should capture the error in the scope of the processing span (or pass its context explicitly).
273+
logger.logRecordBuilder()
274+
.setSeverity(Severity.ERROR)
275+
.addAttribute("messaging.message.id", record.getId())
276+
...
277+
.addException(ex)
278+
.emit();
279+
// ..
280+
}
281+
}
282+
```
283+
284+
See [corresponding Java (tracing) instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/spring/spring-kafka-2.7/library/src/main/java/io/opentelemetry/instrumentation/spring/kafka/v2_7/InstrumentedRecordInterceptor.java) for the details.
285+
231286
## Trade-offs and mitigations
232287

233288
1. Breaking change for any component following existing [exception guidance](https://github.com/open-telemetry/opentelemetry-specification/blob/a265ae0628177be25dc477ea8fe200f1c825b871/specification/trace/exceptions.md) which recommends recording exceptions as span events in every instrumentation that detects them.

0 commit comments

Comments
 (0)