Add JSON and Protobuf native schema support#279
Merged
blankensteiner merged 1 commit intoFeb 11, 2026
Conversation
Implement ISchema<T> for JSON and Protobuf message types, closing the gap with Java and Go Pulsar clients. - JsonSchema<T>: uses System.Text.Json for serialization with optional JsonSerializerOptions and schema definition string - ProtobufSchema<T>: uses Google.Protobuf (already a dependency) with native protobuf descriptor for schema data - Add factory methods Schema.Json<T>() and Schema.Protobuf<T>() - Add inner exception constructor to SchemaSerializationException - Add System.Text.Json dependency for netstandard2.0/2.1 targets - Include 21 unit tests covering round-trip, edge cases, and error handling Fix apache#254
Contributor
|
Thanks for the PR :-) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #254
Description
Implements
ISchema<T>for JSON and Protobuf message types, bringing DotPulsar closer to feature parity with the Java and Go Pulsar clients.New schemas:
JsonSchema<T>: Serializes/deserializes usingSystem.Text.Json. Supports optionalJsonSerializerOptionsfor customization and an optional JSON schema definition string for broker-side schema compatibility checking. UsesSchemaType.Json.ProtobufSchema<T>: Constrained toGoogle.Protobuf.IMessage<T>. Uses native protobuf serialization withSchemaType.ProtobufNative(type 20). Extracts the file descriptor from the message type as schema data. No new dependencies —Google.Protobufis already a project dependency.Other changes:
Schema.Json<T>()andSchema.Protobuf<T>()factory methods added toSchema.cs, following the existing pattern forAvroISpecificRecord<T>()andAvroGenericRecord<T>().SchemaSerializationExceptionto properly wrap serialization errors (matchingSchemaExceptionwhich already has this overload).System.Text.Jsonpackage reference fornetstandard2.0andnetstandard2.1targets only (already part of the BCL fornet8.0+).Design decisions worth discussing:
ProtobufNative(type 20) vsProtobuf(type 3): We choseProtobufNativewhich uses the native protobuf file descriptor as schema data. Type 3 uses Avro-encoded schema representation which would require an Avro dependency. Happy to adjust if maintainers prefer type 3.BooleanSchemawork). An overload accepting a schema definition string is available for broker-side schema evolution checking.IMessage<T>disambiguation:Google.Protobuf.IMessage<T>is fully qualified in the constraint to avoid ambiguity withDotPulsar.Abstractions.IMessage<T>.Regression
No. This is a new feature adding two new schema types. No existing functionality is modified beyond adding an inner exception constructor to
SchemaSerializationException.Testing
JsonSchema<T>covering: construction, schema info, encode, decode, round-trip, customJsonSerializerOptions(camelCase), schema definition passthrough, and empty bytes error handling.ProtobufSchema<T>covering: construction, schema info (type, name, data), encode, decode, round-trip withStringValueandInt32Value, empty message, and empty bytes.ZstdCompressionTestsfailure unrelated to this change).netstandard2.0,netstandard2.1,net8.0,net9.0,net10.0).