Skip to content

Experminental PR for adding metadata to protobufs#905

Open
jp-bennett wants to merge 2 commits into
masterfrom
metadata-experiment
Open

Experminental PR for adding metadata to protobufs#905
jp-bennett wants to merge 2 commits into
masterfrom
metadata-experiment

Conversation

@jp-bennett
Copy link
Copy Markdown
Contributor

Can we set devices to DIY vs fully-assembled, and then hide the diy-only options in the apps when connected to a pre-assembled board? Let's find out!

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 30, 2026

The latest Buf updates on your PR. Results from workflow pull-request / build (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedApr 30, 2026, 8:47 PM

@ianmcorvidae
Copy link
Copy Markdown
Contributor

Just noting here for good measure, the following patch is needed to make this work in python:

diff --git a/bin/regen-protobufs.sh b/bin/regen-protobufs.sh
index 23def7b..c48b856 100755
--- a/bin/regen-protobufs.sh
+++ b/bin/regen-protobufs.sh
@@ -45,6 +45,8 @@ $SEDCMD 's/^import "meshtastic\//import "meshtastic\/protobuf\//' "${INDIR}/"*.p
 
 $SEDCMD 's/^import "nanopb.proto"/import "meshtastic\/protobuf\/nanopb.proto"/' "${INDIR}/"*.proto
 
+$SEDCMD 's/\(meshtastic.config_field\)/(meshtastic.protobuf.config_field)/' "${INDIR}/"*.proto
+
 # Generate the python files
 ./nanopb-0.4.8/generator-bin/protoc -I=$TMPDIR/in --python_out "${OUTDIR}" "--mypy_out=${PYIDIR}" $INDIR/*.proto

once done, the following procedure gets access to the values:

from meshtastic.protobuf import config_pb2,field_metadata_pb2
field_descriptor = config_pb2.Config.PositionConfig.DESCRIPTOR.fields_by_name['rx_gpio']
options = field_descriptor.GetOptions()
options.Extensions[field_metadata_pb2.config_field].diy_only # returns True

jamesarich added a commit to meshtastic/Meshtastic-Android that referenced this pull request Apr 30, 2026
Adds build-time codegen that auto-generates a ConfigFieldMetadataRegistry
from (meshtastic.config_field) annotations in .proto files.
- New FieldMetadataSchemaHandler in build-logic walks all message fields
- Emits a static Kotlin registry mapping (messageType, fieldTag) → metadata
- Zero runtime cost, no wire-schema dep in app binary
- Adding annotations to new fields requires no code changes
Companion to: meshtastic/protobufs#905
jamesarich added a commit to meshtastic/Meshtastic-Android that referenced this pull request Apr 30, 2026
Adds build-time codegen that auto-generates a ConfigFieldMetadataRegistry
from (meshtastic.config_field) annotations in .proto files.
- New FieldMetadataSchemaHandler in build-logic walks all message fields
- Emits a static Kotlin registry mapping (messageType, fieldTag) → metadata
- Zero runtime cost, no wire-schema dep in app binary
- Adding annotations to new fields requires no code changes
Companion to: meshtastic/protobufs#905
jamesarich added a commit to meshtastic/Meshtastic-Android that referenced this pull request Apr 30, 2026
Adds build-time codegen that auto-generates a ConfigFieldMetadataRegistry
from (meshtastic.config_field) annotations in .proto files.
- New FieldMetadataSchemaHandler in build-logic walks all message fields
- Emits a static Kotlin registry mapping (messageType, fieldTag) → metadata
- Zero runtime cost, no wire-schema dep in app binary
- Adding annotations to new fields requires no code changes
Companion to: meshtastic/protobufs#905
Replace the ConfigFieldMetadata message wrapper with a direct
scalar bool extension on FieldOptions. This allows Wire 6's native
emitAppliedOptions to generate @DiyOnlyOption annotations without
a custom SchemaHandler, and simplifies access for all downstream
clients (Swift, Python, Go).
Usage: [(meshtastic.diy_only) = true] instead of
       [(meshtastic.config_field).diy_only = true]
@jamesarich jamesarich force-pushed the metadata-experiment branch from 143d8c0 to fc5a1a2 Compare April 30, 2026 20:47
@jamesarich
Copy link
Copy Markdown
Contributor

jamesarich commented Apr 30, 2026

Updated to use scalars:

Proto Simplification: field_metadata.proto → scalar extensions

TL;DR: Flattening (meshtastic.config_field).diy_only to a simple (meshtastic.diy_only) bool extension eliminates ~200 lines of custom Wire build code on Android and simplifies access for every downstream client.

What changes in the proto

- message ConfigFieldMetadata {
-   required bool diy_only = 1 [default = false];
- }
- extend google.protobuf.FieldOptions {
-   optional ConfigFieldMetadata config_field = 51001;
- }
+ extend google.protobuf.FieldOptions {
+   optional bool diy_only = 51001 [default = false];
+ }

Usage: [(meshtastic.config_field).diy_only = true][(meshtastic.diy_only) = true]

Impact per client

Client Tool Impact
Android Wire 6.2 ✅ Wire natively emits @DiyOnlyOption(true) on generated fields
Apple protoc-gen-swift ✅ generates var diyOnly: Bool extension on FieldOptions directly
Python protoc Extensions[diy_only] returns bool directly instead of unwrapping a message
Firmware nanopb ✅ No impact — nanopb ignores custom options

Extensibility

Future metadata (e.g. admin_only, min_value) → just add more scalar extensions in the 51000–51999 range. No shared wrapper message to evolve.

jamesarich added a commit to meshtastic/Meshtastic-Android that referenced this pull request Apr 30, 2026
…hemaHandler

The upstream proto (meshtastic/protobufs#905) has been simplified to use
a scalar bool extension instead of a message wrapper:
  [(meshtastic.diy_only) = true]
Wire 6's emitAppliedOptions now natively generates @DiyOnlyOption(true)
annotations on fields, eliminating the need for our custom
FieldMetadataSchemaHandler.
Changes:
- Delete FieldMetadataSchemaHandler.kt (~200 LOC)
- Remove wire-schema dependency from build-logic
- Remove custom {} handler block from core:proto build
- Add emitAppliedOptions = true to Wire kotlin {} config
- Update FieldMetadataDemo to demonstrate the native annotation
- Bump protobufs submodule to metadata-experiment branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants