protoc-gen-go: fully generate marshaling and unmarshaling code

[awalterschulze]

The Conversation started in #276

Is there a possibility of golang/protobuf starting to generate some marshaling and unmarshaling code?
I would recommend this over any use of unsafe, since then protocol buffers doesn't just play nice with appengine, but also gopherjs.
Also generated code is much faster, which is one of the reasons people flock to gogoprotobuf.
It would also be great if the generated code didn't depend on the proto package.
But this is just my personal wish, one less dependency for users of a package using protobufs.

• Walter Schulze

We're looking into ways to speed up marshal and unmarshal. Entirely generated code seems unlikely to be the right end point (too much code, not enough performance win), but we're evaluating multiple points in the solution space. Code not depending on the proto package at all is an interesting idea, although that implies even more generated code. I don't think that was on our radar.

• @rsc

[awalterschulze]

I really think the amount of speed gained is significant enough to warrant the extra code generation for marshal and unmarshal.

Here is an independant benchmark
https://github.com/alecthomas/go_serialization_benchmarks

BenchmarkGoprotobufMarshal-8                2000000        746 ns/op      312 B/op        4 allocs/op
BenchmarkGoprotobufUnmarshal-8              2000000        978 ns/op      432 B/op        9 allocs/op
BenchmarkGogoprotobufMarshal-8             10000000        211 ns/op       64 B/op        1 allocs/op
BenchmarkGogoprotobufUnmarshal-8            5000000        289 ns/op       96 B/op        3 allocs/op

Also once you have those methods, the extra generated code required to not use the proto package is very little.

[rsc]

fyi /cc @randall77 @cherrymui @dsnet

[awalterschulze]

This surely cannot be news?

But if it is, this is gogoprotobuf, it does code generation (and other gadgets) for protobuffers for go, since 2013.
https://github.com/gogo/protobuf

It has users, including Kubernetes
https://github.com/gogo/protobuf#users

[awalterschulze]

One of the newer things you can do with gogoprotobuf.
Using this command you can still import github.com/golang/protobuf/proto, but get the generated marshaler, size and unmarshaler. So only your code generation steps depends on gogoprotobuf.

go install github.com/gogo/protobuf/protoc-gen-gofast
protoc --gofast_out=. my.proto

Maybe this can give you an idea of what generating code in golang/protobuf would be like.

[johanbrandhorst]

Bump, this discussion should be revisited, as it still has no satisfying conclusion. Recent discussion implies this still hasn't been closed as a possibility. What are the current thoughts on this?