Best way to clone a protobuf object

[cghawthorne]

protobuf.js version: 6.8.6

I have several places in my code where I'd like to clone a protobuf object and make changes to the clone that don't affect the original. I've tried several methods but haven't been able to find one that works and is efficient.

Here are some examples:

test.proto

package test;
syntax = "proto3";

message Inner {
  string id = 1;
}

message Test {
  string id = 1;
  repeated Inner inner = 2;
}

$ pbjs -t static-module -w commonjs -o compiled.js test.proto

test.js:

var clone = require('clone');
var proto = require('./compiled');

var a = proto.test.Test.create();

// Clone method goes here.

b.inner.push(proto.test.Inner.create());
if (a.inner.length != 0) {
  throw new Error();
}

First, I tried using the clone package:

var b = clone(a);

This causes some kind of problem with the prototype of the object.

test.js:11
b.inner.push(proto.test.Inner.create());
        ^

TypeError: Cannot add property 0, object is not extensible
    at Array.push (<anonymous>)
    at Object.<anonymous> (/usr/local/google/home/fjord/m/protoissue/test.js:11:9)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Function.Module.runMain (module.js:605:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:575:3

Then, I tried just passing the first object to create:

var b = proto.test.Test.create(a);

This doesn't produce a deep clone:

test.js:13
  throw new Error();
  ^

Then, I tired to/from Object:

var b = proto.test.Test.fromObject(proto.test.Test.toObject(a));

This works, but toObject only supports an actual Test proto object, and we want our API to support users supplying any object that matches ITest.

This is the approach we ended up using:

var b = proto.test.Test.decode(proto.test.Test.encode(a).finish());

This produces an actual deep clone (guaranteed because object references clearly aren't preserved across serialization) and it works with either Test or ITest. However, this seems really heavyweight for just cloning an object.

Do you have suggestions for the best way to do this? It would be great if protobuf.js had a clone() method that was the officially supported way of cloning an object.

Thanks!

[cghawthorne]

FYI @adarob

[dcodeIO]

var b = proto.test.Test.fromObject(a);

Can you explain a bit why this one isn't working? Asking because it should. If necessary, maybe clone a the usual object way before passing it to fromObject.

[cghawthorne]

Ah, sorry, I forgot to include that exact example. fromObject seems to not be a full clone:

var b = proto.test.Test.fromObject(a);

test.js:14
  throw new Error();

I tried cloneing before calling fromObject, and that has the same prototype problem:

var b = proto.test.Test.fromObject(clone(a));

b.inner.push(proto.test.Inner.create());
        ^

TypeError: Cannot add property 0, object is not extensible

[dcodeIO]

Seems the clone implementation does some black magic trying to preserve the prototype or something. A simple deep clone to an object should work, I believe.

[cghawthorne]

Do you have a code snippet of what that would look like?

[dcodeIO]

Maybe something along the lines of:

function clone(obj) {
  if (typeof obj !== "object" || !obj) return obj;
  var cpy;
  if (Array.isArray(obj) || ArrayBuffer.isView(obj)) {
    var len = obj.length;
    cpy = new obj.constructor(len); // or just [] if that doesn't work
    for (var i = 0; i < len; ++i) {
      cpy[i] = clone(obj[i]);
    }
  } else {
    cpy = Object.create(obj.constructor.prototype); // or just {} if that doesn't work
    cpy.constructor = obj.constructor; // or leave it out if that doesn't work
    for (var i = 0, keys = Object.keys(obj), len = keys.length, i < len; ++i) {
      cpy[keys[i]] = clone(obj[keys[i]]);
    }
  }
  return cpy;
}

Didn't double check, though.

[cghawthorne]

Would you be open to adding this as a clone method in the protobuf.js library as a clear best practice? Since we use the generated code, it would be great if each message had a .clone method.

[dcodeIO]

Sure, could add a clone function to utility or so (protobuf.util.clone?). Not sure about adding it to messages directly because there might be a field named "clone" already.

[cghawthorne]

I was thinking a static method on the messages, similar to the encode/decode methods.

[khainke]

So, is

const original = Message.create({x:1, y:2});
const copy = MessageType.create(original);

the way to go here?

[domino14]

anyone?

[domino14]

http://gph.is/1TqiIcG