Systematic asset reuse for attached models

[jjyyxx]

The feature, motivation and pitch

asset/model + <attach> and the mjs_attachBody, mjs_attachFrame equivalent in model editing API is the most systematic and convenient way to combine components into a complex scene. But a potential limitation is that: when a model is attached multiple times, or in any other case that assets (mainly meshes) are effectively shared by multiple child models, the assets are unconditionally duplicated in the main model.

In our use case, the model size quickly grows to over 300MiB with over 1M mesh faces. The major sources of bloating are several child models with SDF collision geom (SDF marching cube resolution is hard-coded) attached several times. If we further scale up the scene (only by attaching the existing child models more times), the model size would surely exceed 1GiB. The bloating model size would not cause significant problem for "single model, multiple data" simulation on its own, but:

1. We are trying to store MJB binary for archiving and exact reproducibility;
2. We plan to experiment with certain domain randomization that goes beyond the scope of MjData, necessitating one model for each parallel worker.

Alternatives

1. Refactor the namespacing/scoping mechanism. The current prefix/suffix mechanism is somewhat inflexible.
2. Centralized asset management and asset sharing across models: It is theoretically possible with current mjModel interface, by changing asset pointers currently pointing to the main buffer to external buffers.

Additional context

No response

[yuvaltassa]

This is a valuable feature request, thanks.

I think there are two separate but related requests:

1. Stop copying assets unconditionally during attach, only copy assets which are actually referenced in the child.
2. Detect duplicate assets and merge them. This is obviously valuable during attach, but could potentially happen much later in the compilation step, for any model, whether created using attach or not.

Is this a correct reflection of your request, or did I miss something?

[jjyyxx]

Indeed, I agree. Though 1 would not improve the situation significantly for us, since our components are fairly "atomic".

[demoncoder-crypto]

I have done a very rough implementation of this, let me know if it works, i will refine it iteratively

[demoncoder-crypto]

#2540 Find it here