Allow mesh for sole visualization purpose

[jjyyxx]

The feature, motivation and pitch

For some context, I don't think MuJoCo supports material_index mechanism to associate different faces of a mesh to multiple materials (colors) as some more dedicated renderers do. Two workarounds are:

1. UV unwrap the whole mesh, assign different colors to different parts of the texture;
2. Split the mesh into multiple sub-meshes, then one material each sub-mesh.

I go for the second approach, then encounter several strange bugs related to planar mesh (in <= 3.2.7). In 3.2.8, setting inertia="shell" resolves this issue. But even in upcoming 3.2.8, all meshes are forced to compute inertia, re-centering, and compute BVH, which is wasteful for sole visualization purpose. So, I hope MuJoCo could implement a flag to skip such process for certain meshes.

As a generalization, currently MuJoCo supports mjr_uploadMesh to update mesh dynamically for visualization. But it only takes the meshid-th mesh from the associated mjModel. Since MuJoCo stores meshes in a memory-efficient flat format, adding new meshes or adding / removing vertices from a mesh is very difficult. I think, mesh for sole visualization purpose could safely skip the whole MuJoCo core engine, and be supplied directly to the render context or visual struct.

Alternatives

No response

Additional context

No response

[yuvaltassa]

Yes, this is a good idea.

[zacharias1219]

Hi all – I just want to add my support for this feature request! Being able to tag “visual‐only” meshes would be extremely helpful for reducing overhead on large or decorative elements. I’d be happy to contribute by working on a PR to implement such a feature. If maintainers have any guidelines or suggested approaches, please let me know and I’ll get started. Thanks!

[yuvaltassa]

I think that somehow adding meshes directly to the visual scene is probably not a good idea. If this is what the user wants it should happen further down the line from MuJoCo (in a hopefully-not-distant future world where MuJoCo has better interfacing with external renderers 🙂).

However, adding meshes that are visual-only can speed up the compilation process significantly. This implies two things, in terms of MJCF API. asset/mesh/inertia wants a new attribute value: none, which means, "don't use this mesh for inertia inference", and a completely new unrelated attribute that relates to contact, probably called convexhull = "false/true" (true by default), which means, if false, "I promise to never use this mesh for contacts, so no need to compute the convex hull".

In one way, adding these attributes is easy, since all they do is skip existing code. However the real difficulty would be in writing the tests that assert that the behaviour is indeed correct, that make sure that no existing assumptions in downstream code are violated (which would need to be changed/fixed), and importantly, that the right errors are thrown if used inconsistently. So basically I would not recommend this project for an inexperienced contributor.

[zacharias1219]

Hey @yuvaltassa , thanks for explaining the complexity involved in this feature. I understand now that adding attributes like inertia="none" and convexhull="false" requires careful testing and changes under the hood. I’ll hold off on creating a PR for this specific enhancement until I’m more familiar with MuJoCo’s internals and test requirements. In the meantime, I’d love to contribute elsewhere—so I’ll look out for other issues or enhancements that might be a better fit. Thanks again!