I am training an agent to perform a manipulation task (grasping an object placed on a table). When I visualize the learnt policy, I see that the robotic hand is able to penetrate both the table as well as the object in order to lift the object up. This effect is more pronounced when the mesh is decomposed into a set of convex meshes in the xml. When I load the original mesh (which is then interpreted as a convex hull), penetration still persists but the object seems to resist it to some extent. The visualizations and corresponding xml files can be accessed here. I have a couple of questions in this regard: 1. What could be causing this penetration effect (in decomposed mesh, hull, as well as table)? I tried reducing the timestep, but it doesn't get rid of the penetrations. 2. From the looks of it, the hand acceleration value is very high. Is this the main culprit? Would limiting this value solve the issue? If so, how can I do that? 3. Why is there such a big difference between the decomposed mesh and the hull? Does the decomposed mesh provide lower resistance than the original mesh? What values should I change to make it behave in a similar manner? 4. Is there something else terribly wrong in the xml model that could be causing this? Thanks for your help!
You could try to make the contacts stiffer to avoid penetration. If its just about grasping the pan, I would just use a capsule for a handle and a cylinder for the other part. I haven't looked at your model properly but make sure the table surface is modeled as a plane. Plane collisions are custom and typically much more stable. Finally I've noticed that you use six consecutive joints instead of just using a freejoint. I wouldn't do that. Edit: Ah sorry ... I just realized this is the support request forum ... I'll leave it here anyway.
Thank you Florian for doing my job! All of the above is correct. Some additional comments: When you decomposed the mesh you ended up with small/long objects (i.e. the handle) which makes it easier to penetrate into the hand. Once you have large penetrations with this geometry, it is difficult for the simulator to push the objects out. So all it takes is a couple of timesteps where the penetration first happens. Of course the contacts resist such artifacts at those critical timesteps, but contacts are soft, and the right combination of other contacts and applied forces can make it happen. Making the hand-object contacts stiffer, the handle thicker or the timestep smaller should help. Another important consideration in the context of learning: the learning algorithm is essentially looking for loopholes in the physics. Which is why there are many simulators out there but most of them are not usable in the context of learning. MuJoCo has proven to be usable but no simulator is perfect. So in some sense this is an adversarial game between you and your learning algorithm. The algorithm is trying to cheat. You are trying to stop it from cheating -- by modifying either the physics model or the reward function. If the hand acceleration is very high, this is probably because the learning algorithm discovered that high accelerations can lead to non-physical contact penetrations that are beneficial to it. You could simply put a penalty on large accelerations.
Hi Florian and Emo, thanks for your suggestions. Couple of points: 1. I modeled the table as a plane, but it's of no use in my case. 2. Regarding object modeling, I would rather have them in the original mesh topology, since I also access some other properties of those meshes. There are also quite a lot of objects, so it's not entirely feasible for me to manually decompose them into geometric primitives. When I searched on the forum, the recommended method for mesh decomposition was using VHACD. I am wondering if there is something inherently unstable about attaching multiple geoms to the same body in this way that allows the hand to go right through. Would more damping help? Currently it's close to 0. I set it to a very high value, but the penetrations don't go away. Moreover, like I mentioned, the penetrations even exist for the table and the original convex hull. Which makes me wonder if the real culprit is actually just the hand acceleration. Like you mentioned, adding some penalty on that might help balance things out a bit. I will try this out, thanks! 3. Regarding consecutive joints, I didn't quite catch the issue here. What's wrong with the current setup? Why is using a single free joint better? I am doing something very similar to this paper which was doing object relocation (refer to this). 4. Finally, what is the recommended way to set solref and solimp? I can see that we can include it right at the top of the model xml using the option tag. But individual geoms also have a solref/solimp attribute. Do we need to set it all places? Edit: Emo, you also mentioned that making the object thicker would help. How can I do this in the mesh case? Should it be specified in the material/density?
The plane helps stabilize contacts, but does not change softness and penetration. There is nothing wrong with attaching multiple geoms to the same body. However, if they geoms are small, it is easier for some other object to get stuck between them due to soft contacts. If you used a single geom, MuJoCo will do collisions with its convex hull, and so penetrations will be very rare. Btw, VHACD sounds great but does not always produce good results... but at least you can check the results visually. Using 6 scalar joints to represent a 6DoF floating joint is not recommended. Translations are fine, but rotations will suffer from gimbal locks when you use 3 hinges. The 6DoF joint uses a quaternion to represent rotations. Setting solerf and solimp (and most other parameters): you can specify defaults and apply them automatically to all elements of a given type, or you can set parameters "manually" in each element - which has the effect of overriding the defaults for that element only. If you want to make a single geom have harder contacts and all others have softer contacts, set the soft parameters as default for all geoms, and then specify the hard parameters in that single geom.