I'm using mujoco-py with MuJoCo 2.0. On running the example script body_interaction.py (appropriately modified to run in headless mode), even though I supply the device_id, the code runs on CPU instead of using the GPU, without any error/warning message. From what I can tell by looking at the render function in mujoco-py, it calls MjRenderContextOffscreen, passing the device_id. Any pointers on what might be going on inside that prevents it from using the GPU? (I'm running this code on my university's cluster, on which other students routinely run GPU-accelerated headless rendering, albeit necessarily not with MuJoCo. So, GPU-acceleration should be enabled.)
mujoco-py is developed independently by OpenAI and I am not using, so not sure about the details. In general, to use headless rendering you need EGL for OpenGL context creation (unlike regular rendering which relies on X11). If you are using Mesa instead of EGL, rendering will be done on the CPU. To test if MuJoCo itself can do headless rendering, independent of the Python wrapper, check out this code sample: http://www.mujoco.org/book/programming.html#saRecord The online documentation in the above link has further information.