I'm trying to read out the depth buffer of a <camera> mounted to a body. However, the problem I'm about to describe appears with the default camera, too. The attached screenshot shows `bin/simulate´ with the depth buffer view enabled. As you can see, the depth buffer, for some reason, only contains the *left half* of the camera frame, stretched weirdly to the original aspect ratio. Steps to reproduce: - Start bin/simulate from Mjpro 1.31 - Load https://github.com/openai/gym/blob/master/gym/envs/mujoco/assets/humanoid.xml - Press F4 to activate depth buffer view Expected result: depth field of view is equal to RGB field of view Actual result: depth field of view is only the left half of the RGB field of view Am I missing something here or is this a bug?
I was able to reproduce this in OSX, but it doesn't do it on Windows, even in Bootcamp running on same hardware. Could be due to some OS-specific difference in the way GLFW reports window size on retina displays... to be investigated.
Turns out this is a side effect of multi-sampling. Everyone else's video drivers resolve the sample buffers automatically before allowing glReadPixels to read data from the default/window framebuffer into client memory. But Apple's driver doesn't do that apparently, or at least not for the depth component. Instead it does something weird, causing the depth data to be repeated/stretched. Indeed reading double the data returns something like the complete image, but the right half is messed up. Easy fix: in the GLFW initialization sequence, change glfwWindowHint(GLFW_SAMPLES, 4); to glfwWindowHint(GLFW_SAMPLES, 0); I will investigate more, and hopefully come up with a solution that allows multi-sampling combined with reading the depth buffer on OSX. With offscreen rendering using an FBO, this is done via blit from a sample buffer to a resolve buffer (MuJoCo 1.40 will expose this feature; it already works nicely). But OpenGL does not specify how sample buffers operate for the default/window framebuffer. So we are at the mercy of the video driver, see here: https://www.opengl.org/wiki/Multisampling