I spent the whole day trying to find out why glCompressedTexSubImage2D was returning INVALID_OPERATION when everything seemed to be fine.
I was working with FBOs & sub texture updates; so my guesses were:
Offset X, Y, Width & Height weren't being aligned to multiples of 4. Nope. That was correct.
My interpretation of GL_EXT_texture_compression_s3tc was incorrect and subimage was not possible with DXT textures.
I was causing a buffer overflow/memory corruption. This didn't seem to be the case either.
The calculation of the image data size was wrong.
The PBO's size was wrong
The PBO implementation was incorrect
I even doubted a driver bug. I had to go to Riccio's compressed samples to see if they were working. They were.
I checked if the Debug string extensions were enabled, they were. They were very quiet about the problem.
After 2 hours with CodeXL, I suddenly notice hwGamma in my code was true. Turns out, the GL buffer was being created with GL_COMPRESSED_SRGB_ALPHA_S3TC_DXT5_EXT; but in glCompressedTexSubImage2D, the format being specified was GL_COMPRESSED_RGBA_S3TC_DXT5_EXT.
My suggestion is... pleaseeee!!! For the love of God, add Debug messages when supplying wrong parameters to glCompressedTexSubImage** family of functions. Diagnostics would include:
Offsets/width/height don't meet the restrictions (i.e. SubImage is not supported for this format, the numbers aligned aligned to multiples of X, etc)
The size parameter is no the one the driver expects (i.e. 131.072 bytes vs 131.073 bytes)
The format paremeter doesn't match with the one the texture was created.
Tiny details like this would certainly help all developers and we will greatly appreciate, helping us to fix a problem that could be resolved in 2 minutes instead of wasting a whole day.
Simply an "INVALID_OPERATION" can be quite frustrating to deal with.