This is a new ERROR in RPR 1.7.
When working with a live Viewport and modelling you can no longer tab into Edit mode (or from Edit mode to Object mode) with a Live Viewport render running. Even pressing Tab a second time to "reset" back to where you were causes the Error screen displayed.
This has begun happening on "new" files set up entirely in RPR 1.7. I wasn't aware that it was happening on older v1.6 files while I was testing a few days ago.
However today working on a new scene it is showing as per the screenshots when going in and out of Edit mode with a Live Render running.