This is why I only advise to use frame gen in slower paced single player titles. It does feel smooth when frame gen is implemented properly, but you are correct. The input latency and higher frame time can make games feel sluggish.
With something like Starfield or Dragon's Dogma II it doesn't matter as much. Shooters however feel terrible with it on. Frame gen shouldn't be used in anything fast paced or competitive. Frame gen also causes ghosting in most titles, which makes the overall image quality less clear.
Single player games with frame gen on, typically feel fine. Avoid using frame gen in multiplayer and fast paced titles.
Personally, I am not against frame gen as a tool to give players with lower end hardware a smooth experience. I am against not optimizing games to perform on mid-level to high-end gaming hardware and using upscaling and frame gen as a fix for poor optimization. Unfortunately, this trend doesn't seem to be getting less popular, and the end users are paying for it.
Ryzen 7 7800X3D, ASUS ROG STRIX B650E-F GAMING WIFI, G.SKILL TRIDENT Z NEO 2x16GB DDR5 DRAM 6000MT/s CL30, AMD Radeon RX 7900 XTX, LIAN LI EDGE 1300, Corsair MP600 PRO NH 4TB