Someone has suggested that I add an additional comment to what I said before.
There is another issue with float atomics, which is not a technical one but can easily lead to confusion for the programmer (and this is also true of using cmpxchg to emulate float atomics) which is that the answer you get is no longer deterministic because floating point arithmetic is order dependent. Of course what you end up with is arithmetic error, not completely missed computations as you might trying to do it without atomics at all, but that error can lead to significant differences in the final answer if you're not careful about what you're doing.
(2^25) + 1.0 is one obvious simple example of a lost computation. The answer to that would be 2^25 (I think, unless I miscalculate by a bit. Someone tried this on me in an interview once).