Archives Discussions

[vonti]

Is it possible for an IPI to "overtake" a memory write?

Hi all!

I was wondering if it was possible for an IPI to "overtake" a memory write.

For example:
1. CPU A writes some global variable (and the write happens to stay in the write buffer for a long time)
2. CPU A sends an IPI to CPU B
3. CPU B's IPI ISR reads the global variable

Is it theoretically possible in this scenario that the write buffer of CPU A has not been drained to the cache/memory when CPU B takes the interrupt and thus reads an old value of the variable?
I.e. is an explicit synchronisation instruction needed?

I couldn't find any information on that in chapter 7 (Memory System) of the Programmer's Manual Vol. 2. And while chapter 7.5.1 (Write Buffering) says that interrupts are serializing events that drain the write buffer, I suspect this only refers to the CPU receiving the interrupt, not the one sending it.

Cheers
Michael

[devperformance]

Hi Michael,

First, I assume that the write in the first step is a normal cacheable write. If it is, then it is not weakly ordered, but do read on! If it is a non-cacheable write then yes, you will have to use an SFENSE instruction or some other synchronization scheme.

To be safe , you could add my steps 1.1 and 2.1 below. This will insure that if the flag has been written, then the variable has also been written, again assuming all are cacheable writes.

For example:

1. CPU A writes some global variable (and the write happens to stay in the write buffer for a long time)

1.1 CPU A writes a global flag.
2. CPU A sends an IPI to CPU B

2.1 CPU B reads the global flag to verify it is set. If it is not, insert more code here [keep reading the flag, allow for a failure, etc.]
3. CPU B's IPI ISR reads the global variable.

Regards,

  Randy [AMD]

[vonti]

Hi Randy

Thanks for your reply!
Just to confirm: With "non-cacheable write", do you mean writes to uncached (UC) memory or non-temporal writes? I assume the latter, because the former wouldn't really make sense.

Is this documented in the Programmer's Manual? If yes, where? I couldn't find it even after an "exhaustive search". If no, I think it would be a good thing to include in a future revision.

Best regards
Michael