5 Replies Latest reply on Mar 19, 2013 10:37 AM by LeeHowes

    some insight - concurrency and parallelism


      Hello forum,


      While going though the book "Heterogeneous Computing With OpenCL"  i pondered upon a statement


      "Parallel programs must be concurrent,but concurrent programs need not be parallel". The author could have explained this statement more with an example , but i did not find any in the following sections of the book.


      Could anyone in the forum put some light into it with a real-life example ?





        • Re: some insight - concurrency and parallelism

          I think the book gives sufficient illustration on what is the difference between the two - Page 6 and 7.


          Concurrency is a very broad set. For example, As I am typing this reply, You are probably reading other threads. These are concurrent activities. Concurrency could just happen out of mere existence of universe. Everything is running concurrently at the same time.....But, it cannot be construed as parallel (i.e. done parallel with a purpose).


          Similarly even in computers, there are lot of things that can happen concurrently - e.g. a file being downloaded from internet, the user filling an e-form, music player playing music....This is due to multi-tasking. However, one cannot construe these programs as parallel programs. These are just programs running at same time (without even knowing that they are running concurrent).


          However. when you write a multi-threaded application -- the software explicitly knows that it is running concurrently. Done with a purpose. Concurrent by its DNA. These applications are called Parallel Programs. They are a subset of concurrence.


          The book is talking at a very high conceptual level. Just probably talking about a very subtle difference...


          Lee, You might want to correct/comment on this... Thanks,

          1 of 1 people found this helpful
          • Re: some insight - concurrency and parallelism

            Ah yes. This was an interesting topic of debate at the time, even among the authors. It is an important distinction, though.


            In a concurrent program multiple tasks are live at the same time, when looking at their start to end ranges. However, there is no requirement that those tasks are actually running together at any given time step. So on a single core CPU, in any modern operating system, you have multiple threads. Those threads are concurrent, but you have no task-level parallelism on the CPU. The threads are interleaved via time slicing. However, you may have instruction level parallelism or fine-grained data-parallelism going on whereby multiple instructions execute on each clock cycle on multiple different execution units or across vector units. When you add multiple cores in the system and can run more than one thread at once you get parallelism also from the point of view of the set of concurrent tasks in the system. So in that case you might have 100 way concurrency, and yet only 2 way parallelism.


            A quick google on this subject led me to this post over at stackoverflow: http://stackoverflow.com/questions/1050222/concurrency-vs-parallelism-what-is-the-difference


            Or here is a fascinating insight in terms of gophers: http://concur.rspace.googlecode.com/hg/talk/concur.html#slide-29