parallel work load filtering objects

Discussion created by romulolus on Oct 5, 2011

Hi. I am wondering if it's worth trying to move part of my software over to openCL. Currently I am doing the following with c++ (using zlib and pthread).

(1 day = 288 zlib compressed flow log files from a cisco router)

Using threads, it decompresses the files and aggregates them per router and we end up with millions of unique records in a map (has to be indexed for uniqueness).

Then it loads a bunch of data plans which have a set of subnets. These are then compared to the map where ingress and egress data is calculated. This part can potentially run a lot of iterations, especially when the plan has a large subnet (/19) or 8192 ip addresses. 

At the moment there are around 500 plans to calculate data for. Is it worth moving this part of the processing (which pretty much just does some if's and addition in a loop over the flow map) to openCL for faster processing ?