cancel
Showing results for 
Search instead for 
Did you mean: 

Archives Discussions

aj_guillon
Adept I

Memory Leaks? Valgrind?

Hello everyone,

I am running a GPU program, and it looks like there is a memory leak (on the host side) somewhere.  Normally I use valgrind for this, but the OpenCL library itself generates huge numbers of errors.  Is it possible that the OpenCL libraries and dependencies have a memory leak inside them?   Does AMD run valgrind on these libraries to ensure their quality?

0 Likes
9 Replies
nou
Exemplar

i don't know about OpenCL but fglrx generate huge amount of errors in valgrind. for example running glxgears with valgrind on system with fglxr.

but someone from AMD stated that fglrx do things which valgrind don't understand and considers it as a error.

0 Likes
HarryH
Journeyman III

It is possible to use valgrind if you use suppression files to suppress

the messages caused by libraries etc. See the valgrind documentation.

This way you will only see stuff caused by your own code.

0 Likes

well for me valgrind with openCL cause frozing whole computer and i must reset via SysRq.

0 Likes

I did add suppression files... and running with valgrind indicated that my part of the application was clean.  I have sinced turned off OpenCL profiling and the code does not appear to leak.... but it will take a bit of time to find out.  This application will be running for about 4 days, so any leak will be detrimental.  I made a few other changes along with turning off profiling which might cause the leak to disappear.  Have others let their OpenCL applications run for days?  Any experiences to share?

Also just because valgrind errors are suppressed doesn't mean that they aren't there... I did notice in another posting that SDK 2.3 is going to have better support for valgrind, which would be amazing.

0 Likes

I use OpenCl  with ATI card for scientific reseach of browning dynamics. usually i run my OpenCl application for about 5 days. Several times i'd received defunct process.  Several times after several hours of calculating. When i used NVIDIA card with this application no one  defunct process appeared.

0 Likes

aj_guillon,
We run valgrind on a per changelist basis on our samples. What you are seeing for a lot of the failures are cases where the runtime allocates memory and then hands it off to the driver to manage. Valgrind considers these lost memory even though the base driver deallocates them.
0 Likes

can't AMD release valgrind suppress list for using with fglrx?

so we can detect memory leaks in our OpenCL/GL applications.

0 Likes

At this time I'm happy to say my GPU application has been running for about 3 hours, and no sight of a memory leak yet.  I think that whatever the problem was, the adjustments to my own code, or turning off profiling fixed it.

0 Likes
HarryH
Journeyman III

I have written a couple of scripts to generate suppression files
when using libraries you have no control over.  For Debian based
distro's these scripts depend on the packages valgrind and php5-cli.
For other distro's RTM.

There are three scripts:

vg     - the main script
mksup  - converts valgrind output to suppression file format
dupsup - removes duplicates

Cut the scripts between the <------> and name them vg, mksup and dupsup.
Install the files in a directory in your PATH, and make them
executable (chmod 755). If necessary change the first lines of
the scripts if 'php' or 'sh' live somewhere else on your system.

Use as follows:

vg <program to test> <output suppressions file>

Example:

vg "rbl 128 100 128" OpenCL.sup

This command runs the OpenCL program 'rbl' with its parameters,
the quotes are required for a program with parameters. The output
is written to the file 'OpenCL.sup' and contains ALL suppressions.
This file must be inspected manually and any suppressions generated
for your own code removed

In the case of the above simple program I was left with 28 suppressions.

Of course use is not restricted to OpenCL libraries.

The reason for splitting the functionality among mksup and dupsup is
that you can merge files as follows and remove duplicates:

cat file1.sup file2.sup | dupsup > output.sup

I supply these scripts in the hope they will be useful but without
any guarantees. USE AT YOUR OWN RISK. As far as I am concerned these
are now in the public domain and you can do what you want with them.

Feedback via this thread please.

Harry

<----------- vg --------------> #!/bin/sh valgrind --log-fd=1 --leak-check=yes --gen-suppressions=all $1 | mksup | dupsup > $2 <-----------------------------> <----------- mksup --------------> #!/usr/bin/php <?php $count = 0; $fp = fopen("php://stdin", "r") or die("can't read stdin"); function insert_name($matches) { global $count; return "SUP" . $count++; } function remove_extra($matches) { return ""; } $insup = false; while (!feof($fp)) { $line = fgets($fp); if (false !== strpos($line, '{')){ $insup = true; } $line = preg_replace_callback('|<.*>|', "insert_name", $line); $line = preg_replace_callback('|^==.*$|', "remove_extra", $line); if ($insup && (strlen($line) > 1)){ echo $line; } if (false !== strpos($line, '}')){ $insup = false; } } fclose($fp); ?> <-----------------------------> <----------- dupsup --------------> #!/usr/bin/php <?php $fp = fopen("php://stdin", "r") or die("can't read stdin"); $sups = array(); $insup = false; while (!feof($fp)) { $line = trim(fgets($fp)); if(feof($fp)) break; if (false !== strpos($line, '{')){ $insup = true; } $supname = fgets($fp); // We don't need this but must read it $val = trim(fgets($fp)); while ($insup){ $line = trim(fgets($fp)); if (false !== strpos($line, '}')){ $insup = false; } else { $val .= '|' . $line; } } $sups[] = $val; } fclose($fp); $sups = array_unique($sups); foreach ($sups as $k => $v){ echo "{\n"; $v = ' ' . str_replace('|', "\n ", $v); echo ' SUP' . $k . "\n"; echo $v . "\n"; echo "}\n"; } ?> <----------------------------->

0 Likes