Server Gurus Discussions

[amd0]

I am trying to use AOCL on Linux Mint with gcc 11.

When I try and build the examples I get errors.

The output of the cmake command is ok.

But  the output from make is:

~/aocl/4.1.0/aocc/amd-sparse/examples/out_sparse$ make
[  4%] Building CXX object CMakeFiles/sample_csr2m.dir/sample_csr2m.cpp.o
[  9%] Linking CXX executable sample_csr2m
/usr/bin/ld: warning: libomp.so, needed by /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libflang.so, needed by /home/USER/aocl/4.1.0/aocc/lib_LP64/libflame.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libflame.so: undefined reference to `__kmpc_end_single@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_dispatch_init_4@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libflame.so: undefined reference to `omp_get_thread_num@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libflame.so: undefined reference to `omp_get_max_threads@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_for_static_init_4u@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_dispatch_next_4@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libblis-mt.so: undefined reference to `omp_get_max_active_levels@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libflame.so: undefined reference to `__kmpc_single@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libblis-mt.so: undefined reference to `omp_get_active_level@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libflame.so: undefined reference to `__kmpc_barrier@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libblis-mt.so: undefined reference to `__kmpc_end_critical@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libblis-mt.so: undefined reference to `__kmpc_critical@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_push_num_threads@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_for_static_fini@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libblis-mt.so: undefined reference to `omp_get_num_threads@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_global_thread_num@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_fork_call@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libaoclsparse.so: undefined reference to `__kmpc_for_static_init_4@VERSION'
/usr/bin/ld: /home/USER/aocl/4.1.0/aocc/lib_LP64/libblis-mt.so: undefined reference to `__kmpc_for_static_init_8@VERSION'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/sample_csr2m.dir/build.make:102: sample_csr2m] Error 1
make[1]: *** [CMakeFiles/Makefile2:103: CMakeFiles/sample_csr2m.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

The omp libraries are installed:

$ ldconfig -p | grep libomp
	libomp.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/libomp.so.5

But the libflang.so - I don't have and don't know what to do about.

I do have:

 ldconfig -p | grep fortran
	libgfortran.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/libgfortran.so.5

Am I missing something simple?

EDIT: 1) This forum is hard to write in - surely amd's vast empire can afford a proper forum for scientific computing using these numerical libraries.

2) Installing the aocc compiler first, followed by the aocl library binaries compiled using aocc worked for me. To get the samples to work you need to add a lot of environmental variables - see the read me for each example.

If amd actually want general researchers to use this library, in my opinion, they should spend a half a day making it easier to run examples and add some more general information for us non-experts.

And it would be helpful for non gcc/llvm experts if someone could write a half a page explaining the differences between the library compiled with aocc vs gcc, and which one should I choose.

I also note that starting with amds AOCL has been pretty painful in comparison with starting with CUDA and intels MLK library.

[amd0 · Accepted solution]

Thanks for the reply but:

1) I have already followed the "step by step procedure to build AOCL GCC Libraries " - and it didn't work, as I described. Specifically libflang.so is required and not present, and the omp library isn't found.

My guess is that you need to install AOCC first, even if you are using the gcc compiled libraries.

2) In your test on Ubuntu-22.04 -  was AOCC already installed?

 Someone needs to test if the GCC compiled libraries work without installing AOCC first. Note that on a normal build for C programming on Linux Mint, compilers like flang are not present.

Maybe there is a check happening on the AOCL gcc libries that is unnecessary and causing the samples not to work.

[amd0 · Accepted solution]

Hi shrjoshi,

thanks for the reply.

I installed the gcc version again and now the gcc examples work.

I think it might be that I needed a different version of the omp library that I must have installed in the mean time. For example, using the gcc library the sample_csr2m now works:

$ ./sample_csr2m #
AOCL-Sparse 4.1.0 Build 20230719
Invoking aoclsparse_scsr2m with aoclsparse_stage_nnz_count..DONE
C_M C_N nnz_C
3 3 6
csr_row_ptr_C:
0 2 3 6
Invoking aoclsparse_scsr2m with aoclsparse_stage_finalize..DONE
csr_col_ind_C:
0 1 1 0 1 2
csr_val_C:
1 10 12 4 32 15

And the linked libraries are:

$ ldd ./sample_csr2m
linux-vdso.so.1 (0x00007ffde64eb000)
libaoclsparse.so.4.1.0.0 => /home/cw1/aocl/4.1.0/gcc//lib/libaoclsparse.so.4.1.0.0 (0x00007f72424b7000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f724228b000)
libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f7242063000)
libflame.so.4 => /home/cw1/aocl/4.1.0/gcc//lib/libflame.so.4 (0x00007f7240fae000)
libblis-mt.so.4 => /home/cw1/aocl/4.1.0/gcc//lib/libblis-mt.so.4 (0x00007f7240796000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f724074c000)
libpthread.so.0 => /usr/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7240747000)
libm.so.6 => /usr/lib/x86_64-linux-gnu/libm.so.6 (0x00007f7240660000)
libgcc_s.so.1 => /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7240640000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7242529000)
libgfortran.so.5 => /usr/lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f7240363000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f724031b000)
librt.so.1 => /usr/lib/x86_64-linux-gnu/librt.so.1 (0x00007f7240316000)

note that before I had /lib/x86_64-linux-gnu/libomp.so.5 installed whereas now I also have libgomp.so.1 installed.

As regards the original libflang.so error, I must have not been including the path to /lib/x86_64-linux-gnu/libgfortran.so.5 in my bash.rc.

I have encountered another problem in the scalapack examples. I think it has to do with the MPI part:

Readme says : 1. Set PATH and LD_LIBRARY_PATH appropriately to the MPI installation.

I have set the following:

export LD_LIBRARY_PATH='/usr/include/x86_64-linux-gnu/openmpi/lib/':$LD_LIBRARY_PATH
export LIBRARY_PATH='/usr/include/x86_64-linux-gnu/openmpi/lib/':$_LIBRARY_PATH

However, on build I get this:

$ make "BLASLIB=$AOCL_ROOT/lib/libblis-mt.a" "LAPACKLIB=$AOCL_ROOT/lib/libflame.a" "SCALAPACKLIB=$AOCL_ROOT/lib/libscalapack.a"
mpif90 -c -cpp -DUSE_BLAS -DF2C -O3 -fopenmp -DDYNAMIC_WORK_MEM_ALLOC -DAOCL_DTL_ADVANCED_TRACE_ENABLE pdscaex.f
mpif90 -c -cpp -DUSE_BLAS -DF2C -O3 -fopenmp -DDYNAMIC_WORK_MEM_ALLOC -DAOCL_DTL_ADVANCED_TRACE_ENABLE pdscaexinfo.f
mpif90 -cpp -DUSE_BLAS -DF2C -O3 -fopenmp -DDYNAMIC_WORK_MEM_ALLOC -DAOCL_DTL_ADVANCED_TRACE_ENABLE -o xdscaex pdscaex.o pdscaexinfo.o /home/USER/aocl/4.1.0/gcc/lib/libscalapack.a /home/USER/aocl/4.1.0/gcc/lib/libflame.a /home/USER/aocl/4.1.0/gcc/lib/libblis-mt.a
/bin/ld: /home/USER/aocl/4.1.0/gcc/lib/libscalapack.a(igebs2d_.c.o): relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
make: *** [Makefile:28: pdscaex] Error 1

[amd0]

I have got this to work.

Solution: I installed the amd aocc compiler first and then installed the aocl-linux-aocc-4.1.0.tar.gz version of the library.

I could then build the examples and they worked.

You need a lot of things in the environment variables, and note at the end here I add paths specific to the sparse library:

export LIBRARY_PATH=/usr/lib64:/usr/lib/x86_64-linux-gnu:/usr/lib:/usr/lib32:$LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/lib64:/usr/lib/x86_64-linux-gnu:/usr/lib:/usr/lib32:$LD_LIBRARY_PATH
export PATH=/home/USER/aocc-compiler-4.1.0/bin:$PATH
export PATH=/home/USER/aocc-compiler-4.1.0/share/opt-viewer:$PATH
export LIBRARY_PATH=/home/USER/aocc-compiler-4.1.0/lib:/home/USER/aocc-compiler-4.1.0/lib32:$LIBRARY_PATH
export LD_LIBRARY_PATH=/home/USER/aocc-compiler-4.1.0/ompd:/home/USER/aocc-compiler-4.1.0/lib:/home/USER/aocc-compiler-4.1.0/lib32:$LD_LIBRARY_PATH

if [[ -z $C_INCLUDE_PATH ]];then
    export C_INCLUDE_PATH=/home/USER/aocc-compiler-4.1.0/include
else
    export C_INCLUDE_PATH=$C_INCLUDE_PATH:/home/USER/aocc-compiler-4.1.0/include
fi


if [[ -z $CPLUS_INCLUDE_PATH ]];then
    export CPLUS_INCLUDE_PATH=/home/USER/aocc-compiler-4.1.0/include
else
    export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/home/USER/aocc-compiler-4.1.0/include
fi

export AOCL_ROOT=/home/USER/aocl/4.1.0/aocc;
export C_INCLUDE_PATH=/home/USER/aocl/4.1.0/aocc/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=/home/USER/aocl/4.1.0/aocc/include:$CPLUS_INCLUDE_PATH
export LD_LIBRARY_PATH=/home/USER/aocl/4.1.0/aocc/lib:$LD_LIBRARY_PATH
export LIBRARY_PATH=/home/USER/aocl/4.1.0/aocc/lib:$LIBRARY_PATH

export SPARSE_ROOT=$HOME/amd/aocl-sparse 
export LD_LIBRARY_PATH=$SPARSE_ROOT/lib:$LD_LIBRARY_PATH

[shrjoshi]

Hi @amd0 

Thank you for writing to Server Guru forum.
Based on the information provided in your first post it seemed like you were using aocl-linux-aocc-4.1.0.tar.gz instead of aocl-linux-gcc-4.1.0.tar.gz . Hence using AOCC compiler was the right option.

For AOCL with GCC compiler use : aocl-linux-gcc-4.1.0.tar.gz 
For AOCL with AOCC compiler use : aocl-linux-aocc-4.1.0.tar.gz

[amd0]

Originally I used the gcc version - this didn't work due to no flang present and an issue no finding omp.

Maybe you need the aocc compiler installed even if you are using the gcc compiled version of aocl?

Even though I can build and run the examples, I still get this error about crt* with  the aocc compiler check utility:

 ./AOCC-prerequisites-check.sh 

Failing Checks:
	1)WARNING: Folder containing 64-bit crt files crt1.o crti.o crtn.o are not present in default path /usr/lib64/ 
	Without this 64-bit compilation will fail

	2)WARNING: Folder containing 32-bit crt files crt1.o crti.o crtn.o are not present in default path /usr/lib/ 
	Without this 32-bit compilation will fail


Passing Checks:
	1) AOCC clang compiler bin => /home/cw1/aocc-compiler-4.1.0/bin
	2) AOCC clang++ compiler bin => /home/cw1/aocc-compiler-4.1.0/bin
	3) AOCC flang compiler bin => /home/cw1/aocc-compiler-4.1.0/bin
	4) Glibc version => 2.35-0ubuntu3.4
	5) AMD LibM (libalm) => /usr/lib32
	6) libtinfo.so => /usr/lib/x86_64-linux-gnu/libtinfo.so.6.3
	7) Python 3.5 or above is present
	8) In PATH opt-viewer is present
	9) Check for 64-bit libxml2.so library passed => /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 
	10) Check for 64-bit OpenMP library => /home/cw1/aocc-compiler-4.1.0/lib32/libomp.so
	11) Check for 32-bit OpenMP library => /home/cw1/aocc-compiler-4.1.0/lib32/libomp.so
	12) Check for 64-bit libstdc++.so library passed => /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 
	13) Check for 32-bit libstdc++.so library passed => /usr/libx32/libstdc++.so.6.0.30 
	14) Check for 64-bit libc.so library passed => /lib/x86_64-linux-gnu/libc.so.6 
	15) Check for 32-bit libc.so library passed => /libx32/libc.so.6 
	16) Check for 64-bit libm.so library passed => /lib/x86_64-linux-gnu/libm.so.6 
	17) Check for 32-bit libm.so library passed => /libx32/libm.so.6 
	18) Check for 64-bit libgcc_s.so library passed => /lib/x86_64-linux-gnu/libgcc_s.so.1 
	19) Check for 32-bit libgcc_s.so library passed => /libx32/libgcc_s.so.1 
	20) Check for 64-bit libz.so library passed => /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 
	21) Check for 32-bit libz.so library passed => /usr/lib/i386-linux-gnu/libz.so.1.2.11 
	22) Check for 64-bit libquadmath.so.0 library passed => /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 
	23) Check for 32-bit libquadmath.so.0 library passed => /usr/libx32/libquadmath.so.0.0.0 

Note they are installed:

/aocc-compiler-4.1.0$ find  /usr -name crt1*
/usr/lib32/crt1.o
/usr/x86_64-linux-gnu/lib32/crt1.o
/usr/x86_64-linux-gnu/lib/crt1.o
/usr/lib/x86_64-linux-gnu/crt1.o
/usr/libx32/crt1.o

/aocc-compiler-4.1.0$ find  /usr -name crtn*
/usr/lib32/crtn.o
/usr/x86_64-linux-gnu/lib32/crtn.o
/usr/x86_64-linux-gnu/lib/crtn.o
/usr/lib/x86_64-linux-gnu/crtn.o
/usr/libx32/crtn.o

/aocc-compiler-4.1.0$ find  /usr -name crti*
/usr/lib32/crti.o
/usr/x86_64-linux-gnu/lib32/crti.o
/usr/x86_64-linux-gnu/lib/crti.o
/usr/lib/x86_64-linux-gnu/crti.o
/usr/libx32/crti.o

[shrjoshi]

Hi @amd0 ,

Please find below step by step procedure to build AOCL GCC Libraries and running of aocl-sparse example.  This is tested on Ubuntu-22.04.

• Please download aocl-linux-gcc-4.1.0.tar.gz and untar the AOCL GCC 4.1.0

• cd to aocl-linux-gcc-4.1.0 and run ./install.sh. This will install AOCL libraries with GCC in default location [${HOME}/aocl/4.1.0/gcc].

• Source AOCL libraries [${HOME}/aocl/4.1.0/gcc/amd-libs.cfg] after successful installation.

• cd to amd-sparse/examples and export SPARSE_ROOT and LD_LIBRARY_PATH.
  export SPARSE_ROOT=${HOME}/aocl/4.1.0/gcc
  export LD_LIBRARY_PATH=$SPARSE_ROOT/lib:$LD_LIBRARY_PATH

• Execute cmake and make in examples folder.

• Below are the cmake output

cmake S . -B out_sparse

-- The CXX compiler identification is GNU 11.4.0

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Check for working CXX compiler: /usr/bin/c++ - skipped

-- Detecting CXX compile features

-- Detecting CXX compile features - done

-- LAPACK_LIBRARIES= /home/amd/aocl/4.1.0/gcc/lib_LP64/libflame.so;/home/amd/aocl/4.1.0/gcc/lib_LP64/libblis-mt.so

-- LAPACK_INCLUDE_DIRS= /home/amd/aocl/4.1.0/gcc/include_LP64

-- BLIS_INCLUDE_DIRS= /home/amd/aocl/4.1.0/gcc/include_LP64

-- Found OpenMP_CXX: -fopenmp (found version "4.5")

-- Found OpenMP: TRUE (found version "4.5")

-- AOCL-Sparse header and library locations

-- Headers: /home/amd/aocl/4.1.0/gcc/include_LP64

-- Library: /home/amd/aocl/4.1.0/gcc/lib_LP64/libaoclsparse.so

-- Configuring done

-- Generating done

-- Build files have been written to: /home/amd/aocl-install/aocl-linux-gcc-4.1.0/amd-sparse/examples/out_sparse

 After successful installation, below files are generated :

 sample_itsol_d_cg 

 sample_itsol_d_gmres_rci 

 sample_itsol_s_gmres

 sample_csr2m 

 sample_itsol_d_cg_rci 

 sample_itsol_s_cg         

 sample_itsol_s_gmres_rci

 sample_dtrsv 

 sample_itsol_d_gmres   

 sample_itsol_s_cg_rci     

 sample_spmv

As for the warning section of AOCC compiler related to crt obj files if setenv file for AOCC is sourced correctly, compiler should work fine. Please let us know if you are facing any issues with compiling code using AOCC. 
Please let us know if there is any further clarification is required.

[amd0 · Accepted solution]

Thanks for the reply but:

1) I have already followed the "step by step procedure to build AOCL GCC Libraries " - and it didn't work, as I described. Specifically libflang.so is required and not present, and the omp library isn't found.

My guess is that you need to install AOCC first, even if you are using the gcc compiled libraries.

2) In your test on Ubuntu-22.04 -  was AOCC already installed?

 Someone needs to test if the GCC compiled libraries work without installing AOCC first. Note that on a normal build for C programming on Linux Mint, compilers like flang are not present.

Maybe there is a check happening on the AOCL gcc libries that is unnecessary and causing the samples not to work.

[shrjoshi]

Hi @amd0 

Can you please share the o/p of the below command : 
"ldd sample_csr2m"
sample_csr2m is the executable created in out_sparse[amd-sparse/examples] folder after successful make.
Please share the ldd command of the executable from AOCL-GCC file.

In your test on Ubuntu-22.04 -  was AOCC already installed?
On our test system AOCC was not previously installed. 

[amd0 · Accepted solution]

Hi shrjoshi,

thanks for the reply.

I installed the gcc version again and now the gcc examples work.

I think it might be that I needed a different version of the omp library that I must have installed in the mean time. For example, using the gcc library the sample_csr2m now works:

$ ./sample_csr2m #
AOCL-Sparse 4.1.0 Build 20230719
Invoking aoclsparse_scsr2m with aoclsparse_stage_nnz_count..DONE
C_M C_N nnz_C
3 3 6
csr_row_ptr_C:
0 2 3 6
Invoking aoclsparse_scsr2m with aoclsparse_stage_finalize..DONE
csr_col_ind_C:
0 1 1 0 1 2
csr_val_C:
1 10 12 4 32 15

And the linked libraries are:

$ ldd ./sample_csr2m
linux-vdso.so.1 (0x00007ffde64eb000)
libaoclsparse.so.4.1.0.0 => /home/cw1/aocl/4.1.0/gcc//lib/libaoclsparse.so.4.1.0.0 (0x00007f72424b7000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f724228b000)
libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f7242063000)
libflame.so.4 => /home/cw1/aocl/4.1.0/gcc//lib/libflame.so.4 (0x00007f7240fae000)
libblis-mt.so.4 => /home/cw1/aocl/4.1.0/gcc//lib/libblis-mt.so.4 (0x00007f7240796000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f724074c000)
libpthread.so.0 => /usr/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7240747000)
libm.so.6 => /usr/lib/x86_64-linux-gnu/libm.so.6 (0x00007f7240660000)
libgcc_s.so.1 => /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7240640000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7242529000)
libgfortran.so.5 => /usr/lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f7240363000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f724031b000)
librt.so.1 => /usr/lib/x86_64-linux-gnu/librt.so.1 (0x00007f7240316000)

note that before I had /lib/x86_64-linux-gnu/libomp.so.5 installed whereas now I also have libgomp.so.1 installed.

As regards the original libflang.so error, I must have not been including the path to /lib/x86_64-linux-gnu/libgfortran.so.5 in my bash.rc.

I have encountered another problem in the scalapack examples. I think it has to do with the MPI part:

Readme says : 1. Set PATH and LD_LIBRARY_PATH appropriately to the MPI installation.

I have set the following:

export LD_LIBRARY_PATH='/usr/include/x86_64-linux-gnu/openmpi/lib/':$LD_LIBRARY_PATH
export LIBRARY_PATH='/usr/include/x86_64-linux-gnu/openmpi/lib/':$_LIBRARY_PATH

However, on build I get this:

$ make "BLASLIB=$AOCL_ROOT/lib/libblis-mt.a" "LAPACKLIB=$AOCL_ROOT/lib/libflame.a" "SCALAPACKLIB=$AOCL_ROOT/lib/libscalapack.a"
mpif90 -c -cpp -DUSE_BLAS -DF2C -O3 -fopenmp -DDYNAMIC_WORK_MEM_ALLOC -DAOCL_DTL_ADVANCED_TRACE_ENABLE pdscaex.f
mpif90 -c -cpp -DUSE_BLAS -DF2C -O3 -fopenmp -DDYNAMIC_WORK_MEM_ALLOC -DAOCL_DTL_ADVANCED_TRACE_ENABLE pdscaexinfo.f
mpif90 -cpp -DUSE_BLAS -DF2C -O3 -fopenmp -DDYNAMIC_WORK_MEM_ALLOC -DAOCL_DTL_ADVANCED_TRACE_ENABLE -o xdscaex pdscaex.o pdscaexinfo.o /home/USER/aocl/4.1.0/gcc/lib/libscalapack.a /home/USER/aocl/4.1.0/gcc/lib/libflame.a /home/USER/aocl/4.1.0/gcc/lib/libblis-mt.a
/bin/ld: /home/USER/aocl/4.1.0/gcc/lib/libscalapack.a(igebs2d_.c.o): relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
make: *** [Makefile:28: pdscaex] Error 1

[shrjoshi · Accepted solution]

Hi @amd0 

Thank you for reporting this observation.
We were able to reproduce this at our end and will work on fixing it.
In the meantime, if you need the scalapack example to work you can use shared libraries instead of static ones i.e. 

make "BLASLIB=$AOCL_ROOT/lib/libblis-mt.so" "LAPACKLIB=$AOCL_ROOT/lib/libflame.so" "SCALAPACKLIB=$AOCL_ROOT/lib/libscalapack.so"

".

[shrjoshi]

Hi @amd0 

Can you please share the below details for Amd-Scalapack-GCC example issue.
1. GCC version
2. Kernel and OS details.
3. H/w details [lscpu]

[amd0]

As requested:

USER@USER-Legion-7:~$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

USER@USER-Legion-7:~$ uname -r
5.15.0-88-generic

USER@USER-Legion-7:~$ uname -a
Linux USER-Legion-7 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

USER@USER-Legion-7:~$ cat /etc/lsb-release
DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=21.2
DISTRIB_CODENAME=victoria
DISTRIB_DESCRIPTION="Linux Mint 21.2 Victoria"

USER@USER-Legion-7:~$ lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  16
  On-line CPU(s) list:   0-15
Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 7 5800H with Radeon Graphics
    CPU family:          25
    Model:               80
    Thread(s) per core:  2
    Core(s) per socket:  8
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU max MHz:         3200.0000
    CPU min MHz:         1200.0000
    BogoMIPS:            6387.93
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mc
                         a cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall n
                         x mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_go
                         od nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl p
                         ni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe
                          popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy sv
                         m extapic cr8_legacy abm sse4a misalignsse 3dnowprefetc
                         h osvw ibs skinit wdt tce topoext perfctr_core perfctr_
                         nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate
                          ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 sm
                         ep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflusho
                         pt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc c
                         qm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf 
                         xsaveerptr rdpru wbnoinvd cppc arat npt lbrv svm_lock n
                         rip_save tsc_scale vmcb_clean flushbyasid decodeassists
                          pausefilter pfthreshold avic v_vmsave_vmload vgif v_sp
                         ec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_r
                         ecov succor smca fsrm
Virtualization features: 
  Virtualization:        AMD-V
Caches (sum of all):     
  L1d:                   256 KiB (8 instances)
  L1i:                   256 KiB (8 instances)
  L2:                    4 MiB (8 instances)
  L3:                    16 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-15
Vulnerabilities:         
  Gather data sampling:  Not affected
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Not affected
  Spec rstack overflow:  Mitigation; safe RET, no microcode
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
                          and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer
                          sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIB
                         P always-on, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected

[shrjoshi]

Hello @amd0 

Last week, AOCL 4.2 has been released.  Pre-built binaries can be executed in GCC v12.2 and GCC v13.1 environments. Other than these versions, one need to recompile all required AOCL prerequisites with the specific GCC versions.

Link: : https://www.amd.com/en/developer/aocl.html

Please refer the steps mentioned in the attached image to install AOCL ScaLAPACK (with blis-mt) using GCC compiler.
All these steps are available in user guide.
Note the usage of "-fopenmp and -lstdc++" before blas and lapack library while building scalapack library.

Once the static libraries are available, execute scalapack example as mentioned in user guide.

The site was not allowing to input the text, hence the image.
Sorry for the inconvenience.

Please reach out to toolchainsupport@amd.com for further clarification if required.