Unable to find injection library libsanitizer-collection.so while trying to use compute-sanitizer

Question

compute-sanitizer --tool memcheck my_cuda_program is what I am trying to do. I am trying this because I got Thread 1 my_cuda_program received signal CUDA_EXCEPTION_5, Warp Out-of-range Address. when I ran my program via cuda-gdb.

However, I get

========= COMPUTE-SANITIZER
========= Unable to find injection library libsanitizer-collection.so

this as a output. However, libsanitizer-collection.so exists in my /usr/lib/nvidia-cuda-toolkit.

My nvcc version is 12.4, which is why I am using compute sanitizer. I compile my program using cmake, which looks like this

cmake_minimum_required(VERSION 3.29.2)
project(nnbody VERSION 0.1.0 LANGUAGES CUDA CXX)

include(CTest)
enable_testing()

add_subdirectory(third-party/yaml-cpp) 

enable_language(CUDA)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++20")

set(CMAKE_CUDA_STANDARD 20)
set(CMAKE_CUDA_STANDARD_REQUIRED ON)
# set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -arch=sm_60")

# check openmp
find_package(OpenMP REQUIRED)
if (OPENMP_FOUND)
  set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
  set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
  set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}")
else ()
  message (FATAL_ERROR "The compiler does not support OpenMP.")
endif()

# check BLAS and LAPACK
find_package(BLAS REQUIRED)
find_package(LAPACK REQUIRED)
find_package(FFTW3 REQUIRED)

set(VTK_DIR "/usr/local/include/vtk-9.3/")
find_package(VTK REQUIRED)

file(GLOB ALL_FILES_SRC
     "src/*.cuh"
     "src/*.cu"
     "src/*.h"
     "src/*.hpp"
     "src/*.c"
     "src/*.cpp"
)

set_source_files_properties(ALL_FILES_SRC LANGUAGE CUDA)

add_executable(nnbody ${ALL_FILES_SRC})
target_link_libraries(nnbody PRIVATE ${BLAS_LIBRARIES} ${LAPACK_LIBRARIES} ${FFTW3_LIBRARIES} ${VTK_LIBRARIES} yaml-cpp)
set_property(TARGET nnbody PROPERTY CUDA_ARCHITECTURES 60 86)


set(CPACK_PROJECT_NAME ${PROJECT_NAME})
set(CPACK_PROJECT_VERSION ${PROJECT_VERSION})
include(CPack)

I don't know what to try..

Have you tried adding /usr/lib/nvidia-cuda-toolkit to LD_LIBRARY_PATH? — paleonix
– paleonix, Commented Apr 24, 2024 at 8:07
I tried, but it didn't work. Just to confirm, you are talking about adding the path to nvidia-cuda-toolkit to the env variable LD_LIBRARY_PATH, right? — ryupark1321
– ryupark1321, Commented Apr 24, 2024 at 13:36
You've got things in the wrong places. From a CUDA root directory, there should be various subdirectories. One of those is bin, where the compute-sanitizer tool would be found. Another subdirectory is, you guessed it, compute-sanitizer. In that subdirectory is where the compute sanitizer tool will look for the libraries it needs. If you have those libraries in /usr/lib/nvidia-cuda-toolkit (presumably perhaps your CUDA root directory) that is not the right place. — Robert Crovella
– Robert Crovella, Commented Apr 24, 2024 at 14:00
Is libsanitizer-collection.so directly under that path or is it in something like /usr/lib/nvidia-cuda-toolkit/lib? How did you install the Toolkit? — paleonix
– paleonix, Commented Apr 24, 2024 at 15:23
did you try adding /usr/local/cuda/bin to your $PATH? /usr/local/cuda/compute-sanitizer/libsanitizer-collection.so is where my libsanitizer is — AntPalmer
– AntPalmer, Commented Jul 23, 2024 at 1:55

BlueSky · Accepted Answer · 2024-10-03 17:56:31Z

I had the same bug where /usr/bin/compute-sanitizer would try to load libsanitizer-collection.so from /usr/bin, but it is located at /usr/lib/nvidia-cuda-toolkit/compute-sanitizer/libsanitizer-collection.so. (*buntu 22.04, NVIDIA driver version 550, CUDA 12.4)

You can confirm by using strace to see where compute-sanitizer tries to find the library:

$ strace compute-sanitizer ./main 2>&1 | grep libsanitizer-collection.so
stat("/usr/bin/libsanitizer-collection.so", 0x7ffe926f6c20) = -1 ENOENT (No such file or directory)
write(1, "========= Unable to find injecti"..., 70========= Unable to find injection library libsanitizer-collection.so

The actual location of libsanitizer-collection.so can be determined with the locate command:

$ locate libsanitizer-collection.so
/usr/lib/nvidia-cuda-toolkit/compute-sanitizer/libsanitizer-collection.so

I solved the problem by moving compute-sanitizer to the /usr/lib/nvidia-cuda-toolkit/compute-sanitizer directory and linking it back to /usr/bin.

sudo mv /usr/bin/compute-sanitizer /usr/lib/nvidia-cuda-toolkit/compute-sanitizer/compute-sanitizer
sudo ln -s /usr/lib/nvidia-cuda-toolkit/compute-sanitizer/compute-sanitizer /usr/bin/compute-sanitizer

I then get a new error, but that might be unrelated:

========= COMPUTE-SANITIZER
========= Target application terminated before first instrumented API call
========= Error: couldn't find exit code.

Elvir Crncevic · Accepted Answer · 2024-10-05 23:47:32Z

1

What worked for me is to just download the cuda-toolkit from nvidia directly (https://developer.nvidia.com/cuda-downloads?target_os=Linux) and make sure that the toolkit didn't come from any other source.

answered Oct 5, 2024 at 23:47

Elvir Crncevic

5154 silver badges21 bronze badges

1 Comment

StefanoTrv Jun 10 at 23:43

This is the answer. I removed all the other installations and followed the instructions at docs.nvidia.com/cuda/cuda-installation-guide-linux , including the Post installation actions. I also had to follow these instructions to have the paths always available: stackoverflow.com/questions/14637979/… (the paths I added are the ones in the post installation instructions)

lkl · Accepted Answer · 2024-12-31 11:14:37Z

1

If you deal the problem with moving the .so files to the /usr/bin, the error

========= COMPUTE-SANITIZER
========= Target application terminated before first instrumented API call
========= Error: couldn't find exit code.

will occur and without any way to solve.

just try to install a newer version cuda toolkit, my original is 11.5, and I change it to 12.6, then there is no error.

answered Dec 31, 2024 at 11:14

lkl

212 bronze badges

Comments

Maifee Ul Asad · Accepted Answer · 2025-09-22 15:39:43Z

0

You can try running:

compute-sanitizer --tool memcheck --injection-path=/usr/local/cuda-12.9/compute-sanitizer python3 benchmark/subprocess_benchmark.py

You can find your path by running:

locate libsanitizer-collection.so

For me they were:

/usr/lib/nvidia-cuda-toolkit/compute-sanitizer/libsanitizer-collection.so
/usr/local/cuda-12.9/compute-sanitizer/libsanitizer-collection.so

answered Sep 22 at 15:39

Maifee Ul Asad

4,68910 gold badges52 silver badges120 bronze badges

Collectives™ on Stack Overflow

Unable to find injection library libsanitizer-collection.so while trying to use compute-sanitizer

4 Answers 4

Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related