Persistent Data Race Warnings in C++ Parallel Program despite Mutex Usage

Question

I'm encountering persistent data race warnings in my C++ parallel program despite using mutexes. I've implemented a solution using OpenMP and mutexes to synchronize access to shared data, but I continue to receive ThreadSanitizer warnings regarding data races.

Code Overview:

I have a program that reads data from a file, performs parallel computations on it, and faces issues with the memory deallocation. The relevant portions of the code involve managing a shared vector using OpenMP parallelism and mutexes.

#include <fstream>
#include <sstream>
#include <iostream>
#include <vector>
#include <cstring>
#include <cmath>
#include <omp.h>
#include <mutex>

void readTextPointList(int n, int d, const std::string &strFileName, std::vector<std::vector<double>> &result)
{
  std::vector<std::vector<double>> PointList;
  std::string strLine;
  std::ifstream inFile(strFileName);

  if (inFile.is_open())
  {
    int i = 0;
    while (getline(inFile, strLine) && i < n)
    {
      if (strLine.size() != 0)
      {
        std::stringstream sin(strLine);
        std::vector<double> values(d);
        for (int j = 0; j < d; j++)
        {
          double aa;
          sin >> aa;
          values[j] = aa;
        }
        PointList.push_back(values);
        i++;
      }
    }
    inFile.close();

    // Use std::move to transfer ownership
    result = std::move(PointList);
  }
  else
  {
    std::cout << "Error: Unable to open file." << std::endl;
  }
}

std::mutex solveMutex;

void solve(std::vector<std::vector<double>> &data, size_t width, size_t start)
{
  std::vector<double> tuple(width);

  {
    std::lock_guard<std::mutex> lock(solveMutex); // Lock the critical section
    for (size_t i = 0; i < width; ++i)
    {
      tuple[i] = data[start][i];
    }
  }

  // No need to unlock explicitly; std::lock_guard takes care of it when it goes out of scope
}

auto main(int argc, char **argv) -> int
{
  if (argc < 2)
  {
    std::cout << "Usage: DIMENSIONALITY WINDOW" << std::endl;
    return 0;
  }
  size_t width = strtoul(argv[1], nullptr, 10);
  size_t datasize = strtoul(argv[2], nullptr, 10);

  // Read the data into a shared vector
  std::string filename = "mydata.txt";
  std::vector<std::vector<double>> data;

  readTextPointList(datasize, width, filename, data);

  int core_number = 2;

  int step = datasize / core_number + 1;

#pragma omp parallel for schedule(dynamic)
  for (int i = core_number - 1; i >= 0; i--)
  {
    solve(data, width, i * step);
  }

  return 0;
}

Compilation:

I'm compiling the code with the following commands:


g++ -Wall -g -Wextra -fsanitize=thread -pedantic -std=c++20 -O3 -m64 -fopenmp main.cpp

Issue Details:

Despite using a mutex (solveMutex) to protect critical sections, I'm consistently receiving ThreadSanitizer warnings related to data races. The warnings specifically point to memory deallocation functions such as operator delete.

What I've Tried:

I've attempted to protect critical sections with std::lock_guard<std::mutex>.
Checked for proper mutex usage and placement.

Environment:

Compiler: g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

I would appreciate any insights into why these data race warnings persist and how to resolve them effectively. Additionally, any suggestions on improving the overall design or handling memory would be valuable.

==================
WARNING: ThreadSanitizer: data race (pid=25822)
  Write of size 8 at 0x7b0800000060 by main thread:
    #0 operator delete(void*, unsigned long) ../../../../src/libsanitizer/tsan/tsan_new_delete.cpp:150 (libtsan.so.0+0x8e878)
    #1 __gnu_cxx::new_allocator<double>::deallocate(double*, unsigned long) /usr/include/c++/11/ext/new_allocator.h:145 (a.out+0x484c)
    #2 std::allocator<double>::deallocate(double*, unsigned long) /usr/include/c++/11/bits/allocator.h:199 (a.out+0x484c)
    #3 std::allocator_traits<std::allocator<double> >::deallocate(std::allocator<double>&, double*, unsigned long) /usr/include/c++/11/bits/alloc_traits.h:496 (a.out+0x484c)
    #4 std::_Vector_base<double, std::allocator<double> >::_M_deallocate(double*, unsigned long) /usr/include/c++/11/bits/stl_vector.h:354 (a.out+0x484c)
    #5 std::_Vector_base<double, std::allocator<double> >::~_Vector_base() /usr/include/c++/11/bits/stl_vector.h:335 (a.out+0x484c)
    #6 std::vector<double, std::allocator<double> >::~vector() /usr/include/c++/11/bits/stl_vector.h:683 (a.out+0x484c)
    #7 void std::destroy_at<std::vector<double, std::allocator<double> > >(std::vector<double, std::allocator<double> >*) /usr/include/c++/11/bits/stl_construct.h:88 (a.out+0x484c)
    #8 void std::_Destroy<std::vector<double, std::allocator<double> > >(std::vector<double, std::allocator<double> >*) /usr/include/c++/11/bits/stl_construct.h:149 (a.out+0x484c)
    #9 void std::_Destroy_aux<false>::__destroy<std::vector<double, std::allocator<double> >*>(std::vector<double, std::allocator<double> >*, std::vector<double, std::allocator<double> >*) /usr/include/c++/11/bits/stl_construct.h:163 (a.out+0x484c)
    #10 void std::_Destroy<std::vector<double, std::allocator<double> >*>(std::vector<double, std::allocator<double> >*, std::vector<double, std::allocator<double> >*) /usr/include/c++/11/bits/stl_construct.h:196 (a.out+0x484c)
    #11 void std::_Destroy<std::vector<double, std::allocator<double> >*, std::vector<double, std::allocator<double> > >(std::vector<double, std::allocator<double> >*, std::vector<double, std::allocator<double> >*, std::allocator<std::vector<double, std::allocator<double> > >&) /usr/include/c++/11/bits/alloc_traits.h:848 (a.out+0x484c)
    #12 std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::~vector() /usr/include/c++/11/bits/stl_vector.h:680 (a.out+0x484c)
    #13 main /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:90 (a.out+0x2c79)

  Previous read of size 8 at 0x7b0800000060 by thread T5 (mutexes: write M11):
    #0 solve(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&, unsigned long, unsigned long) /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:56 (a.out+0x310b)
    #1 main._omp_fn.0 /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:86 (a.out+0x329d)
    #2 <null> <null> (libgomp.so.1+0x1dc0d)

  Mutex M11 (0x55bd20bbf160) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4240 (libtsan.so.0+0x53908)
    #1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/11/bits/gthr-default.h:749 (a.out+0x30cf)
    #2 std::mutex::lock() /usr/include/c++/11/bits/std_mutex.h:100 (a.out+0x30cf)
    #3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/include/c++/11/bits/std_mutex.h:229 (a.out+0x30cf)
    #4 solve(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&, unsigned long, unsigned long) /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:53 (a.out+0x30cf)
    #5 main._omp_fn.0 /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:86 (a.out+0x329d)
    #6 <null> <null> (libgomp.so.1+0x1dc0d)

  Thread T5 (tid=25828, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x605b8)
    #1 <null> <null> (libgomp.so.1+0x1e25f)
    #2 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 (libc.so.6+0x29d8f)

SUMMARY: ThreadSanitizer: data race ../../../../src/libsanitizer/tsan/tsan_new_delete.cpp:150 in operator delete(void*, unsigned long)
==================
==================
WARNING: ThreadSanitizer: data race (pid=25822)
  Write of size 8 at 0x7b3000000000 by main thread:
    #0 operator delete(void*, unsigned long) ../../../../src/libsanitizer/tsan/tsan_new_delete.cpp:150 (libtsan.so.0+0x8e878)
    #1 __gnu_cxx::new_allocator<std::vector<double, std::allocator<double> > >::deallocate(std::vector<double, std::allocator<double> >*, unsigned long) /usr/include/c++/11/ext/new_allocator.h:145 (a.out+0x487a)
    #2 std::allocator<std::vector<double, std::allocator<double> > >::deallocate(std::vector<double, std::allocator<double> >*, unsigned long) /usr/include/c++/11/bits/allocator.h:199 (a.out+0x487a)
    #3 std::allocator_traits<std::allocator<std::vector<double, std::allocator<double> > > >::deallocate(std::allocator<std::vector<double, std::allocator<double> > >&, std::vector<double, std::allocator<double> >*, unsigned long) /usr/include/c++/11/bits/alloc_traits.h:496 (a.out+0x487a)
    #4 std::_Vector_base<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::_M_deallocate(std::vector<double, std::allocator<double> >*, unsigned long) /usr/include/c++/11/bits/stl_vector.h:354 (a.out+0x487a)
    #5 std::_Vector_base<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::~_Vector_base() /usr/include/c++/11/bits/stl_vector.h:335 (a.out+0x487a)
    #6 std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::~vector() /usr/include/c++/11/bits/stl_vector.h:683 (a.out+0x487a)
    #7 main /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:90 (a.out+0x2c79)

  Previous read of size 8 at 0x7b3000000000 by thread T5 (mutexes: write M11):
    #0 std::vector<double, std::allocator<double> >::operator[](unsigned long) /usr/include/c++/11/bits/stl_vector.h:1046 (a.out+0x30f6)
    #1 solve(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&, unsigned long, unsigned long) /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:56 (a.out+0x30f6)
    #2 main._omp_fn.0 /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:86 (a.out+0x329d)
    #3 <null> <null> (libgomp.so.1+0x1dc0d)

  Mutex M11 (0x55bd20bbf160) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4240 (libtsan.so.0+0x53908)
    #1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/11/bits/gthr-default.h:749 (a.out+0x30cf)
    #2 std::mutex::lock() /usr/include/c++/11/bits/std_mutex.h:100 (a.out+0x30cf)
    #3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/include/c++/11/bits/std_mutex.h:229 (a.out+0x30cf)
    #4 solve(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&, unsigned long, unsigned long) /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:53 (a.out+0x30cf)
    #5 main._omp_fn.0 /home/walid/Desktop/minimise/build sanitize/1_________decl.cpp:86 (a.out+0x329d)
    #6 <null> <null> (libgomp.so.1+0x1dc0d)

  Thread T5 (tid=25828, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x605b8)
    #1 <null> <null> (libgomp.so.1+0x1e25f)
    #2 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 (libc.so.6+0x29d8f)

SUMMARY: ThreadSanitizer: data race ../../../../src/libsanitizer/tsan/tsan_new_delete.cpp:150 in operator delete(void*, unsigned long)
==================
ThreadSanitizer: reported 2 warnings

As shown, solve is a no-op; it has no side effects, calling it is pointless. Is it the same in your actual code? — Igor Tandetnik
– Igor Tandetnik, Commented Jan 7, 2024 at 1:11
OpenMP libraries don't automatically work together with ThreadSanitizer. Make sure that this is supported on your implementation and what you need to potentially customize (e.g. compile the OpenMP library with thread sanitizer instrumentation). Generally thread sanitizer only guarantees no false positives if all code including libraries is compiled with instrumentation and only uses supported synchronization primitives (and it doesn't support all C++ constructs). — user17732522
– user17732522, Commented Jan 7, 2024 at 1:35
Your lock is quite pointless - you copy from read only to local vector under the lock. — Slava
– Slava, Commented Jan 7, 2024 at 2:21
OpenMP and C++ use very different threads. Not knowing what thread-sanitizer does, it wouldn't surprise me if it couldn't even see the OMP threads. If you really need locks I would use OpenMP locks. — Victor Eijkhout
– Victor Eijkhout, Commented Jan 7, 2024 at 3:34
GCC has no support for OpenMP-aware data race detection for ThreadSanitizer. The data race reports highlight the lack of understanding for OpenMP synchronization (barrier at the end of the parallel region). You can use Clang instead to get OpenMP-aware data race detection from libarcher. Unfortunately, Ubuntu packaging bricks libarcher. You can use the binary release from LLVM instead: github.com/llvm/llvm-project/releases/download/llvmorg-17.0.6/… — Joachim
– Joachim, Commented Jan 8, 2024 at 10:23

ShowCode · Accepted Answer · 2024-01-24 03:29:04Z

0

I have also encountered this problem recently. After my search , I found that need to build the OpenMP runtime library with TSAN Can I use Thread Sanitizer for OpenMP programs? and Is clang's thread sanitizer reporting a false positive?. Hope this can solve your problem. Have a good day:D

answered Jan 24, 2024 at 3:29

ShowCode

918 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Persistent Data Race Warnings in C++ Parallel Program despite Mutex Usage

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related