How to manage same CUDA kernel call from multiple CPU threads?

Question

I have a cuda kernel which works fine when called from a single CPU threads. However when the same is called from multiple CPU threads (~100), most of the kernel seems not be executed at all as the results comes out to be all zeros.Can someone please guide me how to resolve this problem?

In the current version of kernel I am using a cudadevicesynchronize() at the end of kernel call. Will adding a sync command before cudaMalloc() and kernel call be of any help in this case?

There is another thing which need some clarification. i.e. If two CPU threads executes the same cudaMalloc() command, will the later overwrite the former in GPU memory or will they create their own memory?

Thanks in advance for your help

Same kernel --> same array ---> probably same elements --->undefined behaviour. Try same kernel with a different name and different buffers . — huseyin tugrul buyukisik
– huseyin tugrul buyukisik, Commented Feb 14, 2014 at 13:34
are you suggesting to create copies of the kernel with different names for each thread? — Genutek
– Genutek, Commented Feb 14, 2014 at 13:43
If you can create a new .cu file programmatically from initial .cu file, yes, you should try. Compile time can increase. I used that once for my raytracer recursive function(fake recursivity) — huseyin tugrul buyukisik
– huseyin tugrul buyukisik, Commented Feb 14, 2014 at 13:44
I don't think there's any problem calling the same kernel 100 times from 100 different threads. Here's a simple example using pthreads. It's likely that if you do proper error checking, and review your data management carefully, you'll discover the issue. Or else post a simple example that fails. You should not have to create kernels with different names for each thread. You should not have two threads execute the same identical cudaMalloc() command. A pointer should be passed to cudaMalloc only once, before it is passed to cudaFree() — Robert Crovella
– Robert Crovella, Commented Feb 15, 2014 at 3:10

Maghoumi · Accepted Answer · 2014-02-14 14:48:07Z

5

Usually one CPU thread can be used for calling a CUDA kernel. However, since CUDA 4.0, multiple CPU threads can share context. You can use cuCtxSetCurrent to tie the context of the kernel to the current thread. More information about this API function can be found here.

Another workaround for this is to create a GPU worker thread that holds the context and pass any CUDA request to that thread.

Regarding your other question, without setting the context for the proper thread, I remember that cudaMalloc would not even execute (I work with JCuda so the behavior may be a little different). But if the context is currently set to the calling kernel, the memories will not be overwritten.

answered Feb 14, 2014 at 14:48

Maghoumi

3,4083 gold badges35 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Genutek Over a year ago

Thanks a lot for your reply. I would defintely use this in my future tasks, but for my current case, as TripleS suggested texture memory will have only one instance. I think the main issue is with the texture memory. If time permits, I will try replacing the texture with global memory just to check if the problem is still remains.

Collectives™ on Stack Overflow

How to manage same CUDA kernel call from multiple CPU threads?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related