Use Multiple Host Threads on a Single GPU

Question

Now I'm using ROS to develop a CUDA Project. There are two nodes corresponding to 2 host threads that need to launch 2 different CUDA kernels concurrently.

So I'm wondering:

Can a single GPU be called by two host threads concurrently, if I didn't use CUDA Stream?(let's say the GPU resources are enough for launching these two kernels simutaneously.)
If not, Why?
If I didn't use cuda stream, are these two kernels bound to a single CUDA context or two?
If I use CUDA stream, are these two kernels bound to a single CUDA context or two?
If they are in two different CUDA context, how does the GPU handle their launching/operations as they are called concurrently?

I am now using a RTX 3090 with CUDA 11.0, and i9-10900k, which has 10 cores.

2 revs · Accepted Answer · 2021-01-14 08:32:47Z

2

Can a single GPU be called by two host threads concurrently, if I didn't use CUDA Stream?

Basically no. Threads from the same process share a common context. Contexts only expose concurrency when using streams. You might be able to get some degree of concurrency if you build your code to use a per thread default stream, but that is still using streams, just implicitly rather than explicitly.

If not, Why?

Because that is how NVIDIA designed it.

If I didn't use cuda stream, are these two kernels bound to a single CUDA context or two?

One. Since CUDA 4 the model is one context per process per device.

If I use CUDA stream, are these two kernels bound to a single CUDA context or two?

See above

If they are in two different CUDA context, how does the GPU handle their launching/operations as they are called concurrently?

They are not, so this is moot.

edited Jan 14, 2021 at 8:32

community wiki

2 revs
talonmies

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Use Multiple Host Threads on a Single GPU

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related