CUDA how to create arrays in runtime in kernel in shared memory?

Question

I have the task of large number of threads running, each doing a small matrix multiplication. All the small matrices have been loaded to the global memory. I wish to improve performance by letting each thread load its small matrices into shared memory, and then compute the product. But the problem is that I do not know the sizes of the matrices during compile time. So I cannot create variables as in __shared__ double mat1[XSIZE][YSIZE]. On PC, I would have made a dynamic allocation. But I do not know if I could do it on the shared memory. If calling malloc in a kernel would allocate only in global memory (assuming such a call is possible), that does not help either.

Is there a way to declare arrays during runtime in kernel? Is there any other way to resolve this problem?

talonmies · Accepted Answer · 2011-12-24 23:56:44Z

5

You can declare dynamically sized shared memory allocations in CUDA, like this

__global__ void kernel()
{
    extern __shared__ double *mat1;
}

And then launch your kernel like this

kernel<<<grid,block,XSIZE*YSIZE*sizeof(double)>>>();

This is discussed in more detail in the CUDA programming guide.

answered Dec 24, 2011 at 23:56

talonmies

72.7k35 gold badges204 silver badges296 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Elan Over a year ago

This method allows allocation of the same amount of memory to each of the thread dynamically. I have to populate each thread with differently sized matices, sizes whose upper and lower bounds I do not know yet. But thank you very much for the reply and the reference. It is a good starting point. Yes, it has been discussed in the programming guide in section B.16, as I found out from your hint.

talonmies Over a year ago

No, it allocates shared memory to each block dynamically. Shared memory has block scope in CUDA, not thread scope.

Collectives™ on Stack Overflow

CUDA how to create arrays in runtime in kernel in shared memory?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related