Parallelize the other loop of a nested for loop using allgather

Question

I am trying to parallelize a nested for loop below using allgather

for (int i=0; i<N1; i++) {
        for (int j=0; j<N0; j++)
            HS_1[i] += IN[j]*W0[j][i];
    }

Here N1 is 1000 and N2 is 764.

I have four processes and I just want to parallelize the outer loop. Is there a way to do it?

Victor Eijkhout · Accepted Answer · 2022-04-11 19:51:43Z

1

This looks like a matrix-vector multiplication. Let's assume that you've distributed the HS output vector. Each component needs the full IN vector, so you indeed need an allgather for that. You also need to distribute the W0 matrix: each process gets part of the i indices, and all of the j indices.

answered Apr 11, 2022 at 19:51

Victor Eijkhout

5,9902 gold badges29 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Sap BH Over a year ago

IN and W0 is available to all other process, In that case, will the following code work?

Sap BH Over a year ago

partition = N1/num_proc;</br> 	start = rank * partition;   	end = start + partition;    	double hs_1[partition]; 	for (int i = start; i < end; i++){ 		for (int j = 0; j < N0; j++) 			hs_1[i-start] += N[j] * W0[j][i]; 	}  	MPI_Allgatherv(hs_1, N1/rank, MPI_DOUBLE, HS_1, N1/rank, MPI_DOUBLE, MPI_COMM_WORLD);

Victor Eijkhout Over a year ago

That code looks correct, but it's pointless. You have your matrix on each process, but they only use a small part of it. And why do you gather the result at all? A good MPI application works completely distributed.

Collectives™ on Stack Overflow

Parallelize the other loop of a nested for loop using allgather

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related