1

I am working with MPI and I have to send parts of an array to different processes. As an example, consider 3 processes. Then I need to send the red elements to the first process, the greed to the second and the black to the third process.

enter image description here

I know I could use Scatterv twice, but I want to minimize the communication between processes and the real array that I'm splitting apart is huge. Does anyone have a suggestion on how I can accomplish this?

Here is my attempt with a derived data type:

#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>

void print_array(int *array,int n){
    int i;
    printf("\t[");
    for (i=0; i<n; i++) {
        printf(" %d",array[i]);
    }
    printf("]\n");
}

int main(int argc, char **argv){

int rank,world_size,i,n = 16, block_count = 2;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &world_size);

int *array = malloc(n * sizeof(int));
for (i=0; i<n; i++) { array[i]=i;}
if (rank==0) { print_array(array,n);}

int *sendcounts = malloc(world_size * sizeof(int));
int *reccounts = malloc(world_size * sizeof(int));
int *displs = malloc(world_size * sizeof(int));

sendcounts[0]=3; sendcounts[1]=3; sendcounts[2]=2;
displs[0]=0; displs[1]=3; displs[2]=6;

for (i=0; i<world_size; i++) {
    reccounts[i] = sendcounts[i]*block_count;
}

int root = 0;
int *recvbuf = malloc(reccounts[rank] * sizeof(int));
MPI_Datatype newtype;
MPI_Type_contiguous(block_count, MPI_INT, &newtype);
MPI_Type_commit(&newtype);

if (rank==0) {
    MPI_Scatterv(array, sendcounts, displs,
                 newtype, recvbuf, sendcounts[rank],
                 newtype, root, MPI_COMM_WORLD);
}
else {
    MPI_Scatterv(NULL, sendcounts, displs,
                 newtype, recvbuf, reccounts[rank],
                 newtype, root, MPI_COMM_WORLD);
}

MPI_Type_free (&newtype);

print_array(recvbuf,reccounts[rank]);

free(array);array = NULL;
free(sendcounts);sendcounts = NULL;
free(displs);displs = NULL;
free(recvbuf);recvbuf = NULL;
MPI_Finalize();
return 0;
}

1 Answer 1

2

There is a way, but it is a bit convoluted.

The idea is you create a derived datatype with two elements at offset 0 and 8, and then resize this datatype so the upper bound is the size of one element. Then you can MPI_Scatterv() once with counts={3,3,2} and displs={0,3,6}. Note you also need to create a derived datatype on the receive side, otherwise MPI task 1 would receive {3, 11, 4, 12, 5, 13} when i guess you expect {3, 4, 5, 11, 12, 13}

Sign up to request clarification or add additional context in comments.

9 Comments

Thank you for the response! Do you by chance know where there is an example of such a process? Since I am not sure how to implement this
By size of one element do you mean the number of elements I’m sending?
if you send a MPI_INT, then you can MPI_Type_size(MPI_INT, &element_size) (works both for C and Fortran) or you can simply use sizeof(int) in C.
you can find a bunch of tutorials online. simply google ` MPI derived datatypes tutorial`. if you are still stuck, then i suggest you write a Minimal, Complete, and Verifiable example and then i will review it.
Excellent, thank you very much! I will take a look as soon as I can
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.