mpi_gather for struct with dynamic array

Question

I have a struct:

typedef struct 
{ 
double distance;
int* path;
} tour;

Then I trying to gather results from all processes:

   MPI_Gather(&best, sizeof(tour), MPI_BEST, all_best, sizeof(tour)*proc_count, MPI_BEST, 0, MPI_COMM_WORLD);

After gather my root see that all_best containts only 1 normal element and trash in others. Type of all_best is tour*.

Initialisation of MPI_BEST:

 void ACO_Build_best(tour *tour,int city_count, MPI_Datatype *mpi_type /*out*/)
 {
int block_lengths[2];
MPI_Aint displacements[2];
MPI_Datatype typelist[2];
MPI_Aint start_address;
MPI_Aint address;

block_lengths[0] = 1;
block_lengths[1] = city_count;

typelist[0] = MPI_DOUBLE;
typelist[1] = MPI_INT;

MPI_Address(&(tour->distance), &displacements[0]);
MPI_Address(&(tour->path), &displacements[1]);

displacements[1] = displacements[1] - displacements[0];
displacements[0] = 0;

MPI_Type_struct(2, block_lengths, displacements, typelist, mpi_type);
MPI_Type_commit(mpi_type);
}

Any ideas are welcome.

Hristo Iliev · Accepted Answer · 2012-12-07 18:49:33Z

3

Apart from passing incorrect lengths to MPI_Gather, MPI actually does not follow pointers to pointers. With such a structured type you would be sending the value of distance and the value of the path pointer (essentially an address which makes no sense when sent to other processes). If one supposes that distance essentially gives the number of elements in path, then you can kind of achieve your goal with a combination of MPI_Gather and MPI_Gatherv:

First, gather the lengths:

int counts[proc_count];

MPI_Gather(&best->distance, 1, MPI_INT, counts, 1, MPI_INT, 0, MPI_COMM_WORLD);

Now that counts is populated with the correct lengths, you can continue and use MPI_Gatherv to receive all paths:

int disps[proc_count];

disps[0] = 0;
for (int i = 1; i < proc_count; i++)
   disps[i] = disps[i-1] + counts[i-1];
// Allocate space for the concatenation of all paths
int *all_paths = malloc((disps[proc_count-1] + counts[proc_count-1])*sizeof(int));
MPI_Gatherv(best->path, best->distance, MPI_INT,
            all_paths, counts, disps, MPI_INT, 0, MPI_COMM_WORLD);

Now you have the concatenation of all paths in all_paths. You can examine or extract an individual path by taking counts[i] elements starting at position disps[i] in all_paths. Or you can even build an array of tour structures and make them use the already allocated and populated path storage:

tour *all_best = malloc(proc_count*sizeof(tour));
for (int i = 0; i < proc_count; i++)
{
   all_best[i].distance = counts[i];
   all_best[i].path = &all_paths[disps[i]];
}

Or you can duplicate the segments instead:

for (int i = 0; i < proc_count; i++)
{
   all_best[i].distance = counts[i];
   all_best[i].path = malloc(counts[i]*sizeof(int));
   memcpy(all_best[i].path, &all_paths[disps[i]], counts[i]*sizeof(int));
}
// all_paths is not needed any more and can be safely free()-ed

Edit: Because I've overlooked the definition of the tour structure, the above code actually works with:

struct
{
   int distance;
   int *path;
}

where distance holds the number of significant elements in path. This is different from your case, but without some information on how tour.path is being allocated (and sized), it's hard to give a specific solution.

edited Dec 7, 2012 at 18:49

answered Dec 7, 2012 at 13:12

Hristo Iliev

74.9k12 gold badges148 silver badges197 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

T_T Over a year ago

Are you sure that counts[0] and all_paths[0] will be from the same element?

Hristo Iliev Over a year ago

@T_T, MPI_Gather[v] orders the received chunks by the ranks of the processes, so counts[0] will always be the number of elements in rank 0 and all_paths[0] will be the first item in the path from rank 0. But now I see that distance is not integer, so things become more complex. Could you show how tour.path is being allocated?

T_T Over a year ago

I've changed type of distance to int. Your answer helped. Thank you.

Collectives™ on Stack Overflow

mpi_gather for struct with dynamic array

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related