0

I have a matlab script (call that MyProcessing.m) that does some computations based on some random number. Right now I have a fixed seed to get the same sequence of random numbers. I would like to run this script multiple times in parallel to utilize the multiple cores that I have available on the system. I want each of the new "processes" to start with a different (but fixed for the moment) seed. Bellow is the processing file as it is right now.

There is a for loop inside the script but I cannot use parfor because each iteration depends on the previous one.

MyProcessing.m

rng(1);
A = rand(5,5);
x =[];
y = []

% for loop
%   that updates x and y when necessary
% end for

figure(1);
scatter(x, y);
savefig(filename);

I have access to the Parallel Computing Toolbox in MATLAB but I cannot think on what I should do. I believe that I have to write another script to call the processing script with a different random seed but I want also the different processes to be run in parallel so that I can run many experiments.

EDIT:

I want something like

for i = 1:numberOfParallelProcesses
  startANewRunOfTheScript();
end

where the for loop start the process and then it does not wait but it continues to start the next one.

Improve this question
3
  • 1
    parfeval? Just prefer functions before scripts. To make it simple, be sure to avoid shared data. I am not sure if paralellization can be done since you have internal dependencies, but since you sorted away this information as irrelevant I cannot say. Commented Aug 31, 2016 at 10:07
  • Each for loop iteration depends on the previous iteration. But the whole algorithm is independent of another script running the same algorithm on the same data. What you suggested might work. I am going to find out how to use parfeval. Commented Aug 31, 2016 at 10:27
  • 1
    Perhaps this can serve as inspiration: stackoverflow.com/questions/18204663/… Commented Aug 31, 2016 at 11:19

2 Answers 2

Reset to default
2

You could use batch to achieve this. You can do:

for idx = 1:n
  job(idx) = batch('MyProcessing');
end

You can then fetch the results later using the load method of each element of job:

for idx = 1:n
  wait(job(idx)); % wait for results to arrive
  out{idx} = load(job(idx));
end

There's more about batch processing in the doc.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I have used this method of running multiple scripts at the same time, but the problem is that it is kind of slow. I understand that staring the different processing will take some time, but also their execution seems slower. For example I have many available cores and I started 12 processes. The time to execute them in parallel seems comparable to execute them sequentially. Is it possible that I am doing something wrong?
Parallel computing toolbox workers run in "single computational thread" mode (by default - you can change this by calling maxNumCompThreads inside your script), so they can run more slowly if your script is able to take advantage of MATLAB's intrinsic multi-threading. If your script is already taking full advantage of all your CPU resources via multi-threading, then explicit parallelism using batch or similar can only make things slower.
1

For Embarrassingly parallel problems in matlab, by far the easiest solution is to start multiple instances of matlab and run the script on each instance (obviously itterating forwards on the random seed when you begin each instance). I used this simple technique to great effect on a 40 core server in the past. The only limit is your system memory. Use a little trial and error to find the number of instances required to get maximum throughput.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.