Parallel python - how to use output data?

Question

Hey im new to Parallel python, and was writing a simple code that adds numbers together in parallel, the code works fine to add the numbers, but im wondering how you go about using the output from the jobs after its done, if possible? my code is as follows:

import sys, time
import pp
import numpy
x = numpy.arange(-20.0,20.0,0.5)
grav = []
gravity = numpy.zeros([4,1])
def function(raw_input,x,grav):
    f = 0
    for i in numpy.arange(len(x)):
        f+=1
    a=raw_input[0]
    b=raw_input[1]
    c=raw_input[2]
    d=raw_input[3]
    grav.append((a+b+c+d)+f)
    return grav
for i in numpy.arange(2):
    # tuple of all parallel python servers to connect with
    ppservers = ()
    #ppservers = ("10.0.0.1",)

    if len(sys.argv) > 1:
        ncpus = int(sys.argv[1])
        # Creates jobserver with ncpus workers
        job_server = pp.Server(ncpus, ppservers=ppservers)
    else:
        # Creates jobserver with automatically detected number of workers
        job_server = pp.Server(ppservers=ppservers)

    print "Starting pp with", job_server.get_ncpus(), "workers"
    start_time = time.time()

    # The following submits 4 jobs and then retrieves the results
    puts = ([1,2,3,4], [3,2,3,4],[4,2,3,6],[2,3,4,5])

    jobs = [(raw_input, job_server.submit(function,(raw_input,x,grav), (), ("numpy",))) for raw_input in puts]
    for raw_input, job in jobs:
        print "Sum of numbers", raw_input, "is", job()
    print grav
    print "Time elapsed: ", time.time() - start_time, "s"
    job_server.print_stats()
    print gravity
    #gravity[i] = grav

this prints out the 4 results, which are 90,92,95,94, and gives the stats etc. so my question is how can i then use the 4 result numbers, i want them to be dumped into the array called gravity that's there but i cant figure out how. Thanks

I'm sorry but I think I don't understand your question fully. Do you just want to save the numbers in the array after you print them? Or am I missing something? — Eduardo Ivanec
– Eduardo Ivanec, Commented Feb 16, 2012 at 15:35
i want to be able to use the numbers in the 'gravity' array, for further calculations in the loop but i cant get the values out of the parallel job and into the array. — user991926
– user991926, Commented Feb 16, 2012 at 17:36

enderskill · Accepted Answer · 2012-02-16 18:00:54Z

You need to store the result of the job() function. Try the following code:

import sys, time
import pp
import numpy
x = numpy.arange(-20.0,20.0,0.5)
grav = []
gravity = numpy.zeros([4,1])
def function(raw_input,x,grav):
    f = 0
    for i in numpy.arange(len(x)):
        f+=1
    a=raw_input[0]
    b=raw_input[1]
    c=raw_input[2]
    d=raw_input[3]
    grav.append((a+b+c+d)+f)
    return grav

jobsList = []

for i in numpy.arange(2):
    # tuple of all parallel python servers to connect with
    ppservers = ()
    #ppservers = ("10.0.0.1",)

    if len(sys.argv) > 1:
        ncpus = int(sys.argv[1])
        # Creates jobserver with ncpus workers
        job_server = pp.Server(ncpus, ppservers=ppservers)
    else:
        # Creates jobserver with automatically detected number of workers
        job_server = pp.Server(ppservers=ppservers)

    print "Starting pp with", job_server.get_ncpus(), "workers"
    start_time = time.time()

    # The following submits 4 jobs and then retrieves the results
    puts = ([1,2,3,4], [3,2,3,4],[4,2,3,6],[2,3,4,5])

    jobs = [(raw_input, job_server.submit(function,(raw_input,x,grav), (), ("numpy",))) for raw_input in puts]
    for raw_input, job in jobs:
        r = job()
        jobsList.append(r)
        print "Sum of numbers", raw_input, "is", r
    print grav
    print "Time elapsed: ", time.time() - start_time, "s"
    job_server.print_stats()
    print gravity
    for job in jobsList:
        print job

Collectives™ on Stack Overflow

Parallel python - how to use output data?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related