Using the python multiprocessing module for IO with pygame on Mac OS 10.7

Question

I use pygame for running experiments in cognitive science, and often I have heavy I/O demands so I like to fork off these tasks to separate processes (when using a multi-core machine) to improve performance of my code. However, I encountered a scenario where some code works on my colleague's linux machine (Ubuntu LTS), but not on my mac. Below is code representing a minimal reproducible example. My mac is a 2011 Macbook Air running 10.7.2 and using the default python 2.7.1. I tried both pygame as installed via pre-built binary, and I also then tried after installing both SDL and pygame from source.

import pygame
import multiprocessing
pygame.init()

def f():
    while True:
        pygame.event.pump() #if this is replaced by pass, this code works

p = multiprocessing.Process(target=f)
p.start()

while True:
    pass

As noted in the code, it seems that the culprit is putting pygame.event.pump() in a separate process. When I run this on my mac, I first get the following printed repeatedly in terminal:

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

Then I get a crash report as copied to this gist.

Any suggestions for how to fix this?

Wow, you have a minimal example and a clear, concise described phenomenon that sounds a lot like a bug. Why not report it? Seems like you'd have better luck there than SO. — Brian Cain
– Brian Cain, Commented Nov 17, 2011 at 3:51
You submitted it as issue 98. It's a good idea to maintain a chain anyone can follow. — K3---rnc
– K3---rnc, Commented May 11, 2016 at 11:45

Nicolae Dascalu · Accepted Answer · 2011-11-17 23:19:21Z

4

+300

Maybe you should initialize the pygame (which initialize SDL-> OpenGL) in each forked (child) process like in sample:

import multiprocessing

def f():
  import pygame
  pygame.init()

  while True:
    pygame.event.pump()

if __module__ == "__main__"
  p = multiprocessing.Process(target=f)
  p.start()

  import pygame
  pygame.init()

  while True:
    pygame.event.pump()

edited Nov 17, 2011 at 23:19

answered Nov 17, 2011 at 22:59

Nicolae Dascalu

3,5452 gold badges21 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Mike Lawrence Over a year ago

That prevents a crash, but if I attempt to also use pygame in the main process (e.g. for drawing to the screen), I get the same behaviour as reported earlier.

Nicolae Dascalu Over a year ago

@Mike You shuldd initialize pygame in main process only after forking children. The ideea is to avoid pygame initialization before forking! I updated the example to reflect this.

Mike Lawrence Over a year ago

Almost there! While the code you sent works, the idea breaks down when you actually try to collect input in the child while presenting anything fullscreen in the parent because as soon as the parent creates a fullscreen window it steals focus from the child, leaving the child unable to detect input. Any thoughts on how to solve that?

Nicolae Dascalu Over a year ago

@Mike Delay your full screen mode, after all children are fully intialized from the pygame perspective, using a multiprocessing sinchronization objects. If you have no experince with process synchrnization objects just put a sleep(few seconds) only in main process, to see if you are on good track, google it or later I can help you with synchronization, also.

Nicolae Dascalu Over a year ago

@Mike Seams that you encounter other problem that original one ... Can you add more detail related to the second issue?

First Timer Poster · Accepted Answer · 2012-12-01 12:36:34Z

2

Try this link:

http://www.slideshare.net/dabeaz/an-introduction-to-python-concurrency#btnPrevious

It may help. The problem is that you are creating a process that never stops. This should be declared as a daemon:

p = multiprocessing.Process(target=f)
p.daemon = True
p.start()

Not sure if this will solve the problem, I'm just learning about the multiprocessing module as I'm posting this.

answered Dec 1, 2012 at 12:36

First Timer Poster

212 bronze badges

Comments

elmysterio · Accepted Answer · 2011-11-16 23:44:05Z

0

Have you tried using threads instead of processes? I've had issues before using the python multiprocessing module in OS X. http://docs.python.org/library/threading.html

answered Nov 16, 2011 at 23:44

elmysterio

2021 silver badge3 bronze badges

1 Comment

Mike Lawrence Over a year ago

My understanding is that threads in python are still subject to the global interpreter lock, diminishing their benefit, and furthermore that SDL (on which pygame relies) has threading issues.

Collectives™ on Stack Overflow

Using the python multiprocessing module for IO with pygame on Mac OS 10.7

3 Answers 3

5 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related