os.fork() different in cgi-script?

Re: os.fork() different in cgi-script?

Sorry, forgot:

this is Python 2.2.1, apache 1.3.26 on SuSE 8.1

Andreas

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:[color=blue]
> def main():
> print "Content-Type: text/plain\n\n"
> print os.getpid()
>
> childPID = os.fork()
> if childPID == 0:
> sys.exit()
> else:
> os.wait()
>
> if __name__ == "__main__":
> main()
>
> when run on a shell, gives the result:
> ---------------------------
> Content-Type: text/plain
>
>
> 20953
> ----------------------------
>
> but run as a cgi-script it gives:
> ---------------------------
> 21039
> Content-Type: text/plain
>
>
> 21039
> ---------------------------
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?[/color]

Probably because stdout is being buffered here. The pid got written in the buffer (but not yet actually printed), then the process was forked, then the buffer got flushed (written out) by each child.

One solution would be to make sure everything is flushed before you fork.

Mike

--
Mike Coleman, Scientific Programmer, +1 816 926 4419
Stowers Institute for Biomedical Research
1000 E. 50th St., Kansas City, MO 64110

Re: os.fork() different in cgi-script?

On Wed, 02 Jul 2003 09:50:03 +0000, Michael Coleman wrote:
[color=blue]
> Probably because stdout is being buffered here. The pid got written in
> the buffer (but not yet actually printed), then the process was forked,
> then the buffer got flushed (written out) by each child.
>
> One solution would be to make sure everything is flushed before you
> fork.[/color]

Thanx, that's it. Is it possible/advisable to make stdout unbuffered?

Andreas

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:
[color=blue]
> following code:[/color]
[color=blue]
> print "Content-Type: text/plain\n\n"
> print os.getpid()
>
> childPID = os.fork()
> if childPID == 0:
> sys.exit()
> else:
> os.wait()[/color]

[...]
[color=blue]
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?[/color]

The problem is buffering. When you run your script from the command
line, the standard output is a terminal and becomes line-buffered.
Thus at the time you fork, the stdout buffer has been flushed.

When you run it as a CGI script, stdout is a socket and thus not a
terminal -- so it becomes block buffered. At the time you fork, each
copy of stdout holds all of our unflushed output. When the two
processes exit they will both flush the output, resulting in it being
printed twice.

You can use sys.stdout.flus h() to flush the pending output before
forking.

--
=============== =============== =============== =============== ===
<erwin@andrease n.org> Herlev, Denmark
<URL:http://www.andreasen.o rg/> <*>
=============== =============== =============== =============== ===

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:
[color=blue]
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?[/color]

No, it isn't. The problem is that Python uses C STDIO library that has
buffered input and output. The text written in stdout is written in
buffer. Since child process created by fork() is almost identical to
parent process, it will also have a copy of stdout buffer. When
process exits, it closes stdout, which flushes the buffer. So, both
buffers are written to stdout and that causes everything to be printed
twice.

The reason why this doesn happen on command line is that buffers to
terminals are flushed after newline. You can see that by redirecting
script output to file:

$ python script.py >output
$ cat output
Content-Type: text/plain


4309
Content-Type: text/plain


4309
[color=blue]
> I googled for this but only found a similar question from 1997 and no
> answer.[/color]

Which is strange since this common problem and has nothing to do with
Python.
--
Juha Autero

Eschew obscurity!

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:[color=blue]
> def main():
> print "Content-Type: text/plain\n\n"
> print os.getpid()
>
> childPID = os.fork()
> if childPID == 0:
> sys.exit()
> else:
> os.wait()
>
> if __name__ == "__main__":
> main()
>
> when run on a shell, gives the result:
> ---------------------------
> Content-Type: text/plain
>
>
> 20953
> ----------------------------
>
> but run as a cgi-script it gives:
> ---------------------------
> 21039
> Content-Type: text/plain
>
>
> 21039
> ---------------------------
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?[/color]

Probably because stdout is being buffered here. The pid got written in the buffer (but not yet actually printed), then the process was forked, then the buffer got flushed (written out) by each child.

One solution would be to make sure everything is flushed before you fork.

Mike

--
Mike Coleman, Scientific Programmer, +1 816 926 4419
Stowers Institute for Biomedical Research
1000 E. 50th St., Kansas City, MO 64110

Re: os.fork() different in cgi-script?

[color=blue][color=green]
>> I googled for this but only found a similar question from 1997 and no
>> answer.[/color]
>
> Which is strange since this common problem and has nothing to do with
> Python.[/color]

Not so strange, because assuming a Python feature I also included "python"
in the googlequest(tm) .

Andreas

Re: os.fork() different in cgi-script?

On Wed, 02 Jul 2003 09:50:03 +0000, Michael Coleman wrote:
[color=blue]
> Probably because stdout is being buffered here. The pid got written in
> the buffer (but not yet actually printed), then the process was forked,
> then the buffer got flushed (written out) by each child.
>
> One solution would be to make sure everything is flushed before you
> fork.[/color]

Thanx, that's it. Is it possible/advisable to make stdout unbuffered?

Andreas

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:
[color=blue]
> following code:[/color]
[color=blue]
> print "Content-Type: text/plain\n\n"
> print os.getpid()
>
> childPID = os.fork()
> if childPID == 0:
> sys.exit()
> else:
> os.wait()[/color]

[...]
[color=blue]
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?[/color]

The problem is buffering. When you run your script from the command
line, the standard output is a terminal and becomes line-buffered.
Thus at the time you fork, the stdout buffer has been flushed.

When you run it as a CGI script, stdout is a socket and thus not a
terminal -- so it becomes block buffered. At the time you fork, each
copy of stdout holds all of our unflushed output. When the two
processes exit they will both flush the output, resulting in it being
printed twice.

You can use sys.stdout.flus h() to flush the pending output before
forking.

--
=============== =============== =============== =============== ===
<erwin@andrease n.org> Herlev, Denmark
<URL:http://www.andreasen.o rg/> <*>
=============== =============== =============== =============== ===

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:
[color=blue]
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?[/color]

No, it isn't. The problem is that Python uses C STDIO library that has
buffered input and output. The text written in stdout is written in
buffer. Since child process created by fork() is almost identical to
parent process, it will also have a copy of stdout buffer. When
process exits, it closes stdout, which flushes the buffer. So, both
buffers are written to stdout and that causes everything to be printed
twice.

The reason why this doesn happen on command line is that buffers to
terminals are flushed after newline. You can see that by redirecting
script output to file:

$ python script.py >output
$ cat output
Content-Type: text/plain


4309
Content-Type: text/plain


4309
[color=blue]
> I googled for this but only found a similar question from 1997 and no
> answer.[/color]

Which is strange since this common problem and has nothing to do with
Python.
--
Juha Autero

Eschew obscurity!

Re: os.fork() different in cgi-script?

[color=blue][color=green]
>> I googled for this but only found a similar question from 1997 and no
>> answer.[/color]
>
> Which is strange since this common problem and has nothing to do with
> Python.[/color]

Not so strange, because assuming a Python feature I also included "python"
in the googlequest(tm) .

Andreas

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:[color=blue]
> Thanx, that's it. Is it possible/advisable to make stdout unbuffered?[/color]

Probably. I'd be more inclined to just do an explicit flush in the
few places where they are specifically needed, though.

Mike

--
Mike Coleman, Scientific Programmer, +1 816 926 4419
Stowers Institute for Biomedical Research
1000 E. 50th St., Kansas City, MO 64110

Re: os.fork() different in cgi-script?

"Andreas Kuntzagk" <andreas.kuntza gk@mdc-berlin.de> writes:[color=blue]
> Thanx, that's it. Is it possible/advisable to make stdout unbuffered?[/color]

Probably. I'd be more inclined to just do an explicit flush in the
few places where they are specifically needed, though.

Mike

--
Mike Coleman, Scientific Programmer, +1 816 926 4419
Stowers Institute for Biomedical Research
1000 E. 50th St., Kansas City, MO 64110