Thread: Re: [Php-java-bridge-users] java.so module purpose?

Hi again,

So I'm trying to reproduce this bug with java bridge on FreeBSD not
cooperating over a localhost TCP socket (mentioned in README on line
621-646.)  I can't seem to reproduce it.  Originally I had been compiling
java-bridge so that we could run our daemon as follows:
/usr/local/lib/php/20060613/RunJavaBridge /usr/local/bin/java
-Djava.library.path=/usr/local/lib/php/20060613
-Djava.class.path=/usr/local/lib/php/20060613/JavaBridge.jar
-Djava.awt.headless=true -Dphp.java.bride.base=/usr/local/lib/php/20060613
php.java.bridge.Standalone LOCAL:/tmp/php-java-bridge.sock 1
/var/log/java-bridge.log
...so that we can listen on a Unix Domain Socket.  This produces a java
bridge which works fine.

I've now tried just downloading the binary version of java-bridge,
extracting, and simply running:
java -jar JavaBridge.jar SERVLET_LOCAL:8080

Then running a handful of test scripts over localhost against it.  This
should reproduce the FreeBSD bug with repeated requests going slowly due to
nagle's algorithm, correct?  In particular I've run the sample speed check
code mentioned in the FAQ as below:

<?php
$buf=new java("java.lang.StringBuffer");

$i=0;
while($i<400000) {
  $i=$i+1;
  $buf->append($i);
}

print $buf->length() . "\n";
?>

It runs in ~14-16 seconds depending on which of my test hosts I use, so
clearly is not being slowed down.  Some are running FreeBSD 6.2 w/
diablo-jdk 1.5; some are running FreeBSD 7.0 w/ Sun's 1.5 jdk.

My questions is: does anyone know of a specific test case I can run to
reproduce the supposed bug?  FreeBSD version, PJB version, actual code,
etc.

I'm not convinced that this bug actually exists, but I'm happy to be told
how I'm doing this wrong :)  Just trying to sort this out so I can
confidently deploy the latest version of PJB.

Thanks,
Patrick








|---------+------------------------------------------->
|         |           php...@li...|
|         |           ceforge.net                     |
|         |                                           |
|         |           10/20/2008 11:05 AM             |
|         |           Please respond to               |
|         |           php-java-bridge-users           |
|         |                                           |
|---------+------------------------------------------->
  >------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                              |
  |       To:       php...@li...                                                                  |
  |       cc:                                                                                                                    |
  |       Subject:  Re: [Php-java-bridge-users] java.so module purpose?                                                          |
  >------------------------------------------------------------------------------------------------------------------------------|




Hi,


> pure-PHP implementation is the way to go, and the java.so
> module no longer  provides any [...] functionality

Youre right. java.so and php_java.dll are completely obsolete.


> we launch one from a rc script.)  Jost you have mentioned
> that the module provides a workaround for BSD systems

Java doesnt support Unix Domain Sockets, if you want to use the
LOCAL:/tmp/socket channel, you need the natcJavaBridge.so JNI library.
Please see the FreeBSD section from the README.


Regards,
Jost Boekemeier

__________________________________________________
Do You Yahoo!?
Sie sind Spam leid? Yahoo! Mail verfügt über einen herausragenden Schutz
gegen Massenmails.
http://mail.yahoo.com

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
php-java-bridge-users mailing list
php...@li...
https://lists.sourceforge.net/lists/listinfo/php-java-bridge-users





http://www.mintel.com
providing insight + impact

Chicago Office:
Mintel International Group Ltd (Mintel)
351 Hubbard Street, Floor 8
Chicago, IL 60610
USA

Tel: 312 932 0400
Fax: 312 932 0469

London Office:
Mintel International Group Ltd (Mintel)
18-19 Long Lane
London
EC1A 9PL
UK

Tel: 020 7606 4533
Fax: 020 7606 5932


Notice
********
This email may contain information that is privileged,
confidential or otherwise protected from disclosure. It
must not be used by, or its contents copied or disclosed
to, persons other than the addressee. If you have received
this email in error please notify the sender immediately
and delete the email. Any views or opinions expressed in
this message are solely those of the author, and do not
necessarily reflect those of Mintel.

No Mintel staff are authorised to make purchases using
email or over the internet, and any contracts so performed
are invalid.

Warning
**********
It is the responsibility of the recipient to ensure that
the onward transmission, opening or use of this message
and any attachments will not adversely affect their systems
or data. Please carry out such virus and other checks, as
you consider appropriate.

[Sorry to  hijack this thread, just replying from a proper email account 
as my @mintel.com account is using Lotus Notes]

Alright, I've made some good progress here in understanding this bug:

* Grabbed & compiled the test code as on the FreeBSD ML.  Bug 
reproduced.  Also produced a client fix within the java code using 
Socket::setTcpNoDelay(true) to bypass Nagle's Algorithm and it proves 
that this is the socket option that is not being automatically set.
* Split out the test code into two pieces: a java server which listens 
for the packet [TestServ.class], java code to act as a client 
[TestClient.class] and PHP code which does the client end.  I ended up 
with two PHP implementations in an attempt to reproduce the bug:
One) which uses fsockopen(...) which is the same socket method that 
PJB's PHP library uses.  I cannot get this to trigger Nagle's 
Algorithm.  Inspection using tcpdump shows that the socket created with 
fsockopen is buffered, so two calls to fwrite (one with a single byte, 
one with 50 bytes) end up in a single packet as it waits to send data 
until the socket is closed.  Code attached [fsockopen-client.php]
Two) which uses the socket_* functions in the PHP Socket extension and a 
hacky call to socket_set_option to shrink the send buffer size to 1 
before the first packet ("@") and then expand the buffer size to 51 
bytes for the second packet.  tcpdump inspection shows that it doesn't 
produce the exact same size packets as the java code, but it definitely 
triggers Nagle's Algorithm and the slowdown is roughly the same as the 
java test client.  Code attached [sockets-client.php]

 From there, I sought out to set TCP_NODELAY in PHP:
* Took a look around the PHP source code in ext/sockets/sockets.c:500+ 
where the socket options are set.  The PHP constants are [naturally] the 
same value as whatever the local system has defined for the kernel's 
integer value of the constant.  TCP_NODELAY on FreeBSD is 0x01.  I added 
a function call to my sockets-client.php before sending a packet like:
socket_set_option($sock, SOL_TCP, 0x01, 1); // disable Nagle's Algorithm
...and it works like a charm.  Performs like it should.
* I've patched sockets.c in PHP and recompiled, and now have TCP_NODELAY 
available in my sockets extension.  Am submitting this patch to PHP for 
inclusion.  Can now do:
socket_set_option($sock, SOL_TCP, TCP_NODELAY, 1); // disable Nagle's 
Algorithm
...and again, works like a charm.

Importantly, from the PJB angle: I cannot reproduce the bug with Nagle's 
Algorithm using the fsockopen socket method (even using fflush to try 
and force it to send a packet.)  Jost, do you have some PHP code that 
can reproduce this bug?  I'm really curious.  I'm tempted to say that 
the buffering used by fsockopen will dodge this bug, but I don't know 
for sure.  I'm also wondering if the buffering used by fsockopen might 
actually be incurring an unnecessary (albeit minor) performance penalty 
on PJB and if the sockets functions might be a wiser choice.  I'm no 
expert here.

The patch I've written to make TCP_NODELAY available in PHP will fix the 
bug if we're ever communicating using the sockets extension, but we're 
using fsockopen.  It's still worth a submit as it might be useful.  I 
don't know anything about the internals of fsockopen (I'll give the code 
a read.)  Furthermore there's no method that I can see to set socket 
options on a stream opened by fsockopen.

Sorry if this is a lot to digest.  Let me know what your thoughts are 
and most importantly, if you have a way to reproduce this bug using 
fsockopen in PHP (and thus something that will actually affect PJB.)

Thanks for all your help!

Patrick





-- 
Patrick "Trick" van Staveren
http://trick.vanstaveren.us

Sorry, seems my attachments were stripped.  Files available at: 
http://vanstaveren.us/~trick/fbsd-nagle-localhost-errorcase/

Cheers,
Patrick


php...@li... wrote:
> [Sorry to  hijack this thread, just replying from a proper email 
> account as my @mintel.com account is using Lotus Notes]
>
> Alright, I've made some good progress here in understanding this bug:
>
> * Grabbed & compiled the test code as on the FreeBSD ML.  Bug 
> reproduced.  Also produced a client fix within the java code using 
> Socket::setTcpNoDelay(true) to bypass Nagle's Algorithm and it proves 
> that this is the socket option that is not being automatically set.
> * Split out the test code into two pieces: a java server which listens 
> for the packet [TestServ.class], java code to act as a client 
> [TestClient.class] and PHP code which does the client end.  I ended up 
> with two PHP implementations in an attempt to reproduce the bug:
> One) which uses fsockopen(...) which is the same socket method that 
> PJB's PHP library uses.  I cannot get this to trigger Nagle's 
> Algorithm.  Inspection using tcpdump shows that the socket created 
> with fsockopen is buffered, so two calls to fwrite (one with a single 
> byte, one with 50 bytes) end up in a single packet as it waits to send 
> data until the socket is closed.  Code attached [fsockopen-client.php]
> Two) which uses the socket_* functions in the PHP Socket extension and 
> a hacky call to socket_set_option to shrink the send buffer size to 1 
> before the first packet ("@") and then expand the buffer size to 51 
> bytes for the second packet.  tcpdump inspection shows that it doesn't 
> produce the exact same size packets as the java code, but it 
> definitely triggers Nagle's Algorithm and the slowdown is roughly the 
> same as the java test client.  Code attached [sockets-client.php]
>
> From there, I sought out to set TCP_NODELAY in PHP:
> * Took a look around the PHP source code in ext/sockets/sockets.c:500+ 
> where the socket options are set.  The PHP constants are [naturally] 
> the same value as whatever the local system has defined for the 
> kernel's integer value of the constant.  TCP_NODELAY on FreeBSD is 
> 0x01.  I added a function call to my sockets-client.php before sending 
> a packet like:
> socket_set_option($sock, SOL_TCP, 0x01, 1); // disable Nagle's Algorithm
> ...and it works like a charm.  Performs like it should.
> * I've patched sockets.c in PHP and recompiled, and now have 
> TCP_NODELAY available in my sockets extension.  Am submitting this 
> patch to PHP for inclusion.  Can now do:
> socket_set_option($sock, SOL_TCP, TCP_NODELAY, 1); // disable Nagle's 
> Algorithm
> ...and again, works like a charm.
>
> Importantly, from the PJB angle: I cannot reproduce the bug with 
> Nagle's Algorithm using the fsockopen socket method (even using fflush 
> to try and force it to send a packet.)  Jost, do you have some PHP 
> code that can reproduce this bug?  I'm really curious.  I'm tempted to 
> say that the buffering used by fsockopen will dodge this bug, but I 
> don't know for sure.  I'm also wondering if the buffering used by 
> fsockopen might actually be incurring an unnecessary (albeit minor) 
> performance penalty on PJB and if the sockets functions might be a 
> wiser choice.  I'm no expert here.
>
> The patch I've written to make TCP_NODELAY available in PHP will fix 
> the bug if we're ever communicating using the sockets extension, but 
> we're using fsockopen.  It's still worth a submit as it might be 
> useful.  I don't know anything about the internals of fsockopen (I'll 
> give the code a read.)  Furthermore there's no method that I can see 
> to set socket options on a stream opened by fsockopen.
>
> Sorry if this is a lot to digest.  Let me know what your thoughts are 
> and most importantly, if you have a way to reproduce this bug using 
> fsockopen in PHP (and thus something that will actually affect PJB.)
>
> Thanks for all your help!
>
> Patrick
>
>
>
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ------------------------------------------------------------------------
>
> _______________________________________________
> php-java-bridge-users mailing list
> php...@li...
> https://lists.sourceforge.net/lists/listinfo/php-java-bridge-users
>   

-- 
Patrick "Trick" van Staveren
http://trick.vanstaveren.us

Hi,

it is true that PHP buffers data internally. But as soon as read() is called on the same stream, the outgoing data is flushed. The problem is really the "ack delay" caused by BSD's socket implementation.

I suggest to discuss this on the BSD or PHP mailing list, as this affects all PHP applications running on BSD, not only the PHP/Java Bridge.


Regards,
Jost Boekemeier



      

I've learned more. My examples were wrong. Updated examples at 
http://vanstaveren.us/~trick/fbsd-nagle-localhost-errorcase/

My bug: I was not fread() / socket_read()'ing from the opened stream 
after sending packets from the client. So communication was one-way. My 
hacks on the send buffer size had fooled TCP into doing an extra round 
trip to the server which coincidentally caused the delay as well. 
Updated examples now actually send & recieve the same packets as your 
original java test case and the bug reproduces exactly in both the 
fsockopen and socket_ code. socket_set_option with TCP_NODELAY works, 
for the sockets example only of course.

My speculations about buffers and fflush() not working were false, I was 
clearly mistaken...

This means a few things:
1) I'm now a lot more confident that socket_set_option is a viable fix, 
but I don't know if it's best to move PJB to use this API just for this 
reason. What do you think? My patch was accepted; the constant 
TCP_NODELAY is available as of PHP 5.2.7-RC2.
2) After quite a bit of research, the streams library 
(fsockopen/fwrite/fread) apparently does not expose a method for me to 
set socket options. I'll raise this on the PHP ML to make sure I've not 
missed something obvious.

Might be viable to write a PHP function stream_socket_{get,set}_option 
which does the same thing as socket_{get,set}_option if that's the best 
route.

Cheers,
Patrick


php...@li... wrote:
> Hi,
>
> it is true that PHP buffers data internally. But as soon as read() is called on the same stream, the outgoing data is flushed. The problem is really the "ack delay" caused by BSD's socket implementation.
>
> I suggest to discuss this on the BSD or PHP mailing list, as this affects all PHP applications running on BSD, not only the PHP/Java Bridge.
>
>
> Regards,
> Jost Boekemeier
>
>
>
>       
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> php-java-bridge-users mailing list
> php...@li...
> https://lists.sourceforge.net/lists/listinfo/php-java-bridge-users
>   

-- 
Patrick "Trick" van Staveren
http://trick.vanstaveren.us

Hi,

if the socket_* are available on all operating systems and in all PHP versions >= 5.1.4, I'd prefer this API over fwrite/fread. 

However, accoding to my tests on FreeBSD 4, Unix Domain Sockets communication is faster than TCP socket communication, even with ndelay switched off.


Regards,
Jost Boekemeier


--- php...@li... <php...@li...> schrieb am Fr, 24.10.2008:

> Von: php...@li... <php...@li...>
> Betreff: Re: [Php-java-bridge-users] java.so module purpose?
> An: php...@li...
> Datum: Freitag, 24. Oktober 2008, 8:19
> I've learned more. My examples were wrong. Updated
> examples at 
> http://vanstaveren.us/~trick/fbsd-nagle-localhost-errorcase/
> 
> My bug: I was not fread() / socket_read()'ing from the
> opened stream 
> after sending packets from the client. So communication was
> one-way. My 
> hacks on the send buffer size had fooled TCP into doing an
> extra round 
> trip to the server which coincidentally caused the delay as
> well. 
> Updated examples now actually send & recieve the same
> packets as your 
> original java test case and the bug reproduces exactly in
> both the 
> fsockopen and socket_ code. socket_set_option with
> TCP_NODELAY works, 
> for the sockets example only of course.
> 
> My speculations about buffers and fflush() not working were
> false, I was 
> clearly mistaken...
> 
> This means a few things:
> 1) I'm now a lot more confident that socket_set_option
> is a viable fix, 
> but I don't know if it's best to move PJB to use
> this API just for this 
> reason. What do you think? My patch was accepted; the
> constant 
> TCP_NODELAY is available as of PHP 5.2.7-RC2.
> 2) After quite a bit of research, the streams library 
> (fsockopen/fwrite/fread) apparently does not expose a
> method for me to 
> set socket options. I'll raise this on the PHP ML to
> make sure I've not 
> missed something obvious.
> 
> Might be viable to write a PHP function
> stream_socket_{get,set}_option 
> which does the same thing as socket_{get,set}_option if
> that's the best 
> route.
> 
> Cheers,
> Patrick
> 
> 
> php...@li... wrote:
> > Hi,
> >
> > it is true that PHP buffers data internally. But as
> soon as read() is called on the same stream, the outgoing
> data is flushed. The problem is really the "ack
> delay" caused by BSD's socket implementation.
> >
> > I suggest to discuss this on the BSD or PHP mailing
> list, as this affects all PHP applications running on BSD,
> not only the PHP/Java Bridge.
> >
> >
> > Regards,
> > Jost Boekemeier
> >
> >
> >
> >       
> >
> >
> -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move
> Developer's challenge
> > Build the coolest Linux based applications with Moblin
> SDK & win great prizes
> > Grand prize is a trip for two to an Open Source event
> anywhere in the world
> >
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > php-java-bridge-users mailing list
> > php...@li...
> >
> https://lists.sourceforge.net/lists/listinfo/php-java-bridge-users
> >   
> 
> -- 
> Patrick "Trick" van Staveren
> http://trick.vanstaveren.us
> 
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move
> Developer's challenge
> Build the coolest Linux based applications with Moblin SDK
> & win great prizes
> Grand prize is a trip for two to an Open Source event
> anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> php-java-bridge-users mailing list
> php...@li...
> https://lists.sourceforge.net/lists/listinfo/php-java-bridge-users


      

Hi,

> even using fflush to try
> and force it to send a packet.)  Jost, do you have some PHP code that
> can reproduce this bug?  


Are you sure. We use fflush() to send out the header immediately so that the back-end can set up the ContextRunner.

I am sure that this worked.


Regards,
Jost Boekemeier



      

Hi,

FreeBSD uses the "Nagle Algorithm" for local TCP sockets. Since PHP has no API to set NDELAY, local TCP socket communication is useless on BSD. 

A test program which should reproduce this bug is here:

=> http://lists.freebsd.org/pipermail/freebsd-bugs/2006-April/018144.html


If you want to use a local "Unix Domain Socket" channel, you must define a procedure which returns the socket string (e.g.: /tmp/socket). If you return a string, a Unix Domain Socket is used, if you return a number, a local TCP socket is used. See the last 3 lines of the Mono.inc for an example.


Regards,
Jost Boekemeier