Apache http client -> local address bind -> BindException after some time

Question

OS: Linux ( Debian )
Java: Java(TM) SE Runtime Environment (build 1.7.0_05-b06)
Java: Java HotSpot(TM) 64-Bit Server VM (build 23.1-b03, mixed mode)
Apache http client: v4.2.3 ( latest )

I want to create one PoolingConnectionManager with several clients, each with its unique local address (ConnRoutePNames.LOCAL_ADDRESS).
Those clients will be used by several workers ( each worker selects random client and executes request ).

The problem is: When i set local address for clients, after some time (e.g. 1 min) i always get java.net.BindException with message "Address already in use".

Question: Is it bug?

java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_05]
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376) ~[na:1.7.0_05]
at java.net.Socket.bind(Socket.java:627) ~[na:1.7.0_05]
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) ~[httpclient-4.2.3.jar:4.2.3]
at controllers.Test$Worker.run(Test.java:67) ~[test_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_05]

Code ( simplified ):

public static void main(String[] args) throws UnknownHostException
{
    ClientConnectionManager connManager = buildConnectionManager(30);

    List<HttpClient> clients = new ArrayList<>();
    for(int i=0; i<8; ++i)
    {
        HttpClient client = buildClient(connManager, InetAddress.getByName("111.111.111." + (50 + i));
        clients.add(client);
    }

    for(int i=0; i<30; ++i)
    {
        new Thread(new Worker(clients)).start();
    }
}

public static ClientConnectionManager buildConnectionManager(Integer parallelism)
{
    PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
    connectionManager.setMaxTotal(parallelism);
    connectionManager.setDefaultMaxPerRoute(parallelism);

    return connectionManager;
}

public static HttpClient buildClient(ClientConnectionManager connectionManager, InetAddress localAddress)
{
    DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
    HttpParams params = httpClient.getParams();
    params.setParameter(ConnRoutePNames.LOCAL_ADDRESS, localAddress);

    return httpClient;
}

private static class Worker implements Runnable
{
    private List<HttpClient> clients = null;

    public Worker(List<HttpClient> clients)
    {
        this.clients = clients;
    }

    public void run()
    {

        do
        {
            HttpGet httpGet = new HttpGet("http://google.com/robots.txt");
            HttpClient client = this.clients.get(new Random().nextInt(this.clients.size()));

            try
            {
                HttpResponse httpResponse = client.execute(httpGet);
                EntityUtils.consume(httpResponse.getEntity());

                logger.debug("Success request");
            }
            catch(IOException e)
            {
                httpGet.abort();
                logger.info("IO error", e);
            }
        }
        while(true);
    }
}

Try a httpResponse.releaseConnection() after consuming the response. Ideally in a finally block as shown here — Pyranja
– Pyranja, Commented Jan 20, 2013 at 1:20
@Pyranja, no, it does not work. Anyway, it would not be good solve of my problem :( — Oleg Golovanov
– Oleg Golovanov, Commented Jan 20, 2013 at 10:03

Francois-Xavier Bonnet · Accepted Answer · 2013-01-21 10:04:05Z

3

This issue may have to do with your TCP stack. I don't think this is a bug in HttpClient. You may find the solution here: http://planet.jboss.org/post/concurrent_high_throughput_performance_testing_with_jmeter

answered Jan 21, 2013 at 10:04

Francois-Xavier Bonnet

311 bronze badge

Sign up to request clarification or add additional context in comments.

Comments

ok2c · Accepted Answer · 2013-01-20 19:40:39Z

1

The only theory I can think of is that the TCP/IP stack runs out of ports and starts assigning the same port numbers to new connections while old connections have not been fully cleaned up. I see no evidence of this problem being a bug in HttpClient.

answered Jan 20, 2013 at 19:40

ok2c

27.9k5 gold badges66 silver badges74 bronze badges

Comments

Oleg Golovanov · Accepted Answer · 2013-01-21 15:49:58Z

0

Thanks for your answers!
Yes, the problem was with tcp/ip stack, but i was unable to get why :)

So, the reason explained below.

I created connection manager ( e.g. size=10 ), several clients ( each with its unique local address ), and several workers.
Each worker takes random client and executes the request.
So, at one time its possible that one client can be executed by up to 10 workers in parallel.
At another time another client can be executed by up to 10 workers in parallel.
Thats why, all 10 connections in manager can not be keep-alived, they are closing/creating all the time.

Sorry for bothering you!

answered Jan 21, 2013 at 15:49

Oleg Golovanov

9241 gold badge14 silver badges25 bronze badges

Collectives™ on Stack Overflow

Apache http client -> local address bind -> BindException after some time

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related