Using JDK16+ unix domain socket support to connect to clamd randomly fails for the n-th connection (n > 1) when using non-blocking IO

I try to connect to the clamd service (antivirus) on an Ubuntu Linux System. clamd on Ubuntu Linux can be contacted using a unix domain socket, which is by default registered under the path /var/run/clamav/clamd.ctl.

Accessing that socket works from java using the UnixDomainSocketAdress support added in JDK16+, when the corresponding SocketChannel is run in blocking mode. For a basic communication test the clamd PING command is used, which causes the server to reply with a PONG. In blocking mode 100 connections where tested sequentially and worked ok (i.e. all received a PONG). When switching to non-blocking mode the first connection always works, but subsequent connections fail. Sometimes the first few connections work.

Using non-blocking IO has the aim to be able to enforce a timeout for the operations executed in clamd.

This is the test code I use:

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.ProtocolFamily;
import java.net.SocketAddress;
import java.net.StandardProtocolFamily;
import java.net.UnixDomainSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.charset.StandardCharsets;

import static java.nio.charset.StandardCharsets.UTF_8;

public class ClamdTest3 {

    private static final byte[] PING_CMD = "zPING\0".getBytes(UTF_8);

    public static void main(String[] args) throws IOException {
        ProtocolFamily protocolFamily = StandardProtocolFamily.UNIX;
        SocketAddress socketAddress = UnixDomainSocketAddress.of("/var/run/clamav/clamd.ctl");
        for (int i = 0; i < 100; i++) {
            try (SocketChannel socketChannel = SocketChannel.open(protocolFamily);
                    Selector selector = Selector.open();) {
                socketChannel.configureBlocking(false);
                socketChannel.connect(socketAddress);
                SelectionKey selectionKey = socketChannel.register(selector, 0);
                selectionKey.interestOps(SelectionKey.OP_CONNECT);
                selectionKey.selector().select(10_000);
                if (!socketChannel.finishConnect()) {
                    throw new IOException("Connection failed");
                }
                System.out.println("Connected connection: " + i);
                writeToChannel(socketChannel, selectionKey, ByteBuffer.wrap(PING_CMD), 5000);
                System.out.println(new String(readFromChannel(socketChannel, selectionKey, 10000), StandardCharsets.UTF_8));
            }
        }
    }

    private static byte[] readFromChannel(SocketChannel socketChannel, SelectionKey selectionKey, long timeout_milli) throws IOException {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        long start = System.nanoTime();
        ByteBuffer bb = ByteBuffer.allocate(4096);
        selectionKey.interestOps(SelectionKey.OP_READ);
        while (socketChannel.read(bb) >= 0) {
            bb.flip();
            baos.write(bb.array(), bb.position(), bb.limit() - bb.position());
            if (bb.limit() > 0 && bb.get(bb.limit() - 1) == 0) {
                break;
            }
            if (isTimeout(start, timeout_milli)) {
                throw new IOException("Timeout while reading");
            }
            if (!selectionKey.isReadable()) {
                selectionKey.selector().select(timeout_milli / 10);
            }
        }
        return baos.toByteArray();
    }

    private static void writeToChannel(SocketChannel socketChannel, SelectionKey selectionKey, ByteBuffer bb, long timeout_milli) throws IOException {
        long start = System.nanoTime();
        selectionKey.interestOps(SelectionKey.OP_WRITE);
        while (socketChannel.write(bb) >= 0) {
            if (bb.position() >= bb.limit()) {
                break;
            }
            if (isTimeout(start, timeout_milli)) {
                throw new IOException("Timeout while writing");
            }
            if (!selectionKey.isWritable()) {
                selectionKey.selector().select(timeout_milli / 10);
            }
        }
    }

    private static boolean isTimeout(long start, long timeout_milli) {
        long timeout_nano = timeout_milli * 1000L * 1000L;
        Long now = System.nanoTime();
        Long timeoutComp = now - timeout_nano;
        boolean isTimout = Long.compare(timeoutComp, start) > 0;
        return isTimout;
    }
}

What I would expect is a sequence of "Connected connection...PONG" messages, that start at Connected connection: 0 and finish with the 99th connection.

However I observe:

Connected connection: 0
PONG
Connected connection: 1
PONG
Connected connection: 2
PONG
Connected connection: 3
Exception in thread "main" java.io.IOException: Timeout while reading
    at ClamdTest3.readFromChannel(ClamdTest3.java:53)
    at ClamdTest3.main(ClamdTest3.java:36)

Tried:

• adding a delay after the try-catch block (Thread#sleep, 1s, 10s), same result
• adding a explicit GC call, same result
• opening multiple SocketChannels to clamd in blocking mode also works in parallel, so it does not seem to be a problem on the receiving side