How to configure Netty 4 HTTP/2 client to get individual frames

Question

I'm writing a Netty based HTTP/2 service that does "server sent events" (SSE) and I need to write a HTTP/2 test client class for use in integration tests, but I'm having hard time working out how to set up my client pipeline so that I can get individual events from the server.

I first tried using the HTTP/1.1 ↔ HTTP/2 adapter classes (InboundHttp2ToHttpAdapter + HttpToHttp2ConnectionHandler) but with this combo I only got the FullHttpResponse after the stream was closed, not the individual HttpContent objects.

Next, I tried setting up a pipeline where I send individual outgoing HTTP/2 frames and listen for incoming HTTP/2 frames, but it seem the outgoing HTTP/2 frame encoder is not properly installed, as I get the following exception:

Exception in thread "main" java.lang.RuntimeException: io.netty.handler.codec.UnsupportedMessageTypeException: io.netty.handler.codec.http2.DefaultHttp2HeadersFrame (expected: io.netty.buffer.ByteBuf)
    at example.AsyncHttpClient.run(AsyncHttpClient.java:116)
    at example.AsyncHttpClient.main(AsyncHttpClient.java:49)
Caused by: io.netty.handler.codec.UnsupportedMessageTypeException: io.netty.handler.codec.http2.DefaultHttp2HeadersFrame (expected: io.netty.buffer.ByteBuf)
    at io.netty.handler.ssl.SslHandler.write(SslHandler.java:694)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:816)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:723)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.write(Http2ConnectionHandler.java:498)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730)
    at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38)
    at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1081)
    at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1128)
    at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1070)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
    at java.lang.Thread.run(Thread.java:745)

Any ideas about what I'm doing wrong? I find the package io.netty.handler.codec.http2 to be quite confusing with respect to how the classes should be composed. I have tried using the Netty examples on Github as a guide, but from what I can find, most of the use HTTP/1.1 <-> HTTP/2 adapter classes and that's not applicable in my case. If there are better examples on how to use these classes, please let me know!

Full source code (also on github):

package example;

import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.http2.DefaultHttp2Connection;
import io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder;
import io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder;
import io.netty.handler.codec.http2.DefaultHttp2FrameReader;
import io.netty.handler.codec.http2.DefaultHttp2FrameWriter;
import io.netty.handler.codec.http2.DefaultHttp2Headers;
import io.netty.handler.codec.http2.DefaultHttp2HeadersFrame;
import io.netty.handler.codec.http2.Http2ConnectionHandler;
import io.netty.handler.codec.http2.Http2ConnectionHandlerBuilder;
import io.netty.handler.codec.http2.Http2Exception;
import io.netty.handler.codec.http2.Http2FrameAdapter;
import io.netty.handler.codec.http2.Http2Headers;
import io.netty.handler.codec.http2.Http2SecurityUtil;
import io.netty.handler.ssl.ApplicationProtocolConfig;
import io.netty.handler.ssl.ApplicationProtocolNames;
import io.netty.handler.ssl.OpenSsl;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslHandler;
import io.netty.handler.ssl.SslProvider;
import io.netty.handler.ssl.SupportedCipherSuiteFilter;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;

import javax.net.ssl.SSLException;
import java.util.Collections;
import java.util.Map;

import static io.netty.channel.ChannelFutureListener.FIRE_EXCEPTION_ON_FAILURE;

public class AsyncHttpClient {

    private final Channel channel;

    public static void main(String[] args) throws InterruptedException {
        final AsyncHttpClient client = new AsyncHttpClient("google.com", 443);
        client.run("GET", "/", Collections.emptyMap());
        Thread.sleep(10000);
    }

    AsyncHttpClient(String host, int port) throws InterruptedException {
        final Bootstrap bootstrap = new Bootstrap()
                .group(new NioEventLoopGroup())
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel ch) throws Exception {
                        final Http2FrameAdapter adapter = new Http2FrameAdapter() {

                            @Override
                            public void onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int padding, boolean endStream) throws Http2Exception {
                                System.out.println("onHeadersRead(ctx, streamId, headers, padding, endStream)");
                                super.onHeadersRead(ctx, streamId, headers, padding, endStream);
                            }

                            @Override
                            public void onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int streamDependency, short weight, boolean exclusive, int padding, boolean endStream) throws Http2Exception {
                                System.out.println("onHeadersRead(ctx, streamId, headers, streamDependency, weight, exclusive, padding, endStream)");
                                super.onHeadersRead(ctx, streamId, headers, streamDependency, weight, exclusive, padding, endStream);
                            }

                            @Override
                            public int onDataRead(ChannelHandlerContext ctx, int streamId, ByteBuf data, int padding, boolean endOfStream) throws Http2Exception {
                                System.out.println("onDataRead(ctx, streamId, data, padding, endOfStream)");
                                return super.onDataRead(ctx, streamId, data, padding, endOfStream);
                            }
                        };

                        final DefaultHttp2Connection connection = new DefaultHttp2Connection(false);
                        final DefaultHttp2FrameReader frameReader = new DefaultHttp2FrameReader();
                        final DefaultHttp2FrameWriter frameWriter = new DefaultHttp2FrameWriter();
                        final DefaultHttp2ConnectionEncoder encoder = new DefaultHttp2ConnectionEncoder(connection, frameWriter);
                        final DefaultHttp2ConnectionDecoder decoder = new DefaultHttp2ConnectionDecoder(connection, encoder, frameReader);
                        final Http2ConnectionHandler connectionHandler = new Http2ConnectionHandlerBuilder()
                                .codec(decoder, encoder)
                                .frameListener(adapter)
                                .build();

                        final ChannelPipeline pipeline = ch.pipeline();
                        pipeline.addLast(createSslHandler(ch));
                        pipeline.addLast(connectionHandler);
                    }
                });

        final ChannelFuture channelFuture = bootstrap.connect(host, port);
        this.channel = channelFuture.sync().channel();
    }

    void run(String method, String path, Map<String, String> headerMap) {
        final DefaultHttp2Headers headers = new DefaultHttp2Headers(true);
        headers.scheme("https");
        headers.method(method);
        headers.path(path);
        for (Map.Entry<String, String> header : headerMap.entrySet()) {
            headers.add(header.getKey(), header.getValue());
        }

        final DefaultHttp2HeadersFrame frame = new DefaultHttp2HeadersFrame(headers, false);
        try {
            channel.writeAndFlush(frame)
                    .addListener(FIRE_EXCEPTION_ON_FAILURE)
                    .sync();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    private SslHandler createSslHandler(SocketChannel channel) {
        try {
            final SslProvider provider = OpenSsl.isAlpnSupported() ? SslProvider.OPENSSL : SslProvider.JDK;
            final SslContext sslCtx = SslContextBuilder.forClient().sslProvider(provider)
            /* NOTE: the cipher filter may not include all ciphers required by the HTTP/2 specification.
            * Please refer to the HTTP/2 specification for cipher requirements. */
                    .ciphers(Http2SecurityUtil.CIPHERS, SupportedCipherSuiteFilter.INSTANCE)
                    .trustManager(InsecureTrustManagerFactory.INSTANCE)
                    .applicationProtocolConfig(new ApplicationProtocolConfig(
                            ApplicationProtocolConfig.Protocol.ALPN,
                            // NO_ADVERTISE is currently the only mode supported by both OpenSsl and JDK providers.
                            ApplicationProtocolConfig.SelectorFailureBehavior.NO_ADVERTISE,
                            // ACCEPT is currently the only mode supported by both OpenSsl and JDK providers.
                            ApplicationProtocolConfig.SelectedListenerFailureBehavior.ACCEPT,
                            ApplicationProtocolNames.HTTP_2))
                    .build();
            return sslCtx.newHandler(channel.alloc());
        } catch (SSLException e) {
            throw new RuntimeException(e);
        }
    }
}

Norman Maurer · Accepted Answer · 2017-09-07 18:15:06Z

2

If you want to use the Http2*Frame objects you need to use either the Http2FrameCodec or the Http2MultiplexCodec and not the Http2ConnectionHandler at all. See the netty examples and unit-tests for how to use it in detail.

answered Sep 7, 2017 at 18:15

Norman Maurer

23.7k2 gold badges36 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

oddbjorn Over a year ago

Thanks! I had a look at Http2FrameCodec, but I could not find a useable client example neither in the netty examples, nor on Github in general. If I do manage to implement a working client using that class, I'll make sure to post a example PR to the netty repo. The Http2FrameCodec states that its API is unstable and incomplete, but how bad is it really?

oddbjorn Over a year ago

To get this working I ended up using the HTTP/1.1 ↔ HTTP/2 adapter classes InboundHttp2ToHttpAdapter and HttpToHttp2ConnectionHandler. I extended DetailedHttpObjectAggregator for HTTP/1.1 and InboundHttp2ToHttpAdapter for HTTP/2 to emit HttpResponse, HttpHeaders, HttpContent and LastHttpContent to the pipeline. Full source code here.

Norman Maurer Over a year ago

I will add a client example over the next days that use the Http2MultiplexCodec and Http2FameCodec. You are right there are only server examples atm

Bennett Lynch Over a year ago

@NormanMaurer The example package still seems to be lacking a Client example using the new multiplex API, would it be possible to add one still?

Collectives™ on Stack Overflow

How to configure Netty 4 HTTP/2 client to get individual frames

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related