How http2 build connection

Overview

In one of my previous articles: Handling GRPC Go Connection Handshake Issues, I described a problem I encountered with I described a Go GRPC problem I encountered and how it was eventually solved, and finally left myself a pitfall: how the http2 connection was established. Here’s how to fill that hole.

Classification of http2

http2 has a variety of different connection establishment scenarios, which can be divided into two types in a nutshell.

• h2c: http2 without TLS
• h2: http2 with TLS

However, in practice, this division lacks practicality, so in practice it is divided as follows

• Upgrade from http/1.1 to h2c
• h2c with a priori knowledge
• TLS-ALPN

Upgrade from http1.1 to h2c

When the client does not know if the server supports http2, it first tries to send an http request to the server as http/1.1, with a Header: Upgrade: h2c, so that if the server supports http2, it will respond with an upgrade operation: for example

[root@liqiang.io]# cat http1.req
     GET / HTTP/1.1
     Host: server.example.com
     Connection: Upgrade, HTTP2-Settings
     Upgrade: h2c
     HTTP2-Settings: <base64url encoding of HTTP/2 SETTINGS payload>

If the server supports http2, the response is a 101 response for Switch Protocols.

[root@liqiang.io]# cat http101.resp
     HTTP/1.1 101 Switching Protocols
     Connection: Upgrade
     Upgrade: h2c
     [ HTTP/2 connection ...

Here is a detail to note, there is a Header in the request named: HTTP2-Settings, which contains the setting information of http2 (Base64 encoding with “=” ending), that is, if the server accepts http2, then it will use this Setting That is to say, if the server accepts http2, then it will use this Setting for http2 communication, and, in the following 101 response, the body is definitely the server’s Connection Preface, which also contains the Server side of the Setting information, where the Body is a binary stream that meets certain conditions, see the following: Connection Preface.

Although the Client sends an HTTP2-Settings here, the RFC requires that the Cleint side should also send a Setting Frame after receiving the 101 response and will overwrite the initial Setting (it is a bit strange why this step is still needed).

As to what information is placed in the Setting, you can see later: .

h2c with a priori knowledge

If the Client (e.g. Browser) has already communicated with the Server before, it has already had the experience that the Server supports http2. Then the Client can communicate with the Server directly using http2, i.e. without the http/1.1 step. However, according to the RFC, when a TCP connection (h2c is the value of http2 without the TLS connection) is established, the first packet sent by the Client should be Connection Preface.

For the same reason see: Connection Preface below.

Then, if the Server side handles http2 correctly, it must also respond with a Connection Preface, and then the Client side can communicate with the Server side happily with http2.

TLS-ALPN

http2 over TLS is the recommended http2, and there are some differences between http2 over TLS and h2c. In h2c, we have to know if the Server side supports http2, either by switching protocols through 101 or by having a priori knowledge. But in TLS, because of the special nature of TLS in the network protocol stack (between Transport Layer and Application Layer), Google has proposed a protocol negotiation of Application Layer during TLS handshake, the initial version is NPN (Next Protocol Negotiation), which was later abandoned and changed to ALPN (Application Layer Protocol Negotiation).

In other words, it supports negotiating whether the application layer uses http2 during the TLS handshake, so that when the TLS channel is established, there is actually “a priori knowledge” to know whether the Server side supports http2, if it does, then it is a Connection Preface back and forth, if not, then The negotiation message of ALPN is sent in the Client Hello phase of https, and an example Client Hello message is

[root@liqiang.io]# cat https-alpn-client-hello.req
    Handshake Type: Client Hello (1)
    Length: 141
    Version: TLS 1.2 (0x0303)
    Random: dd67b5943e5efd0740519f38071008b59efbd68ab3114587...
    Session ID Length: 0
    Cipher Suites Length: 10
    Cipher Suites (5 suites)
    Compression Methods Length: 1
    Compression Methods (1 method)
    Cipher Suites Length: 10 Cipher Suites (5 suites)
    [other extensions omitted]
    Extension: application_layer_protocol_negotiation (len=14)
        Type: application_layer_protocol_negotiation (16)
        Length: 14
        ALPN Extension Length: 12
        ALPN Protocol
            ALPN string length: 2
            ALPN Next Protocol: h2
            ALPN string length: 8
            ALPN Next Protocol: http/1.1

Connection Preface

I always mention Connection Preface during the connection establishment process, but the requirement in the RFC is that neither the Client nor the Server must send the Connection Preface to the other before a connection can be officially used.

A Connection Preface is actually an HTTP binary request, but it can be partially ASCII decoded, for example, a Client’s request would be

[root@liqiang.io]# cat client-connection-preface.req
0x505249202a20485454502f322e300d0a0d0a534d0d0a0d0a

that can be translated to ASCII.

[root@liqiang.io]# cat client-connection-preface-decode.req
PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n

In addition to this paragraph, the “Body” section of the original http/1.1 may also contain the Setting Frame, which is still necessary from the RFC because the Connection Preface does these things.

• Provide a way for the Client and Server side to understand the configuration of both sides.
• In the Server side or intermediate Proxy channels do not support http2, you can get timely feedback (because they do not support PRI and HTTP/2.0 semantics)

When the Client or Server receives a request for Connection Preface from the other side, both need to give the ACK.

• For the Client, the Server’s ACK may be the Connection Preface returned by the Server after accepting the Client’s Setting
• For the Server side, the ACK of the Client is a Setting Frame with ACK tags

Setting Frame

In http2, the Setting Frame is used to coordinate the communication configuration information between Client and Server. When initializing the connection, the RFC requires that both sides must send a Setting Frame and the received side must give an ACK (Setting Frame with ACK flag). The Client and Server can send the Setting Frame multiple times, and both sides process with the latest Setting received.

What are the negotiation messages in the Setting Frame, which are currently defined in the RFC are.

• SETTINGS_HEADER_TABLE_SIZE (0x1)
• SETTINGS_ENABLE_PUSH (0x2)
• SETTINGS_MAX_CONCURRENT_STREAMS (0x3)
• SETTINGS_INITIAL_WINDOW_SIZE (0x4)
• SETTINGS_MAX_FRAME_SIZE (0x5)
• SETTINGS_MAX_HEADER_LIST_SIZE (0x6)

Others that are not defined will be ignored if they appear in the Setting Frame.

Demo operation

Protocol Watch

Here is the protocol switching process I found after running the h2c server and viewing it via curl.

[root@liqiang.io]# curl --http2 -v http://localhost:8115
* Trying ::1:8115...
* connect to ::1 port 8115 failed: Connection refused
* Trying 127.0.0.1:8115... * Trying 127.0.0.1:8115...
* Connected to localhost (127.0.0.1) port 8115 (#0)
> GET / HTTP/1.1
> Host: localhost:8115
> User-Agent: curl/7.74.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQCAAAAAAIAAAAA
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 101 Switching Protocols
< Connection: Upgrade
< Upgrade: h2c
* Received 101
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< accept-ranges: bytes
< content-type: text/html; charset=utf-8
< last-modified: Mon, 22 Feb 2021 07:04:12 GMT
< content-length: 44912
< date: Mon, 22 Feb 2021 07:50:21 GMT
< 
<html
<head lang="en"></head>

If interested, the source code I’ve put up, you can use it directly: https://github.com/liuliqiang/http2

Off-topic knowledge

HTTPS connection establishment schematic

Personal Insights

Setting Frame in Connection Preface can be null and contradictory

The RFC says in the Connection Preface that the Setting Frame can be empty, but in the Setting Frame it says that the Setting Frame must be sent when the connection is initialized, is this contradictory?

My understanding is that the sending of the Connection Preface does not mean that the http2 connection has been successfully established, so if the Connection Preface does not carry a Setting Frame, then a separate Setting Frame will be sent later (before any data Frame).

How to step in the pitfalls of Grpc Go

The problem with encountering GRPC that I mentioned earlier is described as

Figure 1: grpc go problem

This means that the Client sent the Setting Frame, then waited for the Server’s Setting Frame, but the Server never sent it (at first this was a bug in the Lib of cmux, then later because the Java Client first blocked because of This is the reason why we are using cmux incorrectly).

So to sum up the whole process looks like this.

• Both grpc client and middleware (cmux) use http2 insecurely (but in compliance with the RFC, which describes that the Client can send data as long as it sends the setting frame itself).
• The grpc client fixes the insecurity first and must receive the setting frame from the server before it starts sending data, so it does not match the existing middleware.
• grpc server and middleware fix the insecurity and things are solved.

Ref

• http2 Connection Preface
• ALPN
• Grpc Go Issue #2406