The Paper
I've been working on a SANS Gold paper for the Network Intrusion and Analysis class to complete the STI CyberSecurity Engineering Graduate Certificate program. It was published in the SANS Reading Room last month with the title "A Spicy Approach to WebSockets: Enhancing Bro’s WebSockets Network Analysis by Generating a Custom Protocol Parser with Spicy." This post will cover the topic in a less-academic format.I had played with Spicy for the BACNet project for the BSidesROC CTF, and wanted to see if I could write my own parser with it. I also had read a little about the WebSockets protocol, and saw that Bro did not have a native parser for it. This was the spark for my paper. (The fire was the due date.) Not wanting to reinvent the wheel, and not having a lot of time, I opted to use pre-existing Docker images whenever I could. This allowed me to quickly get the environment set up and more time to focus on writing the scripts. The goal was to write a spicy script to parse WebSockets traffic, and then write Bro scripts to log on specific events.
The Protocol
The WebSockets protocol starts with the HTTP protocol. After the TCP three-way handshake, the client issues an HTTP GET Request. In addition to the standard GET request requirements to meet the HTTP protocol, the handshake request must include the Upgrade header field with a value of “WebSockets”, the Connection header field with a value of “Upgrade”, and the Sec-WebSocket-Version header field with a value of “13”. The handshake GET request may also contain optional HTTP header fields that could affect the WebSockets connection, such as Sec-WebSocket-Protocol, Sec-WebSocket-Extensions, or Origin.The server portion of a successful handshake is an HTTP response with “HTTP/1.1 101 Switching Protocols” as the status line. It also includes the Connection and Upgrade header fields like the client. The server responds to the client’s Sec-WebSocket-Key header with its Sec-WebSocket-Accept field. The value of this field is derived from the client’s Sec-WebSocket-Key using a set algorithm. The client uses the same algorithm to verify the value from the server. This verification proves it is a response to the client’s request and the connection is established.
Because these are still HTTP messages, they may also include standard HTTP header fields, such as cookies or authentication/authorization fields. The headers may appear in any order and if the HTTP RFC allows, may contain more than one value, or appear more than once.
Once the handshake is complete, the WebSockets packets begin within the same TCP connection. The concept of the socket allows further traffic to be initiated by either end. The protocol does not change based on the direction. Each packet is formatted the same.
 |
| Wire Diagram from RFC |
The first two bytes of a WebSockets packet are mostly flags and codes. The first one-bit flag indicates if the packet is the final fragment of a message (1) or not (0). The next three bits are reserved fields and are generally not used unless a subprotocol sets them. The next four bits are the opcode. The opcode defines how the payload data should be interpreted. The most common are text (1) and binary (2) but could also be a connection close (8), ping (9), pong (10), continuation (0), or reserved. The mask flag is a single bit that indicates if the data is masked (1) or not (0).
The last seven bits of the first two bytes are used when the payload is less than 126 bytes. The value of the seven bits is the length of the payload. If the value of the seven bits is 126, the following two bytes represent the payload length (16-bit unsigned integer). If the value of the seven bits is 127, the following eight bytes represent the payload length (64-bit unsigned integer).
If the mask flag from the first two bytes is set to one, the four-byte masking key will start at wherever the final payload length fields end. The next byte will be the start of the payload data which completes the packet.
The Spicy Script
The Spicy script parses HTTP GET request messages looking for the GET string at the start of the packet. Within the headers, the script looks for the required “Sec-WebSocket-Version” header. If both are found, it continues to parse packets within the TCP connection from the same originating IP address and port. Follow-on packets are considered WebSockets packets and parsed accordingly.The Spicy script also parses HTTP response packets starting with “HTTP/1.1 101 Switching Proto” and looks for the required “Sec-WebSocket-Accept” header. If both are found, it continues to parse packets within the TCP connection from the same responding IP address and port. Follow-on packets are considered WebSockets packets and parsed accordingly.
The Spicy script parses WebSockets protocol packets starting with the first two bytes as a bitfield of 16 bits. A bitfield allows the parser to identify individual bits and groups of bits used by the protocol to form header fields. The script includes logic to evaluate the last eight bits of the first two bytes. The first bit of these eight bits indicates if the data is masked and a masking key is included in the packet header. The value of the last seven bits is the payload length field. The payload length field determines where the next header field begins and if it is an extended payload length field or not. It also designates where the data ends.
The Spicy parser did not function as I expected when reaching the end of the data, which was also the end of the last packet. It seemed to need an additional marker called a lookahead token to know when to stop creating the list of WebSockets messages, even though reaching the end of the data should have negated the need for a lookahead token. Adding a distinguishing string to the end of the data before parsing the list of WebSockets messages created the lookahead token necessary for the parser to function correctly.
Although the script detects if the data is masked or not, it was not possible to unmask the data within Spicy. The required functionality to loop through bytes of data to apply the XOR function against the masking key is not present in Spicy. Primitive iterator functionality does exist to iterate through bytes of data, but only if the number of iterations can be hardcoded. Spicy does offer similar functionality, such as Base64 encoding/decoding and sha256 hashing, through functions provided by its runtime library. These are written in C programming language, which I do not know. A knowledgeable C programmer could add the XOR function here to receive the entire data field and mask key from the Spicy script and use the looping available in C, but I just left it to do in the Bro scripts.
