Bro and Spicy for WebSockets Network Analysis, Part 2

In Part 1,  I covered the WebSockets protocol basics and the Spicy script I wrote to parse it. Here I'll cover some of the Bro scripts I wrote to use the parsed protocol.

Types of scripts and their roles in the process

The Plugin:

Using Spicy with Bro to produce WebSockets data in Bro logs involves installing the Spicy/Bro plug-in to add support for Spicy grammars to Bro. For instructions on installing the plugin, see http://www.icir.org/hilti/spicy/bro/index.html.
Once the plugin is installed, the process requires four different types of scripts. These four types are the Bro signature script (*.sig), the Spicy parsing script (*.spicy), the Bro protocol analyzer definition script (*.evt), and the Bro logging script (*.bro) (Sommer, 2016).

*.sig

The Bro signature script (*.sig) can be used to activate a protocol analyzer via a signature instead of relying on the port the traffic is using. The file will define the signature by the IP protocol (TCP or UDP), a key identifying factor of the payload (such as a regular expression), and if it is the traffic originator or responder (optional). It then specifies which protocol analyzer to enable.
I actually had trouble getting the *.sig piece to work without specifying the ports to watch. At some point I will need to go back and figure this out.

*.spicy

The Spicy script (*.spicy) is used to automatically generate the protocol parser at runtime by the Spicy framework. The parser performs its actions on the data coming in from the wire just after Bro parses the lower layers including the TCP header.

*.evt

The Bro protocol analyzer definition script (*.evt) provides the interface between Bro and the Spicy script. It specifies the grammar file that contains the Spicy script and where to hook into Bro’s traffic processing. It then defines the Bro events to create based on the parsed data.

*.bro

The Bro logging scripts (*.bro) are where an analyst can customize what information will be logged. Logs can provide protocol information about each connection, such as logging the WebSockets opcode, masking key, and payload length, or can be narrowed down to provide precise data in the WebSockets payload.

ws_handshake.evt 

This script defines significant events within the WebSockets communication. The events are triggered when a particular unit gets parsed in the Spicy script and events can pass along information about the connection. The ws_handshake.evt script defines events for:

• when a WebSockets handshake is requested, 
• when a WebSockets handshake receives a successful response, 
• and when WebSockets messages are detected. 

The message events provide different information when the WebSockets data is masked, when the data is not masked, and when there is no data.

ws_handshake.bro 

This script takes the WebSockets connection parsed by the Spicy script and focuses on exposing the WebSockets protocol fields and payload data. The messages are separated into three different types:

• masked (from the client), 
• unmasked (from the server), 
• and no data (control messages from either host) for processing within the script, 

but all three log to the same file. The log file contains connection information:
the unique identifier, the client IP address, the server IP address, and the server port 

The entire connection record is passed to the events used in this script so that any connection field could be logged. The file also contains WebSockets information (OpCode, mask key, and payload data).
Since it was not possible to unmask the client data within Spicy, it needed to be done in Bro. Unfortunately, although Bro does have the looping capability that Spicy does not have, Bro does not have the XOR functionality. Github user JustBeck wrote an elegant solution to this same problem with his bintools.bro module (2013). His module is simply a complete lookup table for each hex byte and its corresponding XOR byte. His module was used for this research so that the masked data could be unmasked and evaluated further. As it is, the ws_message.bro Bro script and resulting log file would be inefficient and unnecessary in production but was extremely useful for testing the parser and visualizing the data.

ws_messagenotnormal.bro 

This script demonstrates logging unexpected server responses in WebSockets data. With knowledge of the application being monitored, a regular expression was created that matches only expected responses. After a WebSockets message event is triggered, this Bro script uses the URI field from the connection and its corresponding regular expression to determine if the actual response is expected. If not, it creates a log entry. This technique was used with the DVWS Command Execution, Reflected XSS, and Error Based SQL Injection pages.

The ws_messagenotnormal.bro script also evaluates the request from the client for malicious content. It reviews input to the Error Based SQL Injection and Blind SQL injection pages attempting to identify SQL injection. It scores the message based on the number of SQL keywords it matches in the data. If the maximum threshold is surpassed, a log entry is created. The same technique is used with the Stored XSS page creating the score based on HTML keywords found in the input.

Bro and Spicy for WebSockets Network Analysis, Part 1

The Paper 

I've been working on a SANS Gold paper for the Network Intrusion and Analysis class to complete the STI CyberSecurity Engineering Graduate Certificate program. It was published in the SANS Reading Room last month with the title "A Spicy Approach to WebSockets: Enhancing Bro’s WebSockets Network Analysis by Generating a Custom Protocol Parser with Spicy." This post will cover the topic in a less-academic format.

I had played with Spicy for the BACNet project for the BSidesROC CTF, and wanted to see if I could write my own parser with it. I also had read a little about the WebSockets protocol, and saw that Bro did not have a native parser for it. This was the spark for my paper. (The fire was the due date.) Not wanting to reinvent the wheel, and not having a lot of time, I opted to use pre-existing Docker images whenever I could. This allowed me to quickly get the environment set up and more time to focus on writing the scripts. The goal was to write a spicy script to parse WebSockets traffic, and then write Bro scripts to log on specific events.

The Protocol 

The WebSockets protocol starts with the HTTP protocol. After the TCP three-way handshake, the client issues an HTTP GET Request. In addition to the standard GET request requirements to meet the HTTP protocol, the handshake request must include the Upgrade header field with a value of “WebSockets”, the Connection header field with a value of “Upgrade”, and the Sec-WebSocket-Version header field with a value of “13”. The handshake GET request may also contain optional HTTP header fields that could affect the WebSockets connection, such as Sec-WebSocket-Protocol, Sec-WebSocket-Extensions, or Origin.

The server portion of a successful handshake is an HTTP response with “HTTP/1.1 101 Switching Protocols” as the status line. It also includes the Connection and Upgrade header fields like the client. The server responds to the client’s Sec-WebSocket-Key header with its Sec-WebSocket-Accept field. The value of this field is derived from the client’s Sec-WebSocket-Key using a set algorithm. The client uses the same algorithm to verify the value from the server. This verification proves it is a response to the client’s request and the connection is established.

Because these are still HTTP messages, they may also include standard HTTP header fields, such as cookies or authentication/authorization fields. The headers may appear in any order and if the HTTP RFC allows, may contain more than one value, or appear more than once.

Once the handshake is complete, the WebSockets packets begin within the same TCP connection. The concept of the socket allows further traffic to be initiated by either end. The protocol does not change based on the direction. Each packet is formatted the same.

Wire Diagram from RFC

The first two bytes of a WebSockets packet are mostly flags and codes. The first one-bit flag indicates if the packet is the final fragment of a message (1) or not (0). The next three bits are reserved fields and are generally not used unless a subprotocol sets them. The next four bits are the opcode. The opcode defines how the payload data should be interpreted. The most common are text (1) and binary (2) but could also be a connection close (8), ping (9), pong (10), continuation (0), or reserved. The mask flag is a single bit that indicates if the data is masked (1) or not (0). 

The last seven bits of the first two bytes are used when the payload is less than 126 bytes. The value of the seven bits is the length of the payload. If the value of the seven bits is 126, the following two bytes represent the payload length (16-bit unsigned integer). If the value of the seven bits is 127, the following eight bytes represent the payload length (64-bit unsigned integer).

If the mask flag from the first two bytes is set to one, the four-byte masking key will start at wherever the final payload length fields end. The next byte will be the start of the payload data which completes the packet.

The Spicy Script

The Spicy script parses HTTP GET request messages looking for the GET string at the start of the packet. Within the headers, the script looks for the required “Sec-WebSocket-Version” header. If both are found, it continues to parse packets within the TCP connection from the same originating IP address and port. Follow-on packets are considered WebSockets packets and parsed accordingly.

The Spicy script also parses HTTP response packets starting with “HTTP/1.1 101 Switching Proto” and looks for the required “Sec-WebSocket-Accept” header. If both are found, it continues to parse packets within the TCP connection from the same responding IP address and port. Follow-on packets are considered WebSockets packets and parsed accordingly.

The Spicy script parses WebSockets protocol packets starting with the first two bytes as a bitfield of 16 bits. A bitfield allows the parser to identify individual bits and groups of bits used by the protocol to form header fields. The script includes logic to evaluate the last eight bits of the first two bytes. The first bit of these eight bits indicates if the data is masked and a masking key is included in the packet header. The value of the last seven bits is the payload length field. The payload length field determines where the next header field begins and if it is an extended payload length field or not. It also designates where the data ends.

The Spicy parser did not function as I expected when reaching the end of the data, which was also the end of the last packet. It seemed to need an additional marker called a lookahead token to know when to stop creating the list of WebSockets messages, even though reaching the end of the data should have negated the need for a lookahead token. Adding a distinguishing string to the end of the data before parsing the list of WebSockets messages created the lookahead token necessary for the parser to function correctly.

Although the script detects if the data is masked or not, it was not possible to unmask the data within Spicy. The required functionality to loop through bytes of data to apply the XOR function against the masking key is not present in Spicy. Primitive iterator functionality does exist to iterate through bytes of data, but only if the number of iterations can be hardcoded. Spicy does offer similar functionality, such as Base64 encoding/decoding and sha256 hashing, through functions provided by its runtime library. These are written in C programming language, which I do not know. A knowledgeable C programmer could add the XOR function here to receive the entire data field and mask key from the Spicy script and use the looping available in C, but I just left it to do in the Bro scripts.

In the next post, I'll cover the Bro scripts and testing.

Never Before Seen (NBS)

I don't recall exactly where I first heard of Marcus Ranum's Never Before Seen (NBS) anomaly detector script. It could have been in a SANS class with Paul Henry or in one of the Security Weekly Podcasts, but recently I came across the note to myself to install it and learn how it works.

NBS is a Perl script that uses a Berkeley database to store arbitrary strings of your choosing and will then output when it comes across a string it doesn't have in the database. i.e. a string that is "never before seen."

This can be useful in many situations when looking for anomalies. The man page pdf that downloads with it describes using NBS to detect new access requests to a web server. The web server logs must be parsed and the URL field fed into NBS. If it isn't in the database already, it is added and also written to the output file.  Once enough normal traffic has been seen to have established a baseline in the database, entries into the output would be considered anomalies worthy of review.

Since you provide the strings to NBS, it can be useful in many applications. Here are some that came to mind for me:

• AppLocker allowed executables pulled out of event logs
• Bro log data such as file hashes, known hosts, known services, user agent strings
• DNS queries in the DNS log

Installation:

I wanted to test using NBS with Bro logs so I installed it on my Security Onion Virtual machine. (v14.04.5.2)

1. Downloaded nbs and extracted it:
1. mkdir nbs
2. cd nbs
3. wget http://www.ranum.com/security/computer_security/code/nbs.tar
4. tar -xf nbs.tar
2. Installed BerkelyDB prerequisite. The install file that downloads with NBS is a little dated as far as the link for the libraries from what I could tell. These are the steps I took. (requires you to create an Oracle site login)
1. wget http://download.oracle.com/otn/berkeley-db/db-6.2.23.NC.tar.gz
2. gzip -d db-6.2.23.NC.tar.gz
3. tar -xf db-6.2.23.NC.tar
4. cd db-6.2.23.NC
5. sudo apt-get install g++
6. cd build_unix
7. ../dist/configure --prefix=/usr/local/berkeleydb --enable-compat185 --enable-cxx --enable-debug_rop --enable-debug_wop  (NOTE: the --enable-rpc is no longer supported)
8. make
9. sudo make install
10. sudo su -
11. echo '/usr/local/berkeleydb/lib/' >> /etc/ld.so.conf.d/libc.conf
12. ldconfig
13. apt-get install libdb-dev
14. apt-get install libdb++-dev
3. Installed NBS.  It is an older Perl script and I'm not a Linux guru, so this part might not be perfect. Just using make gave some errors so the following edits had to be made first:
1. I edited the both nbs.c  and nbspreen.o and made the following changes:
1. replaced %d with %ld in the line referenced in the error [fprintf( ...);]
2. added #include "errno.h"
3. commented out //extern int errno;
2. I then ran make in the NBS directory.
1. I did keep getting a "count variable unused" error but as far as I could tell NBS was functioning properly.
4. Configured NBS for use with Bro dns.log data. Bro is running on this system monitoring traffic.
1. Created the database
1. sudo ./nbsmk -d ~/nbs/nbs_dnsDB
2. Established the baseline for DNS data from the Bro dns.log for the db to monitor
1. sudo cat /nsm/bro/logs/current/dns.log | bro-cut query answers | sudo nbs -d nbs_dnsDB -o new_dns_entries
3. Viewed the new_dns_entries file. Since this was the first run, all the domain names from the dns.log file are listed.
1. cat new_dns_entries
4. Did a domain name lookup for a new domain for Bro to catch and add to the dns.log
1. nslookup www.rit.edu
5. Manually ran the updated dns.log file through NBS again
1. sudo cat /nsm/bro/logs/current/dns.log | bro-cut query answers | sudo nbs -d nbs_dnsDB -o new_dns_entries
6. Viewed the new_dns_entries file which was overwritten with just this new "never before seen" domain.
1. cat new_dns_entries   //shows www.rit.edu and the answer received

Next Steps:

So the next steps would be to set up a cron job to send batches of the data to NBS instead of the manual cat/bro-cut command.  Theoretically, you'd want to set this up so that only truly worrisome "never before seen" items are logged. Another mechanism would be necessary to monitor the output file for any activity and either email you or alert in some way.  

References:

• http://www.ranum.com/security/computer_security/code/  
• https://geeksww.com/tutorials/database_management_systems/berkeleydb/installation/installing_berkeleydb_on_ubuntu_linux.php 
• http://askubuntu.com/questions/509663/c-preprocessor-lib-cpp-fails-sanity-check 

BACnet - Bro - Spicy Part 4

So, in BACnet - Bro -Spicy Part 3, I went through building Docker images and starting containers running bacnet, bro, and spicy, and php and apache. These two containers share a volume on the host VM and use the host VM's network to access outside the container itself.

This last post will describe the HMI and it's php script that takes the output from the bro log and uses it to determine which animated gif to display.  The one with the ctf flag is displayed for those IPs in the log which means they successfully crafted and sent a packet with the correct BACnet write-property command.

The Plan:

1. Set up some virtual machines in my VMware environment to experiment. 
1. Install the BACnet-stack on two: server and client. (see BACnet - Bro - Spicy Part 1)
2. Install Docker and the rsmmr-hilti docker image on the bacnet server machine. (see BACnet - Bro - Spicy Part 2)
2. Write a Bro script that uses the events created by the BACnet parser to log the source IP address from packets with the correct BACnet command. (see BACnet - Bro - Spicy Part 2)
3. Install the php-apache docker image on the bacnet server machine and run it with the same shared volume as the rsmmr-hilti container where the log file is saved. (see BACnet - Bro - Spicy Part 3)
4. Write a PHP webpage/script that shows the flag only to the IPs in the log file and a different page to all other IPs

4) Write a PHP webpage/script that shows the flag only to the IPs in the log file and a different page to all other IPs

1. The index.php script is very basic and simply displays the normal.gif file unless the visiting IP address is in the bro log file in the shared volume of the Docker container. If the visiting IP is on the list, the alert.gif file is displayed, along with the flag as text over the top of the image.

<?php
$IPAddrsFile = "/tmp/bacnetCTF.log";
$IPAddrs = file($IPAddrsFile, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$normal = "images/normal.gif";
$alert = "images/alert.gif";
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <title>BACnet Monitoring</title>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    </head>
    <style>       
        h2 {
            position: fixed;
            left: 450px;
            top: 350px;
            color: orange;
        }
        body {
            background-color: #FFFFFF;
            background-repeat: no-repeat;
            background-size: 900px;
            <?php
                $visitor = (string)$_SERVER['REMOTE_ADDR'];
                    if (in_array($visitor, $IPAddrs)) {
                        echo "background-image: url(\"" . $alert . "\");\n";
                        echo "</style>\n";
                        echo "<h2>flag{PatYourselfOnTheBacNet}.</h2>";
                    } else {
                        echo "background-image: url(\"" . $normal . "\");\n";
                        echo "}\n</style>\n";
                    }
            ?>
    </body>
</html>

The normal.gif looks like this:

The alert.gif looks like this and the php script would write the flag text over the black area in the bottom right corner:

BACnet - Bro - Spicy Part 3

So, in BACnet - Bro -Spicy Part 2, I went through setting up a VM with Docker and the rsmmr-hilti docker image that had Bro and Spicy installed. I then described the modifications to the bacnet.spicy and bacnet.evt files that would allow my bacnetctf.bro script to run and detect when a specific bacnet command packet was seen.

The Plan:

1. Set up some virtual machines in my VMware environment to experiment. 
1. Install the BACnet-stack on two: server and client. (see BACnet - Bro - Spicy Part 1)
2. Install Docker and the rsmmr-hilti docker image on the bacnet server machine. (see BACnet - Bro - Spicy Part 2)
2. Write a Bro script that uses the events created by the BACnet parser to log the source IP address from packets with the correct BACnet command.
3. Install the php-apache docker image on the bacnet server machine and run it with the same shared volume as the rsmmr-hilti container where the log file is saved.
4. Write a PHP webpage/script that shows the flag only to the IPs in the log file and a different page to all other IPs

3) Install the php-apache docker image on the third machine and run it with the same shared volume as the rsmmr-hilti container where the log file is saved.

Now that we have the spicy, evt, and bro files working in the rsmmr-hilti docker container, we need to build our own docker image that will incorporate our new files every time we run a new container. Then we can set up our web server using the php-apache docker image and server our the simulated Human Machine Interface (HMI) that will indicate when a successful write-property packet is detected.  

1. In order to build a new image, I needed to create a DockerFile that defined how to get from the base image of rsmmr/hilti to my bacnet-hilti image.  This is basically a file that has its own syntax where you can do things like copy a local file into the container as it starts up, or run specific commands. This is the DockerFile for by bacnet-hilti image:
FROM rsmmr/hilti
MAINTAINER jenngates2004@gmail.com 
COPY ./node.cfg /usr/local/bro/etc/node.cfg
COPY ./bacnet.evt /opt/hilti/bro/spicy/bacnet.evt
COPY ./bacnet.spicy /opt/hilti/bro/spicy/bacnet.spicy
COPY ./properties.bro  /opt/hilti/bro/tests/spicy/bacnet/properties.bro
COPY ./myBACnet.bro /opt/hilti/bro/tests/spicy/bacnet/myBACnet.bro
COPY ./bacnetctf.bro /opt/hilti/bro/tests/spicy/bacnet/bacnetctf.bro
COPY ./*.pcap /opt/hilti/bro/tests/Traces/bacnet/
COPY ./CriticalRoom55-1.cap    /opt/hilti/bro/tests/Traces/bacnet/CriticalRoom55-1.cap
RUN mkdir /opt/bacnet-stack-0.8.3/
COPY ./bacnet-stack /opt/bacnet-stack-0.8.3/
RUN apt-get update
RUN cd /opt/bacnet-stack-0.8.3 &&  make clean all
RUN mkdir /root/brologs



2. Next I use this file to create the bacnet-hilti Docker image.
1. Made a new dir 
1. mkdir ~/bacnet-hilti
2. cd bacnet-hilti/
2. Copied .bro/.evt/.spicy/DockerFile files to it
3. Build the new image with the files inserted
1. NOTE: The space and then period at the end of the command are important!
2.  sudo docker build -t bacnet-hilti .
3. Now I run a container named bacnet-hilti1 in interactive mode (-ti) using the host VM's networking (--network=host) maps port 47808 on the host VM to forward packets to port 47808 in the bacnet-hilti1 container (-p 47808:47808) since bacnet listens on that port inside my container. I add a volume (--volume) to the container that maps a directory on the host VM (/home/bacnetchall/sharedvolume) to the /root/brologs directory in the container and allows read/write access (:rw) and tells the container it is sharing that volume with other containers (, z). The following command should be entered as one line.
1. sudo docker run --network=host --name bacnet-hilti1 -ti -p 47808:47808  --volume=/home/bacnetchall/sharedvolume:/root/brologs:rw,z bacnet-hilti
4. This starts the container and puts me at the root command prompt inside the container. Since I'm just going to run a specific Bro command so that it runs my bro script, I cd into the proper folder for the log file to be written. This is the folder the web server of the HMI will be monitoring.
1. cd brologs   
2. I start Bro listening on interface ens160 (the Docker image doesn't have the interfaces named eth0, eth1, etc). I tell Bro to use the bacnet.evt file which the new bacnet-hilti image made sure the running container has my modified version. I run the traffic through my bacnetctf.bro script, which, again, the container has the new version.  The -C tells it to ignore checksums and the & runs it in the background
1. bro -i ens160 bacnet.evt /opt/hilti/bro/tests/spicy/bacnet/bacnetctf.bro -C & 
1. NOTE!! wait for "listening on ens160". It can take a long time.
2. Once bro is running and listening for traffic to parse with the bacnetctf.bro script, I start the bacnet server installed in Step 1 of this blog series. It has the device ID of 422 and runs in the background.
1. /opt/bacnet-stack-0.8.3/demo/server/bacserv 422 &
3. To break out of the running container and back to the host VM without stopping bro, bacnet, or the container:
1. Ctrl-p Ctrl-q 
5. From the same host VM I'm also going to run a Docker container that will run a web server with the HMI webpage. I'll talk more about the webpage and php script in the next blog post but here I'm just going to get the server piece working.  Again, there is a Docker container that does 98% of what I want already called php:7.0-apache, I just need to build it to copy the webpage files into the container's /var/www/html/ directory when it runs.
1. I make a new dir called php-apache and cd into it
1. mkdir php-apache
2. cd php-apache
2. I create a new file here named DockerFile and add the following lines inside it:
1. FROM php:7.0-apache
2. COPY ./bacnet_monitor/ /var/www/html/
3. Here I make another new dir for the website files called bacnet_monitor.
1. mkdir bacnet_monitor
4. I copy over the website files into the bacnet_monitor directory
5. I build a new Docker image file called php-apache
1. sudo docker build -t php-apache .
2. NOTE: remember the space and period at the end
6. Now I run a new container of the new php-apache image file using the host VMs networking (--network=host) and name it php-apache1. It maps the host's port 80 with the container's port 80 and adds the same volume as the bacnet-hilti container, but with read-only access (:ro). This container doesn't require me to run any commands in it interactively, so we run it detached (-d).
1. sudo docker run --network=host --name php-apache1 -d -p 80:80 --volume=/home/bacnetchall/sharedvolume/:/tmp/:ro,z php-apache

Part 4 will finish up with the PHP script for the HMI and wrap up this series.

BACnet - Bro - Spicy Part 2

So in BACnet - Bro - Spicy Part 1 I explained that I've been working with the BACnet protocol developing a CTF challenge.  I'm using Bro with a Spicy parser and a logging script to detect when a specific BACnet command is send in a packet. When triggered, the sending IP Address is recorded in the log file. That file is then used to restrict access to a webpage that contains the flag. Probably not the best plan, but it got me working on something and thinking through a problem.

The Plan:

1. Set up some virtual machines in my VMware environment to experiment. 
1. Install the BACnet-stack on two: server and client. (see BACnet - Bro - Spicy Part 1)
2. Install Docker and the rsmmr-hilti docker image on the bacnet server machine.
2. Write a Bro script that uses the events created by the BACnet parser to log the source IP address from packets with the correct BACnet command.
3. Install the php-apache docker image on the bacnet server machine and run it with the same shared volume as the rsmmr-hilti container where the log file is saved.
4. Write a PHP webpage/script that shows the flag only to the IPs in the log file and a different page to all other IPs.

1.2) Install Docker and the rsmmr-hilti docker image on a third machine.

Docker was very easy and I was up and running very quickly. The rsmmr-hilti image already has Bro and Spicy installed and working. It also has BACnet parser Spicy scripts included and some test bro scripts to get an idea of how they work.

1. I started with a fresh install of Ubuntu 16.04 LTS and OpenSSH server installed and really just followed the steps at the Docker Community Edition for Ubuntu site. 
• sudo apt-get -y install apt-transport-https ca-certificates curl
• curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
• sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
• sudo apt-get update
• sudo apt-get -y install docker-ce
• sudo docker run hello-world
2. Next I installed the rsmmr-hilti image from https://hub.docker.com/r/rsmmr/hilti/ 
• mkdir ~/hilti-docker
• cd hilti-docker
• sudo docker pull rsmmr/hilti
3. The hilti site has instructions to run the docker image and test bro and spicy.
• Start the container in interactive mode with a pseudo-TTY
• sudo docker run -i -t "rsmmr/hilti"  
• Verify the BRO_PLUGIN_PATH is /opt/hilti/build/bro/
• env
• Use the spicy-driver tool to test executing a spicy parser script:
• spicy-driver hello-world.spicy
• Run bro with the verbose plugin check option. This returns info about what the provides.
• bro -NN Bro::Hilti
• Tests running the script with bro and the spicy plugin 
• bro hello-world.spicy
• Runs bro at the command line reading in the trace pcap file, using the .evt file that defines events and the the .bro file that describes the response to those events. In this case, it writes to standard out information about detected SSH banners.
• bro -r ssh-single-conn.trace ./ssh-banner.bro ssh.evt
4. From here, I started looking at what was available in the BACnet scripts.  I verified the bacnet.evt plugin was installed and working.
• bro --NN bacnet.evt
• grep for bacnet and you should see:

Bro::Hilti - Dynamically compiled HILTI/Spicy functionality (*.spicy, *.evt, *.hlt) (dynamic, version 0.1)
[Analyzer] spicy_BACnet (ANALYZER_SPICY_BACNET, enabled)
[Event] bacnet_message
[Event] bacnet_bvlc_result

1. The bacnet .bro test script files are located in:
• /opt/hilti/bro/tests/spicy/bacnet
2. The bacnet trace files are located in:
•  /opt/hilti/bro/Traces/bacnet
3. The docker image comes with some pcap files capturing specific bacnet traffic and bro scripts to detect bacnet commands.  For example, the who-has.pcap captures who-has traffic and the apdu_who.bro script will detect it.  This is the command to test a .bro script is that uses a definition in the bacnet.evt plugin:

• bro -r /opt/hilti/bro/tests/Traces/bacnet/who-has.pcap bacnet.evt /opt/hilti/bro/tests/spicy/bacnet/apdu_who.bro

2) Write a Bro script that uses the events created using the BACnet parser to log the source IP address from packets with the correct BACnet command.

Spicy (formerly BinPAC++) is a language used to write a protocol parser for Bro. Spicy files have the extension .spicy and they define how to parse a packet.  Bro uses .evt files to determine which parser is used to parse each kind of traffic and which events to generate based on that parsed traffic.  In my case, the bacnet.spicy and bacnet.evt files already existed within the rsmmr/hilti docker image, so I only needed to modify them to create an event based on the traffic I was watching for; specifically the write-property command packet.

bacnet.spicy

In the bacnet.spicy file, I needed to keep track of the segments. All segments of a segmented request must have the same invokeID. The invokeID should be incremented for each new BACnet-Confirmed-Request-PDU. So I modified the /opt/hilti/bro/spicy/bacnet.spicy file as follows:

[...]

type PDU_confirmedRequest = unit(head: bytes, len: uint16) {

  var len: uint16 = 4;

  var segmentation_status: bool = False;

  [...]

  switch ( self.confirmed_service_choice ) {

    BACnetConfirmedServiceChoice::confirmedEventNotification -> confirmedEventNotification: ConfirmedEventNotification_Request( len - self.len );

    BACnetConfirmedServiceChoice::readProperty -> readProperty: ReadProperty_Request( len - self.len, self.invokeID );

    BACnetConfirmedServiceChoice::writeProperty -> writeProperty: WriteProperty_Request( len - self.len, self.invokeID );

  [...]

  switch ( self.confirmed_service_choice ) {

    BACnetConfirmedServiceChoice::confirmedEventNotification -> confirmedEventNotification: ConfirmedEventNotification_Request( len - self.len );

    BACnetConfirmedServiceChoice::readProperty -> readProperty: ReadProperty_Request( len - self.len, self.invokeID );

    BACnetConfirmedServiceChoice::writeProperty -> writeProperty: WriteProperty_Request( len - self.len, self.invokeID );

[...]

type WriteProperty_Request = unit(len: uint16, invokeID: uint8) {
  # context-specific tags 0 - 4

  first_tag: BACnetTag;

[...]

I also needed to work around an issue with the property value I was looking for being part of an array, but the parser not fully working yet with this array. What I changed worked for my very scoped purpose, but really isn't the solution to the array issue overall.  However, I again modified the /opt/hilti/bro/spicy/bacnet.spicy file as follows:

[...]

type WriteProperty_Request = unit(len: uint16, invokeID: uint8) {

  # context-specific tags 0 - 4

  first_tag: BACnetTag;

  objectIdentifier: BACnetContextMessage(BACnetType::BACnetObjectIdentifier, 0, self.first_tag);

  propertyIdentifier_tag: BACnetTag; # tag 1

  propertyIdentifier: bytes &length=self.propertyIdentifier_tag.lvt &convert=BACnetPropertyIdentifier($$.to_uint(Spicy::ByteOrder::Network));

  propertyArrayIndex: BACnetContextMessage(BACnetType::UnsignedInteger, 2, self.propertyIdentifier_tag);

  propertyValue_tag: BACnetTag if ( self.propertyArrayIndex.tag.tag != 3 ); # check if previous tag was optional. This tag = 3

  #propertyValue: BACnetArray_Partial(3);

  propertyValue: BACnetMessage;

[...]

bacnet.evt

I then needed to modify the bacnet.evt file to generate an event for the write property I was watching for. I had to add some logic to create different events for write property packets that were writing analog (realval) values and those writing digital (enumerated) values. Then I used those to generate and event when the property being written was an enumerated value of type 4 (Binary Output), instance 3, property identifier 85 (present_value). This was the packet I would be looking for. 

I added the following lines to the end of  /opt/hilti/bro/spicy/bacnet.evt:

[...]

on BACnet::WriteProperty_Request if (self.propertyValue.tag.tpe == BACnetType::Real)

  -> event bacnet_writeproperty_request($conn, cast<uint64>(invokeID), self.objectIdentifier.oid.tpe, cast<uint64>(self.objectIdentifier.oid.data.instanceNumber), self.propertyIdentifier, self.propertyValue.realval);

on BACnet::WriteProperty_Request if ((self.objectIdentifier.oid.tpe == 4) && (self.propertyValue.tag.tpe == BACnetType::Enumerated) && (self.objectIdentifier.oid.data.instanceNumber == 3) && (self.propertyIdentifier == 85))

  -> event bacnetctf_writeproperty_request($conn, cast<uint64>(invokeID), self.objectIdentifier.oid.tpe, cast<uint64>(self.objectIdentifier.oid.data.instanceNumber), self.propertyIdentifier, self.propertyValue.unsigned);

bacnetctf.bro
A Bro script uses events that are generated to trigger some actions. I wrote a bro script that would pull the IP address from the selected packet that triggered the event my bacnet.evt file additions created. It wrote that IP to a file which would be used later on. Here is the script called bacnetctf.bro:

#Module to parse bacnet ctf writes to binary output 3 on device 422

#load processes the __load__.bro scripts in the directories loaded 
#which basically includes libraries
@load base/protocols/conn

#create namespace 
module bacnetCTF;

export {
#Create an ID for our new stream. 
redef enum Log::ID += { LOG };

#Define the record type that will contain the data to log.
type Info: record {
packet_src: addr &log;
};
}

event bro_init()  &priority=5
{
#Create the stream. this adds a default filter automatically
Log::create_stream(bacnetCTF::LOG, [$columns=Info, $path="bacnetCTF"]);
}

#add a new field to the connection record so that data is accessible in variety of event handlers
redef record connection += {
bacnetctf: Info &optional;
};

event bacnetctf_writeproperty_request(c: connection, invokeID: count, tpe: BACnet::BACnetObjectType, instance: count, property: BACnet::BACnetPropertyIdentifier, propval: string)
   {

   #Don't count packets from ssh box (simulate coming from different subnet to force packet crafting)
 local src_ip: addr;

 if (c$id$orig_h == 192.168.141.111)
 {
       src_ip = 192.168.1.1;
 } else
 {
       src_ip = c$id$orig_h;
 }

  #Log format
  local rec: bacnetCTF::Info = [$packet_src=src_ip];

  c$bacnetctf = rec;

  Log::write(bacnetCTF::LOG, rec);

}

Part 3 will pick up from here with building the docker image with these new files.

BACnet - Bro - Spicy Part 1

I've been working on creating a CTF challenge that involves the BACnet/IP Building and Automation Control Networks protocol. I found Steve Karg's BACnet-Stack project, which is a great way to get to know the protocol.  He has included with the stack a demo server and client that let's you start sending packets without needing a physical device. You can also set it up on a Raspberry Pi or similar device to bring the physical world into play with LEDs, or sensors, or whatever you want to manipulate.

Unfortunately, I don't know enough C programming to get his demo client and server to do exactly what I wanted. I did learn a bunch about compiling and debugging C code as I tried to understand what was happening at different points in the communication. I also found that Steve is tremendously kind and was very responsive to my emails.

So I continued to use the demo server and client, but needed something else to detect a specific action. I'm being vague since it is a CTF challenge I'm creating.  Anyway, I needed not only to detect that the change was made, but also "who" made the change. Since BACnet is unauthenticated and over UDP, I was going to have to rely on the IP address in the packet, and due to the way the protocol layers of BACnet are stacked on UDP, it didn't look like I could get that information from the bacnet-stack.  At least not without learning C programming and rewriting the demo server code.

I then got the idea to use Bro to detect a specific packet as it came across the wire. Bro doesn't have a parser for BACnet, and I wasn't about to try to write one for the whole protocol, but I thought maybe I could write one for that very specific packet.  Luckily, I didn't have to try that path. I found a BACnet parser that is written in Spicy (formerly BinPAC++) so I went about trying to use that.

Here is my learning adventure:

I started by watching this presentation by Robin Sommer from a few years ago describing how easy it is to create a parser. (BinPAC++ Demo by Robin Sommer)  At least it sounded easy when he explained it!

I also found these articles:

• "Specification Mining for Intrusion Detection in Networked Control Systems"
• "Spicy: A Unified Deep Packet InspectionFramework Dissecting All Your Data"

Which led me to these sites:

• Spicy - http://www.icir.org/hilti
• Bro Scripting - https://www.bro.org/sphinx-git/scripting/index.html
• Docker - https://docs.docker.com/engine/installation/
• BACnet - http://bacnet.sourceforge.net/

The Plan:

1. Set up some virtual machines in my VMware environment to experiment. 
1. Install the BACnet-stack on two: server and client.
2. Install Docker and the rsmmr-hilti docker image on the bacnet server machine.
2. Write a Bro script that uses the events created by the BACnet parser to log the source IP address from packets with the correct BACnet command.
3. Install the php-apache docker image on the bacnet server machine and run it with the same shared volume as the rsmmr-hilti container where the log file is saved.
4. Write a PHP webpage/script that shows the flag only to the IPs in the log file and a different page to all other IPs.

1) Virtual Machines - BACnet

I ended up just using some VMs I already had. It works with many operating systems fairly easily. I used Ubuntu and a Kali Linux box.

bacnet-stack Installation Steps:

1. Get up to date
• sudo apt-get update
2. Install compiler package
• sudo apt-get install build-essential
3. Make a directory to install to and run it from
• mkdir /opt/bacnet 
• cd /opt/bacnet 
4. Download the latest version of bacnet-stack
• wget http://sourceforge.net/projects/bacnet/files/bacnet-stack/bacnet-stack-0.8.3/bacnet-stack-0.8.3.tgz
5. Unzip
• tar xvfz bacnet-stack-0.8.3.tgz
6. Make/install all including demos.
• cd /opt/bacnet/bacnet-stack-0.8.3
• make clean all

Edit device being served by bacserv (server)

The server represents a device that uses BACnet to communicate. They are often sensors or controllers of some sort that report back to a control system.  They are object oriented. Each device is a "Device Object" and has a set of properties with values and services or methods. A device object can have will have several standard objects, such as sensor objects like an Analog Input object. Each standard object will also have properties with values and services to perform on them. (See http://www.bacnet.org/Bibliography/ES-7-96/ES-7-96.htm for more info.)

I wanted to add some customization to my device, but didn't really know how. I poked through the code and found some static char variables that I edited just to see if it would work.

1. Make a backup of original
• cp demo/object/device.c  demo/object/device.bak
2. Edit device.c with changes
• static uint16_t Vendor_Identifier = BACNET_VENDOR_ID;
  static char Model_Name[MAX_DEV_MOD_LEN + 1] = "My Changed Model Name";
  static char Application_Software_Version[MAX_DEV_VER_LEN + 1] = "2017.0";
  static char Location[MAX_DEV_LOC_LEN + 1] = "My Changed Location";
  static char Description[MAX_DEV_DESC_LEN + 1] = "My Changed Description";
3. From /opt/bacnet/bacnet-stack-0.8.3/  run:
1. make clean all

Run Demo device (server)

• sudo ./demo/server/bacserv    //let device ID be auto generated 
• sudo ./demo/server/bacserv 123   //designate the device ID

Run query and discovery tools on device (client)

• Get help
• use the --help switch on any of the demo tools to get usage.
• Device discovery on network
• ./demo/whois/bacwi -1 
• Read all the required properties from the device 123 and display values
• ./demo/epics/bacepics -v  123
• Read the object_identifier property from device 123
• ./demo/readprop/bacrp 123 8 123 75
• Write 100.0 (REAL=4 datatype) to the present_value property (85) of Device 123 Analog Output (1) One (1) at priority 16 with no index (-1)
• ./demo/writeprop/bacwp 123 1 1 85 16 -1 4 100.0  
• Read in value set
• ./demo/readprop/bacrp 123 1 1 85  
• Read in the present_value property (85) of Analog Output (1) One (1) of Device 123
• ./demo/readprop/bacrp 123 1 1 85 
• Write to device 1234's Binary Output (4) number one (1) present_value property (85) at priority 16 with no index (-1) the data type Enumerated (9) to make active (1)
• /demo/writeprop/bacwp 1234 4 1 85 16 -1 9 1

Part 2 will pick up from here with Docker, Spicy, and Bro files.