Thursday, January 19, 2017

Bro Custom Scripts

The Premise:

While working with Security Onion, I wanted to do more with Bro and custom scripts. I worked on the exercises at try.bro.org and watched a few videos on YouTube. I had a pretty good understanding of how the language worked, but it wasn't until I bought a new router and started sending the syslog output to the onion that I found a "problem" I could work on solving with a custom Bro script.

Basically, the router's syslog would have entries for accepted and dropped traffic, DHCP IP assignments, and other router specific logs. I wanted a cleaner Bro log for just the firewall accept and drop logs. The default syslog.log had the info, and I parsed it out with bro-cut, grep, and awk, but that didn't give me any practice with bro scripting.

So I wrote a new module called RouterFW that would be a new namespace for my router firewall log parsing script. This script would in essence react to the syslog_message event in bro, parse out the data I wanted, and then log it to the RouterFW.log log.

Bro scripting tips:

Mostly, I wrote this script based on the exercises at try.bro.org and the Bro documentation.

I wrote the script as just one file called RouterFW.bro, but should probably go back and break it out into the recommended convention of having a directory called RouterFW named after the module, and in that having a __load__.bro file that just loads the script (renamed main.bro) and any other script files in the directory.

With Bro, you can test a script by running it at the command line with just

bro -C -i eth0 /pathto/script.bro

But remember that when running it this way it writes the log files to the directory you are in when running the command.

Once you have the script working and want it in production, you need to put the script in the  /opt/bro/share/bro/site/ directory. Here is where you can create a directory named the same as your module, put your script in it and name it main.bro (to follow convention).  Create a file called  __load__.bro that has load statements for each of the files in the directory, including main.bro.

You'll also need to add a load statement to /opt/bro/share/bro/site/local.bro that loads your new module directory.

In order for Bro to use the script in normal production, you'll need to run the following commands:

  1. Check for errors with:   broctl check
  2. Tell Bro to use the script with:   broctl install
  3. Restart Bro with:   broctl restart

You can verify your script loaded by grepping for your script's module name in the loaded scripts log:  /nsm/bro/logs/current/loaded_scripts.log

The Script:

So, here's a breakdown of the script:

The first few lines load any necessary base functionality that will be used in my module script. Since I'm using the syslog_message event, I'll need to load syslog. Since I'll be using items from the connection event, I've loaded conn also. These will most likely already be loaded by Bro running in production but it is recommended to add it to the script in case it is run by itself.

@load base/protocols/syslog @load base/protocols/conn


The next line names the module namespace.

module RouterFW;

Next, I identify any record, variable, or function that needs to be accessed from other scripts. Basically I'm just creating the ID for the new stream and defining the record type for the log file.

export {
#Create an ID for the new stream.
redef enum Log::ID += { LOG };
#Define the record type that will contain the data to log.
type Info: record {
syslog_ts: time &log;
syslog_uid: string &log;
fw_ts: string &log;
packet_src: addr &log;
packet_dest: addr &log;
packet_dport: string &log;
packet_proto: string &log;
action: string &log;
};
}

Next is tying it to the bro_init() so the stream is created when bro starts up.

event bro_init() &priority=5
{
#Create the stream. this adds a default filter automatically
Log::create_stream(RouterFW::LOG, [$columns=Info, $path="RouterFW"]);
}

This next part is cool. It will add the RouterRW record to the connection record so it can be accessed by other scripts in Bro using the $ notation. For example c$routerfw$action would return the action drop or accept.

#add a new field to the connection record so that data is accessible in variety of event handlers
redef record connection += {
routerfw: Info &optional;
};

The last part is the logic that parses out the data from the syslog message field of the syslog record. This field is a string and holds whatever the router put in the log message.  This is specific to my router and took a bit of playing around to be sure I was getting the right fields. 

#use syslog_message event as defined in Bro_Syslog.events.bif.bro
event syslog_message(c:connection; facility:count; severity:count; msg: string)
{
#split message field to get data we want
local messagedata = split_string(c$syslog$message, / /);

#concatenate back together the time and date from the log message itself
local fw_time = cat_sep(" ", "-", messagedata[0], messagedata[1], messagedata[2]);

#log any ACCEPT or DROP message from the firewall
if (( "ACCEPT" in msg ) || ("DROP" in msg))
{
local action = messagedata[4];
for (i in messagedata)
{
if ("SRC=" in messagedata[i])
{
local src_ip = to_addr((split_string(messagedata[i], /=/ ))[1]);
};

if ("DST=" in messagedata[i])
{
local dst_ip = to_addr((split_string(messagedata[i], /=/))[1]);
};
if ("DPT=" in messagedata[i])
{
local dst_p = (split_string(messagedata[i], /=/))[1];
};

if ("PROTO=" in messagedata[i])
{
local proto = (split_string(messagedata[i], /=/))[1];
dst_p = "No Port";
};

};

#Log format
local rec: RouterFW::Info = [$syslog_ts=c$syslog$ts, $syslog_uid=c$uid, $fw_ts=fw_time,
$packet_src=src_ip, $packet_dest=dst_ip, $packet_dport=dst_p, $packet_proto=proto,
$action=action];

c$routerfw = rec;

Log::write(RouterFW::LOG, rec);
};

}

The Log Files:

I was confused at first because the order of the fields in the log didn't match the order I used in the statement
starting with "local rec: RouterFW::Info =". I found the order is determined by the export statement at the top
of the script.
The log file looks like this.




So my first script worked out well enough that I duplicated it and reworked the logic to be a RouterDHCP
module that creates logs of DHCP ACKs.
That log looks like this:




All in all, not the most necessary scripts, but I learned a bunch figuring out how to make them work.


Sunday, January 8, 2017

Security Onion Cheatsheet

I've been spending some time learning Security Onion and jotting down paths and commands that I can't seem to memorize. After a little Google searching I noticed there aren't many quick reference cheatsheets for Security Onion. So, I decided to try to put one together.

Mostly, this is information from the Security Onion website reformatted to fit on a few pages with just the paths or commands, and less of the instructions.  Other bits are from helpful presentations, such as Eric Conrad's C2 Phone Home Leveraging Security Onion, Jon Schipp's Working with Bro Logs: Queries By Example , or from my own experimenting.

Check it out: Security Onion Cheatsheet

Saturday, December 10, 2016

My Own Cloud - Learning DevStack

Setting up DevStack on an old Dell Latitude D820 (yep, old but this is just for learning).

Most of the instructions I got from http://docs.openstack.org/developer/devstack/Introduction to OpenStack Neutron, and openstack demo and then modified to fit my needs.

1) Install Ubuntu - Downloaded Ubuntu 16.04.1
    ubuntu-16.04.1-server-i386.iso from http://releases.ubuntu.com/16.04

    2) Used my Zalman ZM-VE300 HDD/Virtual ODD drive to boot the Dell and run the install.
    If you aren't familiar with this enclosure/tool, check it out at http://www.zalman.com/contents/products/view.html?no=212
    It lets you drop ISOs on the hard drive you enclose in it, and then boot devices from them. It's like having a case full of bootable DVDs in one small device.
    3) I pretty much used all the defaults for the ubuntu install except added the OpenSSH server.

    4)  I had some video issues that may just be failing hardware, but I ended up modifying the grub menu so that it used the BIOS video and not the drivers until I could install the correct nVidia driver.
    I rebooted and at the grub menu with "ubuntu" selected, I hit the 'e' key.
    In the config that appears, I found the line that starts with "linux" and added to the end of the line "nomodeset xforcevesa
    5) Add Stack User - These commands create a non-root user with sudo enabled to run Devstack.
    $ adduser stack
    $ echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
    $ su stack
    6) Download DevStack - The devstack repo contains a script that installs OpenStack and templates for configuration files
    $ cd ~$ git clone https://git.openstack.org/openstack-dev/devstack$ cd devstack
    7) Create a local.conf - Create a local.conf file with 4 passwords preset at the root of the devstack git repo. Also add some Neutron networking info and a fix for a VNC bug.

    [[local|localrc]]
    HOST_IP=10.41.0.12
    SERVICE_HOST=10.41.0.12
    MYSQL_HOST=10.41.0.12
    RABBIT_HOST=10.41.0.12
    GLANCE_HOSTPORT=10.41.0.12:9292
    ADMIN_PASSWORD=secret
    DATABASE_PASSWORD=$ADMIN_PASSWORD
    RABBIT_PASSWORD=$ADMIN_PASSWORD
    SERVICE_PASSWORD=
    $ADMIN_PASSWORD
    NOVNC_BRANCH=v0.6.0

    ## Neutron options
    Q_USE_SECGROUP=True
    FLOATING_RANGE="10.41.0.0/24"
    IPV4_ADDRS_SAFE_TO_USE="10.0.0.0/22"
    Q_FLOATING_ALLOCATION_POOL=start=10.41.0.250,end=10.41.0.254
    PUBLIC_NETWORK_GATEWAY="10.41.0.1"
    PUBLIC_INTERFACE=enp9s0


    ## Open vSwitch provider networking configuration
    Q_USE_PROVIDERNET_FOR_PUBLIC=True
    OVS_PHYSICAL_BRIDGE=br-ex
    PUBLIC_BRIDGE=br-ex
    OVS_BRIDGE_MAPPINGS=public:br-ex

    8) Start the install - run the script and it pretty much does everything. Took about 30 min or so on this old Dell.
    $ ./stack.sh
    9) Login - At this point, the Horizon web interface should now be available at http://10.41.0.12 and looks like this:

     User Name: admin     Password: secret

    10) Change Password - Login as admin and in the menu on the left expand 
    Settings -> Change Password.  Set the password to something better than secret.

    11) Add images - The basic install does contain a Cirros image, but it is 64 bit and my old dell is 32 bit.

    a) Download the image from http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-i386-disk.img. (qcow2 format works well for Devstack)
    b) Log into Horizon as admin. You can do it from other accounts, but if you get "TypeError: Cannot read property 'data' of undefined" it is because your account doesn't have admin rights.
    c) Expand Admin -> System -> Images click Create Image.
    d) Give it a name and browse to the downloaded file location. Set the format to "QCOW2 - QEMU Emulator".  Leave Visibility as Public and Protected as No.  I didn't set any additional settings.
    e) Click Create Image.


    12) Create a User and Key Pair
    a) In Horizon, expand Identity -> Users -> Create User
    - Give it username, email, password and select a primary project
    b) Expand Identity -> Projects -> manage members of the primary project for the new user
    -Add the user and give it roles. (at least member)
    c) Log out and log in as the new user
    d) Expand Project -> Computer -> Access & Security and click the key pairs tab.
    - Click Create Key Pair and give the pair a name. It will download it to the machine you are running the browser from and can be used to ssh into systems it will be used on.

    13) Create virtual networks - (like vlans - logical slice to separate broadcast domains)
    a) Expand Project -> Network -> Networks and click Create Network
    b) Name it, leave Admin State UP and Create Subnet checked. Click Next.
    c) Name the subnet and enter the Network Address as a CIDR address ex: 192.168.1.0/24
    d) Leave the gateway blank but do not check disable gateway (It will become the .1) Click Next.
    e) Check enable DHCP  and leave the rest blank. Click Create.

    14) Create Neutron Router
    a) Expand Project -> Network -> Networks -> Routers and click Create Router.
    b) Name it, leave Admin State UP and do not select an External Network yet. Click Create Router.
    c) Click on the new router, the Interfaces tab and then Add Interface. In the drop down, select the desired network and click Submit. Do this for each network.
    d) Expand Project -> Network -> Networks and verify the "public" network is the external network the devstack box is on (ex. 10.41.0.0/24 for home network)
    e) Set up source routing (NAT for outgoing traffic)
    1. On routers page, click Add Gateway to the router and choose the public network.

    15) Create a Security Group
    Security groups are sets of IP filter rules that are applied to the network settings for the VM. After the security group is created, you can add rules to the security group.  They are like ACLs.

    a) Expand Project -> Compute -> Access & Security and click the Security Group tab. 
    b) Click Create Security Group
    1. Give it a name and description
    2. To the right of the new group, click Manage Rules
    3. Click Add Rule and create rules as necessary with protocol, ports, and IPs. 
    Example allow incoming SSH traffic from home network:
    •  Custom TCP Rule | Ingress | Port | 22 | CIDR | 10.41.0.0/24

    16) Deploy an instance (VM)

    a) Expand Project -> Compute -> Instances
    1. Click Launch Instance and give it a name. Click Next.
    2. Choose Image as the Boot Source. At the bottom under Available, find the name of the 32 bit Cirros image you created in step 11 and click the up arrow to the right of it. Click Next.
    3. Click the up arrow to the right of m1.tiny and Click Next.
    4. Click the up arrow to the right of the network you want the instance to live on and Click Next.
    5. Leave Network Ports the same and click Next.
    6. Click the up arrow to the right of the Security Group you created and click Next.
    7. If you created more than one key pair, click the up arrow to the right of the one you want to use. Click Next.
    8. Click Next through the rest of the screens and then click Launch Instance.
    9. Click Launch and wait for it to spawn.
    10. It should get an IP address for the internal subnet selected.


    17) Set up Floating IPs for each instance that needs access from outside in. (one-to-one IP pairing internal-external)
    a) Expand Project -> Compute -> Instances and from the dropdown menu to the right of the instance, choose Associate Floating IP.
    b) Click the plus [+] and it will grab from the pool of public IPs (designated in the local.conf)
    c) Click Allocate IP and then Associate. This should now show both IPs in the instance list.

    18) Test Connectivity and SSH - You should now be able to use the key you download when creating the key pairs and ssh into the machines you used it on.