NBS is a Perl script that uses a Berkeley database to store arbitrary strings of your choosing and will then output when it comes across a string it doesn't have in the database. i.e. a string that is "never before seen."
This can be useful in many situations when looking for anomalies. The man page pdf that downloads with it describes using NBS to detect new access requests to a web server. The web server logs must be parsed and the URL field fed into NBS. If it isn't in the database already, it is added and also written to the output file. Once enough normal traffic has been seen to have established a baseline in the database, entries into the output would be considered anomalies worthy of review.
Since you provide the strings to NBS, it can be useful in many applications. Here are some that came to mind for me:
- AppLocker allowed executables pulled out of event logs
- Bro log data such as file hashes, known hosts, known services, user agent strings
- DNS queries in the DNS log
Installation:
I wanted to test using NBS with Bro logs so I installed it on my Security Onion Virtual machine. (v14.04.5.2)
- Downloaded nbs and extracted it:
- mkdir nbs
- cd nbs
- wget http://www.ranum.com/security/computer_security/code/nbs.tar
- tar -xf nbs.tar
- Installed BerkelyDB prerequisite. The install file that downloads with NBS is a little dated as far as the link for the libraries from what I could tell. These are the steps I took. (requires you to create an Oracle site login)
- wget http://download.oracle.com/otn/berkeley-db/db-6.2.23.NC.tar.gz
- gzip -d db-6.2.23.NC.tar.gz
- tar -xf db-6.2.23.NC.tar
- cd db-6.2.23.NC
- sudo apt-get install g++
- cd build_unix
- ../dist/configure --prefix=/usr/local/berkeleydb --enable-compat185 --enable-cxx --enable-debug_rop --enable-debug_wop (NOTE: the --enable-rpc is no longer supported)
- make
- sudo make install
- sudo su -
- echo '/usr/local/berkeleydb/lib/' >> /etc/ld.so.conf.d/libc.conf
- ldconfig
- apt-get install libdb-dev
- apt-get install libdb++-dev
- Installed NBS. It is an older Perl script and I'm not a Linux guru, so this part might not be perfect. Just using make gave some errors so the following edits had to be made first:
- I edited the both nbs.c and nbspreen.o and made the following changes:
- replaced %d with %ld in the line referenced in the error [fprintf( ...);]
- added #include "errno.h"
- commented out //extern int errno;
- I then ran make in the NBS directory.
- I did keep getting a "count variable unused" error but as far as I could tell NBS was functioning properly.
- Configured NBS for use with Bro dns.log data. Bro is running on this system monitoring traffic.
- Created the database
- sudo ./nbsmk -d ~/nbs/nbs_dnsDB
- Established the baseline for DNS data from the Bro dns.log for the db to monitor
- sudo cat /nsm/bro/logs/current/dns.log | bro-cut query answers | sudo nbs -d nbs_dnsDB -o new_dns_entries
- Viewed the new_dns_entries file. Since this was the first run, all the domain names from the dns.log file are listed.
- cat new_dns_entries
- Did a domain name lookup for a new domain for Bro to catch and add to the dns.log
- nslookup www.rit.edu
- Manually ran the updated dns.log file through NBS again
- sudo cat /nsm/bro/logs/current/dns.log | bro-cut query answers | sudo nbs -d nbs_dnsDB -o new_dns_entries
- Viewed the new_dns_entries file which was overwritten with just this new "never before seen" domain.
Next Steps:
So the next steps would be to set up a cron job to send batches of the data to NBS instead of the manual cat/bro-cut command. Theoretically, you'd want to set this up so that only truly worrisome "never before seen" items are logged. Another mechanism would be necessary to monitor the output file for any activity and either email you or alert in some way.References:
- http://www.ranum.com/security/computer_security/code/
- https://geeksww.com/tutorials/database_management_systems/berkeleydb/installation/installing_berkeleydb_on_ubuntu_linux.php
- http://askubuntu.com/questions/509663/c-preprocessor-lib-cpp-fails-sanity-check
