Monitoring RSA Netwitness

I have been working and supporting RSA Netwitness for over a year now. And by now I have build a reasonable lab environment. But I was thinking of how to monitor the RSA Netwitness lab with my Icinga2 monitoring appliance (Raspberry PI).

I decided to see how much I could get out of the REST API without spending days on programming. And a bit of tinkering on the REST API webinterface allowed my to build something as you can read here.

For example:
You can get the generic appliance status from a URL like https://10.1.2.3:50106/sys/stats/service.status
After logging in you get a simple “Ready” if everything is working.

So now you need to translate that to a set of options to test automatically with check_http. And I came up with a result in the end with the commandline:

/usr/lib/nagios/plugins/check_http -I 10.1.2.3 -p 50106 -a "admin:netwitness" -u "/sys/stats/service.status?force-content-type=text/plain" -r "Ready"

Which gives me a response of

HTTP OK: HTTP/1.1 200 OK - 188 bytes in 0.004 second response time |time=0.003878s;;;0.000000;10.000000 size=188B;;;0

So for Icinga I wrote a Services section (add to /etc/icinga2/conf.d/services.conf) like this:

/*
        RSA Netwitness
*/
object CheckCommand "RSAappliance" {
        command = [ PluginDir + "/check_http" ]
        arguments = {
                "-H" = "$addr$"
                "-p" = "$applianceport$"
                "-a" = "$apiauth$"
                "-u" = "/sys/stats/service.status?force-content-type=text/plain"
                "-r" = "Ready"
                }
        vars.addr = "$address$"
        }
apply Service "RSA Appliance" {
        check_command = "RSAappliance"
        vars.applianceport = 50106
        vars.apiauth = "admin:netwitness"
        assign where host.address && host.vars.os == "RSA"
        }
object CheckCommand "RSAbroker" {
        command = [ PluginDir + "/check_http" ]
        arguments = {
                "-H" = "$addr$"
                "-p" = "$brokerport$"
                "-a" = "$apiauth$"
                "-u" = "/broker/stats/status?force-content-type=text/plain"
                "-r" = "started"
                }
        vars.addr = "$address$"
        }
apply Service "RSA broker" {
        check_command = "RSAbroker"
        vars.brokerport = 50103
        vars.apiauth = "admin:netwitness"
        assign where host.address && host.vars.os == "RSA" && host.vars.broker == "yes"
        }
object CheckCommand "RSAconcentrator" {
        command = [ PluginDir + "/check_http" ]
        arguments = {
                "-H" = "$addr$"
                "-p" = "$concentratorport$"
                "-a" = "$apiauth$"
                "-u" = "/concentrator/stats/status?force-content-type=text/plain"
                "-r" = "started"
                }
        vars.addr = "$address$"
        }
apply Service "RSA concentrator" {
        check_command = "RSAconcentrator"
        vars.concentratorport = 50105
        vars.apiauth = "admin:netwitness"
        assign where host.address && host.vars.os == "RSA" && host.vars.concentrator == "yes"
        }
object CheckCommand "RSAdecoder" {
        command = [ PluginDir + "/check_http" ]
        arguments = {
                "-H" = "$addr$"
                "-p" = "$decoderport$"
                "-a" = "$apiauth$"
                "-u" = "/decoder/stats/capture.status?force-content-type=text/plain"
                "-r" = "started"
                }
        vars.addr = "$address$"
        }
apply Service "RSA decoder" {
        check_command = "RSAdecoder"
        vars.apiauth = "admin:netwitness"
        assign where host.address && host.vars.os == "RSA" && host.vars.decoder == "yes"
        }

And then you can very simply define you RSA appliances by adding just a few variables to you hosts.
So in my /etc/icinga2/conf.d/hosts i have entries like:

object Host "RSA Netwitness HOST" {
        address = "10.1.2.3"
        check_command = "hostalive"
        vars.os = "RSA"
        vars.broker = "yes"
        vars.geolocation = "51.997540,4.384650"
        vars.http_vhosts["Console"] = {
                http_uri = "/"
                }
        vars.http_ssl = "1"
        }
object Host "RSA Netwitness PKT (Hybrid)" {
        address = "10.1.2.4"
        check_command = "hostalive"
        vars.os = "RSA"
        vars.concentrator = "yes"
        vars.decoder = "yes"
        vars.decoderport = 50104
        vars.geolocation = "51.997540,4.384650"
        }
object Host "RSA Netwitness LOG (Hybrid)" {
        address = "10.1.2.5"
        check_command = "hostalive"
        vars.os = "RSA"
        vars.concentrator = "yes"
        vars.decoder = "yes"
        vars.decoderport = 50102
        vars.geolocation = "51.997540,4.384650"
        }

The variables allow me to just define a host and add checks to them easily.

In real applications I would imagine you propably want to add more checks as you also want to be sure you are ingesting the data and you also might want to use some results for their performance data. For example you might want to count the number of log events ingested so you can graph it and perhaps also detect anomalies there. However that is not part of todays exercise and left for another rainy day.