Sunday, August 9, 2015

Connecting IBM QRadar SIEM with a Java client for event collection

What is IBM Qradar ?

IBM® Security QRadar® SIEM consolidates log source event data from thousands of devices endpoints and applications distributed throughout a network. It performs immediate normalization and correlation activities on raw data to distinguish real threats from false positives.

What is Syslog ?

In computing, syslog is a widely used standard for message logging. It permits separation of the software that generates messages, the system that stores them, and the software that reports and analyzes them.

Computer system designers may use syslog for system management and security auditing as well as general informational, analysis, and debugging messages. A wide variety of devices, such as printers and routers, and message receivers across many platforms use the syslog standard. This permits the consolidation of logging data from different types of systems in a central repository. Implementations of syslog exist for many operating systems.



Each message is labeled with a facility code, and assigned a severity label. The facility code indicates the software type of the application that generated the message.


The destination of messages may be directed to various destinations, tuned by facility and severity, including console), files, remote syslog servers, or relays.
Most implementations provide a command line utility, often called logger, as well as a link library, to send messages to the log.

How can we submit events to QRadar?


The simplest way to send events to QRadar is using LEEF format to Syslog .



LEEF format formula:



LEEF:Version|Vendor|Product|Version|EventID|Key1=Value1<tab>Key2=Value2<tab>Key3=Value3<tab>...<tab>KeyN=ValueN

LEEF Event format example:

Jan 18 11:07:53 192.168.1.1 LEEF:1.0|QRadar|QRM|1.0|NEW_PORT_DISCOVERD|src=172.5.6.67 dst=172.50.123.1 sev=5 cat=anomaly msg=this is a message

More about LEEF format here:
LEEF Format in IBM QRadar

Definition of a "log source" in IBM QRadar

Through the Admin tab , go to "log sources" and define a new Log Source as pictured by the instructions below:























Sending events to QRadar using apache log4j 



public class QRadarSink {



private Logger logger = Logger.getLogger(QRadarSink.class.getName());

       private static final String LOGGER_NAME = "SYSLOG";

       private static final String PATTERN_PREFIX = "%d{MMM dd HH:mm:ss}  ";
       private static final String PATTERN_POSTFIX = "      %m%n";

       public QRadarSink(String destinationHost, String port)
                    throws UnknownHostException {

             String hostname = InetAddress.getLocalHost().getHostName();
PatternLayout layout = new PatternLayout(PATTERN_PREFIX + hostname + PATTERN_POSTFIX);
             SyslogAppender syslog = new SyslogAppender();
             syslog.setName(LOGGER_NAME);
             syslog.setSyslogHost(destinationHost + ":" + port);
             syslog.setFacilityPrinting(false);
             syslog.setHeader(false);
             syslog.setLayout(layout);
             syslog.activateOptions();
             logger.addAppender(syslog);
       }

       public void pushEventToQradar(String message) {
             logger.error(message);
       }
      
       public static void main(String[] args) throws Exception {
             QRadarSink sink = new QRadarSink("9.148.5.113","514");     
              sink.pushEventToQradar("LEEF:1.0|InfoSphereStreams|PedictiveBlacklisting|1.0|NEW_EVENT_DISCOVERD|src=206.64.49.42       dst=172.50.123.1    devTime=Jul 20 2015 14:05:20     devTimeFormat=MMM dd yyyy HH:mm:ss proto=4      sev=9         filterMatched=nonMatched");
       }
}


QRadar Log Activity





5 comments:

  1. What is the "SyslogAppender" class you referring to?

    ReplyDelete
  2. Hi, I need to write LEEF audits to syslog and have been searching for a Java client to do this. So if I use Log4J, there aren't any problems with QRadar parsing the messages? I was concerned about extra or different header, or preceding, info before the LEEF statement. Not sure how forgiving QRadar is with parsing.

    ReplyDelete
  3. Hi Scott. I tried this on 7.2.4 and it worked perfectly. QRadar was able to parse all standard headers successfully

    ReplyDelete
  4. HI laser,

    First of all great article. With the LEEF format are the key/value pairs in the payload with the predefined keys automatically discovered & mapped to the normalized fields ? like severity, source, destination and category . Or are there some mandatory fields to send over ?
    Or do you have a LSX or DSM loaded ?

    my message looks like (k/v are tab separated) (done with developed logstash leef codec ) and the log source is recognized from the syslog header, but k/v are not mapped / discovered.

    LEEF:1.0|Elastic|ELK|1.0|ELK|src=10.10.10.1 dst=10.10.10.2 cat=firewall sev=4 log=appliance

    Is the devTime and devTimeFormat mandatory for correct parsing ?

    ReplyDelete