Zeek¶
Zeek is formerly known as Bro. From https://www.zeek.org/:
Zeek is a powerful network analysis framework that is much different from the typical IDS you may know. (Zeek is the new name for the long-established Bro system. Note that parts of the system retain the “Bro” name, and it also often appears in the documentation and distributions.)
Zeek logs are sent to Elasticsearch for parsing and storage and can then be found in Dashboards, Hunt, and Kibana. Here’s an example of Zeek conn (connection) logs in Hunt:

Community ID¶
Security Onion enables Zeek’s built-in support for Community ID.
Packet Loss and Capture Loss¶
Zeek reports both packet loss and capture loss and you can find graphs of these in Grafana. If Zeek reports packet loss, then you most likely need to adjust the number of Zeek workers as shown below or filter out traffic using BPF. If Zeek is reporting capture loss but no packet loss, this usually means that the capture loss is happening upstream in the tap or span port itself.
Performance¶
Zeek uses AF-PACKET so that you can spin up multiple Zeek workers to handle more traffic.
To change the number of AF-PACKET workers for Zeek:
Stop Zeek:
sudo so-zeek-stop
Edit
/opt/so/saltstack/local/pillar/minions/$SENSORNAME_$ROLE.sls
and change thezeek_lbprocs
variable to the desired number of cores.Start Zeek:
sudo so-zeek-start
For best performance, Zeek should be pinned to specific CPUs. In most cases, you’ll want to pin sniffing processes to a CPU in the same Non-Uniform Memory Access (NUMA) domain that your sniffing NIC is bound to. Accessing a CPU in the same NUMA domain is faster than across a NUMA domain.
Note
For more information about determining NUMA domains using lscpu
and lstopo
, please see https://github.com/brokenscripts/cpu_pinning.
To pin Zeek workers to specific CPUs:
Stop sensor processes:
sudo so-zeek-stop
Edit
/opt/so/saltstack/local/pillar/minions/$SENSORNAME_$ROLE.sls
and add the following undersensor:
zeek_pins: - <cpu_1> - <cpu_2> - <cpu_3>
Note: To avoid inconsistent Zeek workers being allocated, ensure
zeek_lbprocs
is removed from undersensor:
or is equivalent to the number of cpu cores being pinned.Start sensor processes:
sudo so-zeek-start
Syslog¶
To forward Zeek logs to an external syslog collector, please see the Syslog Output section.
Logs¶
Zeek logs are stored in /nsm/zeek/logs
. They are collected by Filebeat, parsed by and stored in Elasticsearch, and viewable in Dashboards, Hunt, and Kibana.
We configure Zeek to output logs in JSON format. If you need to parse those JSON logs from the command line, you can use jq.
If you want to specify what Zeek logs are ingested, you can use so-zeek-logs.
Zeek monitors your network traffic and creates logs, such as:
conn.log¶
- TCP/UDP/ICMP connections
- For more information, see:
https://docs.zeek.org/en/latest/scripts/base/protocols/conn/main.zeek.html#type-Conn::Info
dns.log¶
- DNS activity
- For more information, see:
https://docs.zeek.org/en/latest/scripts/base/protocols/dns/main.zeek.html#type-DNS::Info
ftp.log¶
- FTP activity
- For more information, see:
https://docs.zeek.org/en/latest/scripts/base/protocols/ftp/info.zeek.html#type-FTP::Info
http.log¶
- HTTP requests and replies
- For more information, see:
https://docs.zeek.org/en/latest/scripts/base/protocols/http/main.zeek.html#type-HTTP::Info
ssl.log¶
- SSL/TLS handshake info
- For more information, see:
https://docs.zeek.org/en/latest/scripts/base/protocols/ssl/main.zeek.html#type-SSL::Info
notice.log¶
- Zeek notices
- For more information, see:
https://docs.zeek.org/en/latest/scripts/base/frameworks/notice/main.zeek.html#type-Notice::Info
Other Zeek logs¶
Zeek also provides other logs by default and you can read more about them at https://docs.zeek.org/en/latest/script-reference/log-files.html.
Security Onion includes protocol analyzers for STUN, TDS, and Wireguard traffic and several different ICS/SCADA protocols. By default, these analyzers are enabled and will log to the filesystem. For new installations, these logs are automatically ingested into Elasticsearch. If you upgrade an existing deployment and would like to ingest these logs, you can enable them via so-zeek-logs.
Security Onion also includes MITRE BZAR scripts and you can read more about them at https://github.com/mitre-attack/bzar. Please note that the MITRE BZAR scripts are disabled by default. If you would like to enable them, you can add bzar
to your global.sls
as shown in the Custom Scripts section below. Once enabled, you can then check for BZAR detections by going to Dashboards and selecting the Zeek Notice dashboard.
As you can see, Zeek log data can provide a wealth of information to the analyst, all easily accessible through Dashboards, Hunt, or Kibana.
VLAN tags¶
If you want to include VLAN tags in your conn.log, then you can enable the protocols/conn/vlan-logging
script (already included in Zeek but disabled by default) in your Zeek configuration as shown in the Custom Scripts section below.
Intel¶
You can add your own intel to /opt/so/saltstack/local/salt/zeek/policy/intel/intel.dat
on the manager and it will automatically replicate to all forward nodes. If the /opt/so/saltstack/local/salt/zeek/policy/intel/
directory is empty, you can copy the default files (both intel.dat
and __load__.zeek
) from /opt/so/saltstack/default/salt/zeek/policy/intel/
as follows:
sudo cp /opt/so/saltstack/default/salt/zeek/policy/intel/* /opt/so/saltstack/local/salt/zeek/policy/intel/
Please note that Zeek is very strict about the format of intel.dat
. When editing this file, please follow these guidelines:
- no leading spaces or lines
- separate fields with a single literal tab character
- no trailing spaces or lines
The default intel.dat
file follows these guidelines so you can reference it as an example of the proper format.
When finished editing intel.dat
, run sudo salt $SENSORNAME_$ROLE state.highstate
to sync /opt/so/saltstack/local/salt/zeek/policy/intel/
to /opt/so/conf/zeek/policy/intel/
. If you have a distributed deployment with separate forward nodes, it may take up to 15 minutes for intel to sync to the forward nodes.
If you experience an error, or do not notice /nsm/zeek/logs/current/intel.log
being generated, try having a look in /nsm/zeek/logs/current/reporter.log
for clues. You may also want to restart Zeek after making changes by running sudo so-zeek-restart
.
Custom Scripts¶
Custom scripts can be added to /opt/so/saltstack/local/salt/zeek/policy/custom/<$custom-module>
on the manager. The custom folder is mapped to Zeek through Docker on the minions. Once the script module is created, the configuration for local.zeek
will need to be updated. In Security Onion 2, this configuration is abstracted into a Salt pillar. For example, we would copy the following lines from /opt/so/saltstack/default/pillar/zeek/init.sls
into the global.sls
file, replacing $module-name
on the last line with the actual module name:
zeek:
local:
'@load':
- misc/loaded-scripts
- tuning/defaults
- misc/capture-loss
- misc/stats
- frameworks/software/vulnerable
- frameworks/software/version-changes
- protocols/ftp/software
- protocols/smtp/software
- protocols/ssh/software
- protocols/http/software
- protocols/dns/detect-external-names
- protocols/ftp/detect
- protocols/conn/known-hosts
- protocols/conn/known-services
- protocols/ssl/known-certs
- protocols/ssl/validate-certs
- protocols/ssl/log-hostcerts-only
- protocols/ssh/geo-data
- protocols/ssh/detect-bruteforcing
- protocols/ssh/interesting-hostnames
- protocols/http/detect-sqli
- frameworks/files/hash-all-files
- frameworks/files/detect-MHR
- policy/frameworks/notice/extend-email/hostnames
- ja3
- hassh
- intel
- cve-2020-0601
- securityonion/bpfconf
- securityonion/communityid
- securityonion/file-extraction
- oui-logging
- icsnpp-modbus
- icsnpp-dnp3
- icsnpp-bacnet
- icsnpp-ethercat
- icsnpp-enip
- icsnpp-opcua-binary
- icsnpp-bsap
- icsnpp-s7comm
- zeek-plugin-tds
- zeek-plugin-profinet
- zeek-spicy-wireguard
- zeek-spicy-stun
- custom/$module-name
One the configuration has been updated, Zeek can be restarted with sudo so-zeek-restart
on applicable nodes to pick up the changes. Finally, /nsm/zeek/logs/current/loaded_scripts.log
can be checked to ensure the new module has been loaded. For example:
grep mynewmodule /nsm/zeek/logs/current/loaded_scripts.log
Custom Script Example: log4j¶
Corelight has developed a Zeek package to detect log4j exploitation attempts and it can be found at https://github.com/corelight/cve-2021-44228. This package contains Zeek scripts which can easily be loaded into your Security Onion deployment.
First, we need to make sure that the custom
directory exists on the manager:
sudo mkdir -p /opt/so/saltstack/local/salt/zeek/policy/custom/
Next, download the Zeek package to a temporary location:
git clone https://github.com/corelight/cve-2021-44228.git
Now we need to move the Zeek scripts to the Zeek custom
directory:
sudo mv cve-2021-44228/scripts /opt/so/saltstack/local/salt/zeek/policy/custom/cve-2021-44228
Next, we need to configure Zeek to load the new scripts. If /opt/so/saltstack/local/pillar/global.sls
does not already contain a zeek:
section, then copy and paste the following at the end of the file (be careful when pasting to respect yaml indentation):
zeek:
local:
'@load':
- misc/loaded-scripts
- tuning/defaults
- misc/capture-loss
- misc/stats
- frameworks/software/vulnerable
- frameworks/software/version-changes
- protocols/ftp/software
- protocols/smtp/software
- protocols/ssh/software
- protocols/http/software
- protocols/dns/detect-external-names
- protocols/ftp/detect
- protocols/conn/known-hosts
- protocols/conn/known-services
- protocols/ssl/known-certs
- protocols/ssl/validate-certs
- protocols/ssl/log-hostcerts-only
- protocols/ssh/geo-data
- protocols/ssh/detect-bruteforcing
- protocols/ssh/interesting-hostnames
- protocols/http/detect-sqli
- frameworks/files/hash-all-files
- frameworks/files/detect-MHR
- policy/frameworks/notice/extend-email/hostnames
- ja3
- hassh
- intel
- cve-2020-0601
- securityonion/bpfconf
- securityonion/communityid
- securityonion/file-extraction
- custom/cve-2021-44228
Within 15 minutes, Salt should automatically restart Zeek where necessary. If you don’t want to wait, you can manually restart Zeek with the following command. If you have a distributed deployment, you could run this command on each sensor manually or use Salt to run the command across all sensors at once:
sudo so-zeek-restart
Modifying base protocol scripts¶
If you need to modify base protocol scripts, you can do so as follows. In this example, we are modifying the default ports that Zeek considers for the MySQL analyzer:
const ports = { 11434/tcp, 13306/tcp };
event zeek_init() &priority=5
{
Analyzer::register_for_ports(Analyzer::ANALYZER_MYSQL, ports);
}
Configuration¶
You can use Salt to manage Zeek’s local.zeek
, node.cfg
and zeekctl.cfg
:
local.zeek
: The allowed options for this file are @load
, @load-sigs
and redef
. An example of configuring this pillar can be seen below.
node.cfg
: The pillar items to modify this file are located under the sensor pillar in the minion pillar file. The options that can be customized in the file include: interface
, lb_procs
, pin_cpus
, and af_packet_buffer_size
.
zeekctl.cfg
: An example of customizing this can be seen below. The allowed options can be seen in https://github.com/Security-Onion-Solutions/securityonion/blob/master/salt/zeek/files/zeekctl.cfg.jinja.
Here is an example of how we would modify local.zeek
. We can see the default pillar assignments used for local.zeek
in /opt/so/saltstack/default/pillar/zeek/init.sls
. This file should never be modified as it could be updated in the future and any modification made would be overwritten. The global or minion pillar files should be used for making changes as they are stored in /opt/so/saltstack/local/
, and that directory isn’t overwritten during a Security Onion code update.
zeek:
zeekctl:
MailTo: root@localhost
MailConnectionSummary: 1
MinDiskSpace: 5
MailHostUpDown: 1
LogRotationInterval: 3600
LogExpireInterval: 0
StatsLogEnable: 1
StatsLogExpireInterval: 0
StatusCmdShowAll: 0
CrashExpireInterval: 0
SitePolicyScripts: local.zeek
LogDir: /nsm/zeek/logs
SpoolDir: /nsm/zeek/spool
CfgDir: /opt/zeek/etc
CompressLogs: 1
local:
'@load':
- misc/loaded-scripts
- tuning/defaults
- misc/capture-loss
- misc/stats
- frameworks/software/vulnerable
- frameworks/software/version-changes
- protocols/ftp/software
- protocols/smtp/software
- protocols/ssh/software
- protocols/http/software
- protocols/dns/detect-external-names
- protocols/ftp/detect
- protocols/conn/known-hosts
- protocols/conn/known-services
- protocols/ssl/known-certs
- protocols/ssl/validate-certs
- protocols/ssl/log-hostcerts-only
- protocols/ssh/geo-data
- protocols/ssh/detect-bruteforcing
- protocols/ssh/interesting-hostnames
- protocols/http/detect-sqli
- frameworks/files/hash-all-files
- frameworks/files/detect-MHR
- policy/frameworks/notice/extend-email/hostnames
- ja3
- hassh
- intel
- cve-2020-0601
- securityonion/bpfconf
- securityonion/communityid
- securityonion/file-extraction
'@load-sigs':
- frameworks/signatures/detect-windows-shells
redef:
- LogAscii::use_json = T;
- LogAscii::json_timestamps = JSON::TS_ISO8601;
In this file, there are two keys under zeek, zeekctl
and local
. We will be using zeek:local
for this example since we are modifying the zeek.local
file. We will address zeek:zeekctl
in another example where we modify the zeekctl.cfg
file.
Under zeek:local
, there are three keys: @load
, @load-sigs
, and redef
. In the pillar definition, @load
and @load-sigs
are wrapped in quotes due to the @
character. Under each of the keys, there is a list of items that will be added to the local.zeek
file with the appropriate directive of either @load
, @load-sigs
or redef
. In order to modify either of the lists, the entire list must redefined in either the global or minion pillar file.
If we have a node where protocols/ssh/detect-bruteforcing
is generating a lot of noise and we want to tell Zeek to stop loading that script, we would do the following. Since we just want to turn it off for that specific node, we would open /opt/so/saltstack/local/pillar/minions/$SENSORNAME_$ROLE.sls
. At the bottom, we would append the following:
zeek:
local:
'@load':
- misc/loaded-scripts
- tuning/defaults
- misc/capture-loss
- misc/stats
- frameworks/software/vulnerable
- frameworks/software/version-changes
- protocols/ftp/software
- protocols/smtp/software
- protocols/ssh/software
- protocols/http/software
- protocols/dns/detect-external-names
- protocols/ftp/detect
- protocols/conn/known-hosts
- protocols/conn/known-services
- protocols/ssl/known-certs
- protocols/ssl/validate-certs
- protocols/ssl/log-hostcerts-only
- protocols/ssh/geo-data
- protocols/ssh/interesting-hostnames
- protocols/http/detect-sqli
- frameworks/files/hash-all-files
- frameworks/files/detect-MHR
- policy/frameworks/notice/extend-email/hostnames
- ja3
- hassh
- intel
- cve-2020-0601
- securityonion/bpfconf
- securityonion/communityid
- securityonion/file-extraction
We redefined the @load
list in the minion pillar file, but we left out the `protocols/ssh/detect-bruteforcing
. This will override the value defined in the /opt/so/saltstack/default/pillar/zeek/init.sls
and the global pillar file if it is defined there, and prevent the script from being added to the local.zeek
file. If we wanted to add a script to be loaded, then we would add out script to the list. Since we aren’t changing @load-sigs
or redef
, then we do not need to add them here. Once the file is saved, and the node checks in the with manager, the local.zeek
file will be updated and the so-zeek
docker container will be restarted.
Let’s see an example of how we would modify the zeekctl.cfg
file. From the example above, we know that the default pillar values are set for zeek in /opt/so/saltstack/default/pillar/zeek/init.sls
. The default pillar values for zeekctl.cfg
are as follows:
zeek:
zeekctl:
MailTo: root@localhost
MailConnectionSummary: 1
MinDiskSpace: 5
MailHostUpDown: 1
LogRotationInterval: 3600
LogExpireInterval: 0
StatsLogEnable: 1
StatsLogExpireInterval: 0
StatusCmdShowAll: 0
CrashExpireInterval: 0
SitePolicyScripts: local.zeek
LogDir: /nsm/zeek/logs
SpoolDir: /nsm/zeek/spool
CfgDir: /opt/zeek/etc
CompressLogs: 1
For anything not defined here, Zeek will use its own defaults. The options that are allowed to be managed with the pillar can be found at https://github.com/Security-Onion-Solutions/securityonion/blob/master/salt/zeek/files/zeekctl.cfg.jinja.
In order to add or modify an option in zeekctl
, we will need to modify either the global
or minion
pillar file. For example, if we wanted to turn log compression off and change the timeout for Broker communication events to 20 seconds globally, we would add the following to the global pillar file.
zeek:
zeekctl:
compresslogs: 0
commtimeout: 20
Since zeek:zeekctl
is a dictionary with dictionary values, we do not need to redefine the entire pillar here like we did for zeek:local
above. Once the pillar file is saved and the node checks in with the manager, the zeekctl.cfg
file will be updated and the so-zeek container
will be restarted.
Diagnostic Logging¶
Zeek diagnostic logs can be found in /nsm/zeek/logs/
. Look for files like reporter.log
, stats.log
, stderr.log
, and stdout.log
. Depending on what you’re looking for, you may also need to look at the Docker logs for the container:
sudo docker logs so-zeek
Disabling¶
Zeek can be disabled by setting enabled: false
in the zeek
Salt pillar.
If you just want to disable Zeek on a single sensor, then you can edit that sensor’s minion.sls
file. If the file doesn’t already have a zeek
section, then add the following to the end of the file:
zeek:
enabled: false
If you want to disable Zeek globally across all your sensors, then you could add that entry to your global.sls
file.
More Information¶
Note
For more information about Zeek, please see https://www.zeek.org/.