Ingest¶

Here's an overview of how logs are ingested in various deployment types.

Import¶

Core Pipeline: Elastic Agent [IMPORT Node] --> Elasticsearch Ingest [IMPORT Node]

Logs: Zeek, Suricata

Eval¶

Core Pipeline: Elastic Agent [EVAL Node] --> Elasticsearch Ingest [EVAL Node]

Logs: Zeek, Suricata

Standalone¶

Core Pipeline: Elastic Agent [SA Node] --> Logstash [SA Node] --> Redis [SA Node] <--> Logstash [SA Node] --> Elasticsearch Ingest [SA Node]

Logs: Zeek, Suricata, syslog

Elastic Agent: Elastic Agent [Windows Endpoint] --> Logstash [SA Node] --> Redis [SA Node] <--> Logstash [SA Node] --> Elasticsearch Ingest [SA Node]

Logs: WEL, Sysmon

Fleet Standalone¶

Pipeline: Elastic Agent [Fleet Node] --> Logstash [M | MS] --> Elasticsearch Ingest [S | MS]

Logs: Elastic Agent

Manager (separate search nodes)¶

Core Pipeline: Elastic Agent [Fleet | Sensor] --> Logstash [Manager] --> Redis [Manager]

Logs: Zeek, Suricata, syslog

Elastic Agent: Elastic Agent [Windows Endpoint] --> Logstash [Manager] --> Redis [Manager]

Logs: WEL, Sysmon

Manager Search¶

Core Pipeline: Elastic Agent [Fleet | Sensor] --> Logstash [MS] --> Redis [MS] <--> Logstash [MS] --> Elasticsearch Ingest [MS]

Logs: Zeek, Suricata, syslog

Pipeline: Elastic Agent [MS] --> Logstash [MS] --> Elasticsearch Ingest [MS]

Logs: Local Elastic Agent

Elastic Agent: Elastic Agent [Windows Endpoint] --> Logstash [MS] --> Elasticsearch Ingest [MS]

Logs: WEL, Sysmon

Heavy¶

Pipeline: Elastic Agent [Heavy Node] --> Elasticsearch Ingest [Heavy]

Logs: Zeek, Suricata, syslog

Search¶

Pipeline: Redis [Manager] --> Logstash [Search] --> Elasticsearch Ingest [Search]

Logs: Zeek, Suricata, syslog

Sensor¶

Pipeline: Elastic Agent [Sensor] --> Logstash [M | MS] --> Elasticsearch Ingest [S | MS]

Logs: Zeek, Suricata, syslog