Skip to main content

Posts about logging

Graylog Syslog Grok Extractor

I have several apps in my k8s cluster which run behind a lighttpd web server where I wanted the access logs to be sent to graylog. The setup of lighttpd is relatively basic:

# /etc/lighttpd/lighttpd.conf

var.basedir  = "/var/www/localhost"
var.logdir   = "/var/log/lighttpd"
var.statedir = "/var/lib/lighttpd"

server.modules = (

include "mime-types.conf"

server.username      = "lighttpd"
server.groupname     = "lighttpd"

server.port          = 80
server.document-root = var.basedir + "/htdocs"      = "/run/"

server.indexfiles    = ("index.html")

server.errorlog-use-syslog = "enable"
accesslog.use-syslog       = "enable"

Next we need to tell rsyslog to send logs to graylog:

# /etc/rsyslog.d/dxc-graylog.conf

*.* @graylog-udp.graylog.svc.cluster.local:5410;RSYSLOG_SyslogProtocol23Format

Note that you will need to change the URL and port to match the graylog syslog URL and port you have. Also notice that we are using RSYSLOG_SyslogProtocol23Format format. This results in a log message with a message value similar to the following in graylog: - [28/Oct/2020:06:28:06 -0400] "GET / HTTP/1.1" 200 6341 "-" "kube-probe/1.16"

This should all work; however, we would like to have the message parsed - enter the grok pattern! In Graylog, go to your Syslog Input and click Manage Extractors. Add an Extractor with the following info:

  • select extractor type for the message field with a Grok Pattern
  • enter grok pattern:
%{SYSLOGHOST:host} %{SYSLOGHOST:referer} - \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}
  • Condition: Always try to extract
  • Extraction strategy: Copy
  • Extractor title: RFC5424
  • click the Create extractor

That should be all that is needed, go to your syslog stream and check (make sure you pick a message that came in after you created the extractor) to ensure the new fields are being created!