Introduction
Logstash is a powerful tool for centralizing and analyzing logs, which can help to provide and overview of your environment, and to identify issues with your servers. One way to increase the effectiveness of your Logstash setup is to collect important application logs and structure the log data by employing filters, so the data can be readily analyzed and query-able. We will build our filters around “grok” patterns, that will parse the data in the logs into useful bits of information.
This guide is a sequel to the How To Install Elasticsearch, Logstash, and Kibana 4 on Ubuntu 14.04 tutorial, and focuses primarily on adding filters for various common application logs.
Prerequisites
To follow this tutorial, you must have a working Logstash server, and a way to ship your logs to Logstash. If you do not have Logstash set up, here is another tutorial that will get you started: How To Install Elasticsearch, Logstash, and Kibana 4 on Ubuntu 14.04.
Logstash Server Assumptions:
- Logstash is installed in
/opt/logstash - You are receiving logs from Logstash Forwarder on port 5000
- Your Logstash configuration files are located in
/etc/logstash/conf.d - You have an input file named
01-lumberjack-input.conf - You have an output file named
30-lumberjack-output.conf
Continue reading Adding Logstash Filters To Improve Centralized Logging