Daily Archives: October 17, 2015

Linux

Initial Server Setup with Ubuntu 14.04

Leave a comment

Introduction

When you first create a new Ubuntu 14.04 server, there are a few configuration steps that you should take early on as part of the basic setup. This will increase the security and usability of your server and will give you a solid foundation for subsequent actions.

Step One — Root Login

To log into your server, you will need to know your server’s public IP address and the password for the “root” user’s account. If you have not already logged into your server, you may want to follow the first tutorial in this series, How to Connect to Your Droplet with SSH, which covers this process in detail.

If you are not already connected to your server, go ahead and log in as the root user using the following command (substitute the highlighted word with your server’s public IP address):

  • ssh root@SERVER_IP_ADDRESS

Complete the login process by accepting the warning about host authenticity, if it appears, then providing your root authentication (password or private key). If it is your first time logging into the server, with a password, you will also be prompted to change the root password.

About Root

The root user is the administrative user in a Linux environment that has very broad privileges. Because of the heightened privileges of the root account, you are actually discouraged from using it on a regular basis. This is because part of the power inherent with the root account is the ability to make very destructive changes, even by accident.

The next step is to set up an alternative user account with a reduced scope of influence for day-to-day work. We’ll teach you how to gain increased privileges during the times when you need them. Continue reading Initial Server Setup with Ubuntu 14.04 →

Linux

Install Graylog2 And Centralize Logs On Ubuntu 14.04

Leave a comment

Introduction

In this tutorial, we will cover the installation of Graylog2 (v0.20.2), and configure it to gather the syslogs of our systems in a centralized location. Graylog2 is a powerful log management and analysis tool that has many use cases, from monitoring SSH logins and unusual activity to debugging applications. It is based on Elasticsearch, Java, MongoDB, and Scala.

It is possible to use Graylog2 to gather and monitor a large variety of logs, but we will limit the scope of this tutorial to syslog gathering. Also, because we are demonstrating the basics of Graylog2, we will be installing all of the components on a single server.

About Graylog2 Components

Graylog2 has four main components:

  • Graylog2 Server nodes: Serves as a worker that receives and processes messages, and communicates with all other non-server components. Its performance is CPU dependent
  • Elasticsearch nodes: Stores all of the logs/messages. Its performance is RAM and disk I/O dependent
  • MongoDB: Stores metadata and does not experience much load
  • Web Interface: The user interface

Here is a diagram of the Graylog2 components (note that the messages are sent from your other servers):

Basic Graylog2 Setup

For a very basic setup, all of the components can be installed on the same server. For a larger, production setup, it would be wise to set up some high-availability features because if the server, Elasticsearch, or MongoDB components experiences an outage, Graylog2 will not gather the messages generated during the outage. Continue reading Install Graylog2 And Centralize Logs On Ubuntu 14.04 →

Linux

Troubleshoot Common HTTP Error Codes

Leave a comment

How To Troubleshoot Common HTTP Error Codes

Introduction

When accessing a web server or application, every HTTP request that is received by a server is responded to with an HTTP status code. HTTP status codes are three-digit codes, and are grouped into five different classes. The class of a status code can be quickly identified by its first digit:

  • 1xx: Informational
  • 2xx: Success
  • 3xx: Redirection
  • 4xx: Client Error
  • 5xx: Server Error

This guide focuses on identifying and troubleshooting the most commonly encountered HTTP error codes, i.e. 4xx and 5xx status codes, from a system administrator’s perspective. There are many situations that could cause a web server to respond to a request with a particular error code–we will cover common potential causes and solutions.

Client and Server Error Overview

Client errors, or HTTP status codes from 400 to 499, are the result of HTTP requests sent by a user client (i.e. a web browser or other HTTP client). Even though these types of errors are client-related, it is often useful to know which error code a user is encountering to determine if the potential issue can be fixed by server configuration.

Server errors, or HTTP status codes from 500 to 599, are returned by a web server when it is aware that an error has occurred or is otherwise not able to process the request.

General Troubleshooting Tips

  • When using a web browser to test a web server, refresh the browser after making server changes
  • Check server logs for more details about how the server is handling the requests. For example, web servers such as Apache or Nginx produce two files called access.log and error.log that can be scanned for relevant information
  • Keep in mind that HTTP status code definitions are part of a standard that is implemented by the application that is serving requests. This means that the actual status code that is returned depends on how the server software handles a particular error–this guide should generally point you in the right direction

Now that you have a high-level understanding of HTTP status codes, we will look at the commonly encountered errors. Continue reading Troubleshoot Common HTTP Error Codes →

Linux

Set Up a Node.js Application for Production on Ubuntu 14.04

Leave a comment

How To Set Up a Node.js Application for Production on Ubuntu 14.04

Introduction

Node.js is an open source Javascript runtime environment for easily building server-side and networking applications. The platform runs on Linux, OS X, FreeBSD, and Windows, and its applications are written in JavaScript. Node.js applications can be run at the command line but we will teach you how to run them as a service, so they will automatically restart on reboot or failure, so you can use them in a production environment.

In this tutorial, we will cover setting up a production-ready Node.js environment that is composed of two Ubuntu 14.04 servers; one server will run Node.js applications managed by PM2, while the other will provide users with access to the application through an Nginx reverse proxy to the application server.

The CentOS version of this tutorial can be found here.

Prerequisites

This guide uses two Ubuntu 14.04 servers with private networking (in the same datacenter). We will refer to them by the following names:

  • app: The server where we will install Node.js runtime, your Node.js application, and PM2
  • web: The server where we will install the Nginx web server, which will act as a reverse proxy to your application. Users will access this server’s public IP address to get to your Node.js application.

It is possible to use a single server for this tutorial, but you will have to make a few changes along the way. Simply use the localhost IP address, i.e. 127.0.0.1, wherever the app server’s private IP address is used.

Here is a diagram of what your setup will be after following this tutorial:

Reverse Proxy to Node.js Application

Before you begin this guide, you should have a regular, non-root user with sudo privileges configured on both of your servers–this is the user that you should log in to your servers as. You can learn how to configure a regular user account by following steps 1-4 in our initial server setup guide for Ubuntu 14.04.

If you want to be able to access your web server via a domain name, instead of its public IP address, purchase a domain name then follow these tutorials: Continue reading Set Up a Node.js Application for Production on Ubuntu 14.04 →

Linux

Protect your Linux Server Against the GHOST Vulnerability

Leave a comment

Introduction

On January 27, 2015, a GNU C Library (glibc) vulnerability, referred to as the GHOST vulnerability, was announced to the general public. In summary, the vulnerability allows remote attackers to take complete control of a system by exploiting a buffer overflow bug in glibc’s GetHOST functions (hence the name). Like Shellshock and Heartbleed, this vulnerability is serious and affects many servers.

The GHOST vulnerability can be exploited on Linux systems that use versions of the GNU C Library prior to glibc-2.18. That is, systems that use an unpatched version of glibc from versions 2.2 to 2.17 are at risk. Many Linux distributions including, but not limited to, the following are potentially vulnerable to GHOST and should be patched:

  • CentOS 6 & 7
  • Debian 7
  • Red Hat Enterprise Linux 6 & 7
  • Ubuntu 10.04 & 12.04
  • End of Life Linux Distributions

It is highly recommended that you update and reboot all of your affected Linux servers. We will show you how to test if your systems are vulnerable and, if they are, how to update glibc to fix the vulnerability.

Check System Vulnerability

The easiest way to test if your servers are vulnerable to GHOST is to check the version of glibc that is in use. We will cover how to do this in Ubuntu, Debian, CentOS, and RHEL.

Note that binaries that are statically linked to the vulnerable glibc must be recompiled to be made safe—this test does not cover these cases, only the system’s GNU C Library. Continue reading Protect your Linux Server Against the GHOST Vulnerability →

Linux

Use Bash’s Job Control to Manage Foreground and Background Processes

Leave a comment

Introduction

In this guide, we’ll talk about how bash, the Linux system, and your terminal come together to offer process and job control. In a previous guide, we discussed how the ps, kill, and nice commands can be used to control processes on your system.

This article will focus on managing foreground and background processes and will demonstrate how to leverage your shell’s job control functions to gain more flexibility in how you run commands.

Managing Foreground Processes

Most processes that you start on a Linux machine will run in the foreground. The command will begin execution, blocking use of the shell for the duration of the process. The process may allow user interaction or may just run through a procedure and then exit. Any output will be displayed in the terminal window by default. We’ll discuss the basic way to manage foreground processes below.

Starting a Process

By default, processes are started in the foreground. Until the program exits or changes state, you will not be able to interact with the shell.

Some foreground commands exit very quickly and return you to a shell prompt almost immediately. For instance, this command: Continue reading Use Bash’s Job Control to Manage Foreground and Background Processes →

Linux

Install Nagios 4 and Monitor Your Servers on Ubuntu 14.04

Leave a comment

How To Install Nagios 4 and Monitor Your Servers on Ubuntu 14.04

Introduction

In this tutorial, we will cover the installation of Nagios 4, a very popular open source monitoring system, on Ubuntu 14.04. We will cover some basic configuration, so you will be able to monitor host resources via the web interface. We will also utilize the Nagios Remote Plugin Executor (NRPE), that will be installed as an agent on remote hosts, to monitor their local resources.

Nagios is useful for keeping an inventory of your servers, and making sure your critical services are up and running. Using a monitoring system, like Nagios, is an essential tool for any production server environment.

Prerequisites

To follow this tutorial, you must have superuser privileges on the Ubuntu 14.04 server that will run Nagios. Ideally, you will be using a non-root user with superuser privileges. If you need help setting that up, follow the steps 1 through 3 in this tutorial: Initial Server Setup with Ubuntu 14.04.

A LAMP stack is also required. Follow this tutorial if you need to set that up: How To Install Linux, Apache, MySQL, PHP (LAMP) stack on Ubuntu 14.04.

This tutorial assumes that your server has private networking enabled. If it doesn’t, just replace all the references to private IP addresses with public IP addresses.

Now that we have the prerequisites sorted out, let’s move on to getting Nagios 4 installed.

Install Nagios 4

This section will cover how to install Nagios 4 on your monitoring server. You only need to complete this section once. Continue reading Install Nagios 4 and Monitor Your Servers on Ubuntu 14.04 →

Linux

Map User Location with GeoIP and ELK (Elasticsearch, Logstash, and Kibana)

Leave a comment

How To Map User Location with GeoIP and ELK (Elasticsearch, Logstash, and Kibana)

Introduction

IP Geolocation, the process used to determine the physical location of an IP address, can be leveraged for a variety of purposes, such as content personalization and traffic analysis. Traffic analysis by geolocation can provide invaluable insight into your user base as it allows you to easily see where they users are coming from, which can help you make informed decisions about the ideal geographical location(s) of your application servers and who your current audience is. In this tutorial, we will show you how to create a visual geo-mapping of the IP addresses of your application’s users, by using a GeoIP database with Elasticsearch, Logstash, and Kibana.

Here’s a short explanation of how it all works. Logstash uses a GeoIP database to convert IP addresses into latitude and longitude coordinate pair, i.e. the approximate physical location of an IP address. The coordinate data is stored in Elasticsearch in geo_point fields, and also converted into a geohash string. Kibana can then read the Geohash strings and draw them as points on a map of earth, known in Kibana 4 as a Tile Map visualization.

Let’s take a look at the prerequisites now.

Prerequisites

To follow this tutorial, you must have a working ELK stack. Additionally, you must have logs that contain IP addresses that can be filtered into a field, like web server access logs. If you don’t already have these two things, you can follow the first two tutorials in this series. The first tutorial will set up an ELK stack, and second one will show you how to gather and filter Nginx or Apache access logs:

Download Latest GeoIP Database

MaxMind provides free and paid GeoIP databases—the paid versions are more accurate. Logstash also ships with a copy of the free GeoIP City database, GeoLite City. In this tutorial, we will download the latest GeoLite City database, but feel free to use a different GeoIP database if you wish. Continue reading Map User Location with GeoIP and ELK (Elasticsearch, Logstash, and Kibana) →

Linux

How To Use Kibana Dashboards and Visualizations

Leave a comment

How To Use Kibana Dashboards and Visualizations

Introduction

Kibana 4 is an analytics and visualization platform that builds on Elasticsearch to give you a better understanding of your data. In this tutorial, we will get you started with Kibana, by showing you how to use its interface to filter and visualize log messages gathered by an Elasticsearch ELK stack. We will cover the main interface components, and demonstrate how to create searches, visualizations, and dashboards.

Prerequisites

This tutorial is the third part in the Centralized Logging with Logstash and Kibana series.

It assumes that you have a working ELK setup. The examples assume that you are gathering syslog and Nginx access logs. If you are not gathering these types of logs, you should be able to modify the demonstrations to work with your own log messages.

If you want to follow this tutorial exactly as presented, you should have the following setup, by following the first two tutorials in this series:

When you are ready to move on, let’s look at an overview of the Kibana interface.

Kibana Interface Overview

The Kibana interface is divided into four main sections:

  • Discover
  • Visualize
  • Dashboard
  • Settings

We will go over the basics of each section, in the listed order, and demonstrate how each piece of the interface can be used. Continue reading How To Use Kibana Dashboards and Visualizations →

Linux

Adding Logstash Filters To Improve Centralized Logging

Leave a comment

Adding Logstash Filters To Improve Centralized Logging

Introduction

Logstash is a powerful tool for centralizing and analyzing logs, which can help to provide and overview of your environment, and to identify issues with your servers. One way to increase the effectiveness of your Logstash setup is to collect important application logs and structure the log data by employing filters, so the data can be readily analyzed and query-able. We will build our filters around “grok” patterns, that will parse the data in the logs into useful bits of information.

This guide is a sequel to the How To Install Elasticsearch, Logstash, and Kibana 4 on Ubuntu 14.04 tutorial, and focuses primarily on adding filters for various common application logs.

Prerequisites

To follow this tutorial, you must have a working Logstash server, and a way to ship your logs to Logstash. If you do not have Logstash set up, here is another tutorial that will get you started: How To Install Elasticsearch, Logstash, and Kibana 4 on Ubuntu 14.04.

Logstash Server Assumptions:

  • Logstash is installed in /opt/logstash
  • You are receiving logs from Logstash Forwarder on port 5000
  • Your Logstash configuration files are located in /etc/logstash/conf.d
  • You have an input file named 01-lumberjack-input.conf
  • You have an output file named 30-lumberjack-output.conf

Continue reading Adding Logstash Filters To Improve Centralized Logging →