Graylog2 Installation & Configuration for Log Management on CentOS /RHEL 6.x with MongoDB

Learn | Teach Open Source Technologies

Graylog2 Installation & Configuration for Log Management on CentOS /RHEL 6.x with MongoDB

Graylog2

is for data analysis

Features

Manage billions of events and process hundreds of thousands of new events every second. All of the infrastructure runs in your own data center: No need to worry about sensitive data leaving your company or shipping your logs away over the internet. 

Graylog2 is leveraging proven technologies e.g. Java, Scala and ElasticSearch. All communication is done via REST APIs so you can even build your own tools on top of it. The graylog2-server package ships with an interactive API browser.

http://graylog2.org

To Download http://graylog2.org/download

http://graylog2.org/support

Here we start installation and configuration of Graylog2 with MongoDB on CentOS/RHEL 6.x server

Install  prerequisites

yum install -y gcc gcc-c++ gd gd-devel glibc glibc-common glibc-devel glibc-headers make automake httpd httpd-devel java-1.7.0-openjdk java-1.7.0-openjdk-devel wget tar vim nc libcurl-devel openssl-devel zlib-devel zlib patch readline readline-devel libffi-devel curl-devel libyaml-devel libtoolbisonlibxml2-devel libxslt-devel libtool bison wget

yum install make gcc-c++ httpd httpd-devel readline-devel make httpd httpd-devel readline-devel gcc automake autoconf curl-devel openssl-devel zlib-devel apr-devel apr-util-devel sqlite-devel java git wget

 

Download and install elasticsearch

cd /opt
git clone https://github.com/elasticsearch/elasticsearch-servicewrapper.git
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.6.tar.gz
tar zxvf elasticsearch-0.20.6.tar.gz
ln -s elasticsearch-0.20.6/ elasticsearch
mv *servicewrapper*/service elasticsearch/bin/
rm -Rf *servicewrapper*
/opt/elasticsearch/bin/service/elasticsearch install
ln -s `readlink -f elasticsearch/bin/service/elasticsearch` /usr/bin/elasticsearch_ctl
sed -i -e 's|# cluster.name: elasticsearch|cluster.name: graylog2|' /opt/elasticsearch/config/elasticsearch.yml
/etc/init.d/elasticsearch start

 

new version of graylog web interface require ruby version

cd
wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
tar zxvf yaml-0.1.4.tar.gz
cd yaml-0.1.4
./configure --prefix=/usr/local
make
make install
cd
wget http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p0.tar.gz
tar xzvf ruby-1.9.3-p0.tar.gz
cd ruby-1.9.3-p0
./configure --prefix=/usr/local --enable-shared --disable-install-doc --with-opt-dir=/usr/local/lib
make
make install
Verify the version
 
ruby -v
 gem --version

Add 10gen Repository for mongodb

vi /etc/yum.repos.d/mongodb.repo
[10gen]
name=10gen Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686
gpgcheck=0
:wq!
yum clean all
yum install mongo-10gen-server mongo-10gen readline-devel
vi /etc/mongod.conf
Enable authentication by uncommenting the line
auth = true
:wq!
/etc/init.d/mongod start
chkconfig mongod on

 

Login to mongo db and add user and database for graylog2

mongo
use admin
db.addUser('admin', 'redhat')
db.auth('admin', 'redhat')
use graylog2
db.addUser('grayloguser', 'redhat')
exit

 

Download the latest version of graylog2-server and untar it

 

https://github.com/Graylog2/graylog2-server/releases/download/0.12.0/graylog2-server-0.12.0.tar.gz
tar xvfz graylog2-server-0.12.0.tar.gz
ln -s graylog2-server-0.12.0/ graylog2-server
cd graylog2-server-0.12.0

Copy the conf files

 

cp graylog2.conf.example /etc/graylog2.conf
cp elasticsearch.yml.example /etc/graylog2-elasticsearch.yml

 

Open the graylog2.conf file in an edit

vi /etc/graylog2.conf
syslog_enable_tcp = true
mongodb_user = grayloguser
mongodb_password = redhat

Create the initialization script

vi /etc/init.d/graylog2-server
#!/bin/bash
# graylog2-server:   graylog2 message collector
# chkconfig: - 98 02
# description:  This daemon start graylog2-server
# Source function library.
. /etc/rc.d/init.d/functions
CMD=$1
NOHUP=`which nohup`
STOP_TIMEOUT=30
BINARY=java
PROG=graylog2-server
HOME_DIR=/opt/graylog2-server
LOG_FILE=${HOME_DIR}/log/${PROG}.log
JAR_FILE=graylog2-server.jar
GRAYLOG2_CONFIG_SH=${GRAYLOG2CTL_DIR}/bin/graylog2_config.sh
CONF_FILE=/etc/graylog2.conf
PID_FILE=/var/run/graylog2.pid
[ -f $GRAYLOG2_CONFIG_SH ] && . $GRAYLOG2_CONFIG_SH
start() {
graylog2_status > /dev/null 2>&1
if [ ${RETVAL} -eq 3 ]
then
echo "Starting ${PROG} ..."
cd ${HOME_DIR}
$NOHUP > /dev/null 2>&1 ${BINARY} -jar ${JAR_FILE} -f ${CONF_FILE} -p ${PID_FILE} >> ${LOG_FILE} &
RETVAL=0
else
echo "${PROG} is already running"
fi
}
stop() {
echo -n $"Stopping $PROG: "
killproc -p ${PID_FILE} -d ${STOP_TIMEOUT} ${PROG}
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f ${PID_FILE}
}
graylog2_status() {
status -p ${PID_FILE} ${PROG}
RETVAL=$?
}
restart() {
echo "Restarting ${PROG} ..."
stop
start
}
case "$CMD" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
status)
graylog2_status
;;
*)
echo "Usage $0 {start|stop|restart|status}"
RETVAL=1
esac
exit ${RETVAL}
chmod +x /etc/init.d/graylog2-server
chkconfig --add graylog2-server
chkconfig graylog2-server on
service graylog2-server start

install graylog2 web interface

wget https://github.com/Graylog2/graylog2-web-interface/releases/download/0.12.0/graylog2-web-interface-0.12.0.tar.gz
tar zxvf graylog2-web-interface-0.12.0.tar.gz
mv graylog2-web-interface-0.12.0 /var/www/html/graylog2
cd /var/www/html/graylog2/
gem update
gem install git rake bundler
bundle install

Update the mongoid,yml file

cd /var/www/html/graylog2/config
vi mongoid.yml

 

Add the username and password

production:
host: localhost
port: 27017
username: grayloguser
password: redhat
database: graylog2
:wq!

Create the indexes

cd /var/www/html/graylog2
bundle exec rake db:mongoid:create_indexes RAILS_ENV=production –trace

 

Install apache passenger module

 

#passenger-install-apache2-module   (it should take 2 hours installation time)

 

Create a new conf file to load passenger module

 vi /etc/httpd/conf.d/passenger.conf
LoadModule passenger_module /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.18
PassengerRuby /usr/local/bin/ruby
Cd /var/www/html
Mkdir graylog2/public
 

Create virtual hosts

vi /etc/httpd/conf/httpd.conf
<VirtualHost *:80>
ServerName graylog.royalcyber.local
DocumentRoot /var/www/html/graylog2/public
RailsEnv production
ServerAlias gray.logger
ErrorLog logs/graylog2-error_log
CustomLog logs/graylog2-access_log custom
<Directory /var/www/html/graylog2/public>
Allow from all
Options -MultiViews
</Directory>
</VirtualHost>
:wq!

/etc/init.d/httpd restart

 

Open your I.E, Firefox use IP of your Web server.