Graylog2 Installation & Configuration for Log Management on CentOS /RHEL 6.x with MongoDB
Graylog2
is for data analysis
Field-tested open source data analytics system used and trusted all around the world. Search your logs, create charts, send reports and be alerted when something happens. All running on the existing JVM in your datacenter.
Features
Manage billions of events and process hundreds of thousands of new events every second. All of the infrastructure runs in your own data center: No need to worry about sensitive data leaving your company or shipping your logs away over the internet.
Graylog2 is leveraging proven technologies e.g. Java, Scala and ElasticSearch. All communication is done via REST APIs so you can even build your own tools on top of it. The graylog2-server package ships with an interactive API browser.
To Download http://graylog2.org/download
Here we start installation and configuration of Graylog2 with MongoDB on CentOS/RHEL 6.x server
Install prerequisites
yum install -y gcc gcc-c++ gd gd-devel glibc glibc-common glibc-devel glibc-headers make automake httpd httpd-devel java-1.7.0-openjdk java-1.7.0-openjdk-devel wget tar vim nc libcurl-devel openssl-devel zlib-devel zlib patch readline readline-devel libffi-devel curl-devel libyaml-devel libtoolbisonlibxml2-devel libxslt-devel libtool bison wget
yum install make gcc-c++ httpd httpd-devel readline-devel make httpd httpd-devel readline-devel gcc automake autoconf curl-devel openssl-devel zlib-devel apr-devel apr-util-devel sqlite-devel java git wget
Download and install elasticsearch
cd /opt
git clone https://github.com/elasticsearch/elasticsearch-servicewrapper.git wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.6.tar.gz tar zxvf elasticsearch-0.20.6.tar.gz ln -s elasticsearch-0.20.6/ elasticsearch mv *servicewrapper*/service elasticsearch/bin/ rm -Rf *servicewrapper* /opt/elasticsearch/bin/service/elasticsearch install ln -s `readlink -f elasticsearch/bin/service/elasticsearch` /usr/bin/elasticsearch_ctl sed -i -e 's|# cluster.name: elasticsearch|cluster.name: graylog2|' /opt/elasticsearch/config/elasticsearch.yml /etc/init.d/elasticsearch start
new version of graylog web interface require ruby version
cd
wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
tar zxvf yaml-0.1.4.tar.gz
cd yaml-0.1.4
./configure --prefix=/usr/local
make
make install
cd
wget http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p0.tar.gz
tar xzvf ruby-1.9.3-p0.tar.gz
cd ruby-1.9.3-p0
./configure --prefix=/usr/local --enable-shared --disable-install-doc --with-opt-dir=/usr/local/lib
make
make install
Verify the version
ruby -v gem --version
Add 10gen Repository for mongodb
vi /etc/yum.repos.d/mongodb.repo
[10gen]
name=10gen Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686
gpgcheck=0
:wq!
yum clean all
yum install mongo-10gen-server mongo-10gen readline-devel
vi /etc/mongod.conf
Enable authentication by uncommenting the line
auth = true
:wq!
/etc/init.d/mongod start
chkconfig mongod on
Login to mongo db and add user and database for graylog2
mongo
use admin
db.addUser('admin', 'redhat')
db.auth('admin', 'redhat')
use graylog2
db.addUser('grayloguser', 'redhat')
exit
Download the latest version of graylog2-server and untar it
https://github.com/Graylog2/graylog2-server/releases/download/0.12.0/graylog2-server-0.12.0.tar.gz
tar xvfz graylog2-server-0.12.0.tar.gz
ln -s graylog2-server-0.12.0/ graylog2-server
cd graylog2-server-0.12.0
Copy the conf files
cp graylog2.conf.example /etc/graylog2.conf
cp elasticsearch.yml.example /etc/graylog2-elasticsearch.yml
Open the graylog2.conf file in an edit
vi /etc/graylog2.conf
syslog_enable_tcp = true
mongodb_user = grayloguser
mongodb_password = redhat
Create the initialization script
vi /etc/init.d/graylog2-server
#!/bin/bash # graylog2-server: graylog2 message collector # chkconfig: - 98 02 # description: This daemon start graylog2-server # Source function library. . /etc/rc.d/init.d/functions CMD=$1 NOHUP=`which nohup` STOP_TIMEOUT=30 BINARY=java PROG=graylog2-server HOME_DIR=/opt/graylog2-server LOG_FILE=${HOME_DIR}/log/${PROG}.log JAR_FILE=graylog2-server.jar GRAYLOG2_CONFIG_SH=${GRAYLOG2CTL_DIR}/bin/graylog2_config.sh CONF_FILE=/etc/graylog2.conf PID_FILE=/var/run/graylog2.pid [ -f $GRAYLOG2_CONFIG_SH ] && . $GRAYLOG2_CONFIG_SH start() { graylog2_status > /dev/null 2>&1 if [ ${RETVAL} -eq 3 ] then echo "Starting ${PROG} ..." cd ${HOME_DIR} $NOHUP > /dev/null 2>&1 ${BINARY} -jar ${JAR_FILE} -f ${CONF_FILE} -p ${PID_FILE} >> ${LOG_FILE} & RETVAL=0 else echo "${PROG} is already running" fi } stop() { echo -n $"Stopping $PROG: " killproc -p ${PID_FILE} -d ${STOP_TIMEOUT} ${PROG} RETVAL=$? echo [ $RETVAL = 0 ] && rm -f ${PID_FILE} } graylog2_status() { status -p ${PID_FILE} ${PROG} RETVAL=$? } restart() { echo "Restarting ${PROG} ..." stop start } case "$CMD" in start) start ;; stop) stop ;; restart) restart ;; status) graylog2_status ;; *) echo "Usage $0 {start|stop|restart|status}" RETVAL=1 esac exit ${RETVAL}
chmod +x /etc/init.d/graylog2-server chkconfig --add graylog2-server chkconfig graylog2-server on service graylog2-server start
install graylog2 web interface
wget https://github.com/Graylog2/graylog2-web-interface/releases/download/0.12.0/graylog2-web-interface-0.12.0.tar.gz
tar zxvf graylog2-web-interface-0.12.0.tar.gz
mv graylog2-web-interface-0.12.0 /var/www/html/graylog2
cd /var/www/html/graylog2/
gem update
gem install git rake bundler
bundle install
Update the mongoid,yml file
cd /var/www/html/graylog2/config
vi mongoid.yml
Add the username and password
production:
host: localhost
port: 27017
username: grayloguser
password: redhat
database: graylog2
:wq!
Create the indexes
cd /var/www/html/graylog2
bundle exec rake db:mongoid:create_indexes RAILS_ENV=production –trace
Install apache passenger module
#passenger-install-apache2-module (it should take 2 hours installation time)
Create a new conf file to load passenger module
vi /etc/httpd/conf.d/passenger.conf
LoadModule passenger_module /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.18
PassengerRuby /usr/local/bin/ruby
Cd /var/www/html
Mkdir graylog2/public
Create virtual hosts
vi /etc/httpd/conf/httpd.conf
<VirtualHost *:80>
ServerName graylog.royalcyber.local
DocumentRoot /var/www/html/graylog2/public
RailsEnv production
ServerAlias gray.logger
ErrorLog logs/graylog2-error_log
CustomLog logs/graylog2-access_log custom
<Directory /var/www/html/graylog2/public>
Allow from all
Options -MultiViews
</Directory>
</VirtualHost>
:wq!
/etc/init.d/httpd restart
Open your I.E, Firefox use IP of your Web server.