Ntopng How to install Ntopng Network Traffic Monitoring Tool

ntop4

ntop2

Ntopng is a very useful network traffic monitoring system, its a monitoring tool with detailed graphs and  flows. Its interface having a number of view options of network traffic, including the top flow talkers, top hosts (Send/Receive) data, application protocols in use, top flow senders data live. Each and every node’s active flow can be viewed using the ntopng.

For reference www.ntop.org.

To Install Ntopng, the following steps are followed, first of all we need to upgrade the repository files, create the following file to start up for installation:

# vi /etc/yum.repos.d/ntop.repo
[ntop]
name=ntop packages
baseurl=http://www.nmon.net/centos/6.5/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://www.nmon.net/centos/RPM-GPG-KEY-deri

Now create the file in “/etc/yum.repos.d/epel.repo extra repositories”

# vi /etc/yum.repos.d/epel.repo
[epel]
name=Extra Packages for Enterprise Linux 6 -
baseurl=http://mirror.digmia.com/epel/6Server/x86_64/
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

[epel-debuginfo]
name=Extra Packages for Enterprise Linux 6 -  - Debug
baseurl=http://download.fedoraproject.org/pub/epel/6//debug
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-6&arch=
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
gpgcheck=1

[epel-source]
name=Extra Packages for Enterprise Linux 6 -  - Source
baseurl=http://download.fedoraproject.org/pub/epel/6/SRPMS
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-source-6&arch=
failovermethod=priority
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
gpgcheck=1

Now do the following:
To clean all yum data

yum clean all

To update the repositories and all installed packages following command is used.

yum update -y

Redis and Hiredis are the required packages for the Ntopng installation, so need to install it before going to install ntopng

yum install redis hiredis

if there will be problem for installation of redis or hiredis, do install the following rpm:

rpm -ivh http://mirror.digmia.com/epel/6Server/x86_64/epel-release-6-8.noarch.rpm

Now install the pf_ring

yum install pfring

after successful installation of all the above mentioned packages, install ntopng with its packages

yum install ntopng ntopng-data nbox

Now edit the ntopng configuration file and make the following changes.

vi /etc/ntopng/ntopng.conf 
-n=1
-g=-1
-G=/var/tmp/ntopng.pid
-i=eth0
-i=eth1

Now edit the ntopng start “ntopng.start” file and make following changes.

vi /etc/ntopng/start.conf
--local-networks 192.168.0.0 ## give your local IP Ranges here.
--interface 0

Now run the following commands to start the services of redis and ntopng.

service redis start 
service ntopng start

Al Hamdu Lillah we have done it.
The Following graphs will be displayed on completion of installation process and configuration of ntopng.

ntop8ntop7

Postfix Mail Delivery SMTP Troubleshooting

To avoid delay in emails, specially for gmail. Just find which MX is working by issuing below commands:

# dig mx gmail.com

# telnet mx-IP/name 25

upon success, just see the name of first mx tried from /var/log/maillog

and add an entry in /etc/hosts file.

In my case, the entry is given below:
# vim /etc/hosts

74.125.25.26 aspmx.l.google.com
74.125.25.26 alt1.aspmx.l.google.com
74.125.25.26 alt2.aspmx.l.google.com

:save and exit

now open postfix main.cf and add smtp_host_lookup directive under inet_protocols = all

# vim /etc/postfix/main.cf

inet_protocols = all
smtp_host_lookup=native
save and exit

now restart postfix

# service postfix restart

by default postfix host lookup is done via DNS, native mode uses nsswitch.conf settings and will check /etc/hosts first and then DNS if entry not found in /etc/hosts

Highly Available WEB Servers Cluster Configuration by using rsync and heartbeat Linux based Utilities

Highly Available  WEB Servers Cluster Configuration  by using rsync and  heartbeat Linux based Utilities

Written by: Muhammad Farrukh Siddique (LPIC)

Operating System on both machines: CentOS-5.3 Final  Required RPMs: Heartbeat,rsync,httpd
Service to be mirrored: web service (httpd)
First Machine name (fqdn): node1.ha.int 
Second Machine name (fqdn): node2.ha.int
IP Address of node1: 192.168.3.224
IP Address of node2: 192.168.3.225
Default Gateway of both machines: 192.168.3.1
DNS Server: 192.168.2.11

Continue reading “Highly Available WEB Servers Cluster Configuration by using rsync and heartbeat Linux based Utilities”

Snort RPM Based Installation with SnortSnarf and Snort Rule Creation Methods on Linux RHEL /CentOS / Fedora

RPM Based Installation of Snort with SnortSnarf and Snort Rule Creation Methods:

written by: Muhammad Farrukh Siddique

Snort is an open source Network Intrusion Detection System
Snort can also act as Sniffer and Logger
In this section we will discuss the more beneficial part of Snort that is Intrusion Detection System (IDS)

Snort can also be installed on Windows Platform which will be covered later.
Currently the Operating System is CentOS-5.3

Snort Installation and Configuration:

First of all download and install the following required packages

# snort and snort-mysql rpm (download-able from http://www.snort.org/downloads)
# mysql and mysql-server-5.0 with all affected packages
# libpcap
# Apache
# php5
# php-cli
# php-common
# php-devel
# php-ldap
# Php5-mysql

You can check these rpms through rpm -qa command.
you can install missing packages either from source DVD or through yum command

After downloading snort rpms, just install them

[root@snortserver tmp]# rpm -ivh snort-2.8.5.3-1.RH5.i386.rpm
[root@snortserver tmp]# rpm -ivh snort-mysql-2.8.5.3-1.RH5.i386.rpm

now configure some basic settings for snort:

# vim /etc/snort/snort.conf
Change var HOME_NET any to var HOME_NET 192.168.2.0/24 # you can give a single ip address or range of IP addresses by syntax [192.168.2.1,192.168.2.10] and similarly different networks can also be defined as [192.168.1.0/24,192.168.2.0/24]
Change var EXTERNAL_NET any to var EXTERNAL_NET !$HOME_NET # (It states that everything except HOME_NET is external).
The rule path should be
var RULE_PATH /etc/snort/rules
save and exit

Now add rules to /etc/snort/rules directory

just download all the .rules file from the link http://cvs.snort.org/viewcvs.cgi/snort/rules/ one by one.

or just follow a simple way, i have created all the links for your convenience and time saving.

# cd /tmp
# mkdir rules
# cd rules
# vim download-rules.txt
http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/attack-responses.rules?rev=1.23.2.7
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/backdoor.rules?rev=1.50
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/bad-traffic.rules?rev=1.22.2.5
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/chat.rules?rev=1.27
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/ddos.rules?rev=1.14.2.7
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/deleted.rules?rev=1.39
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/dns.rules?rev=1.42
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/dos.rules?rev=1.43
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/experimental.rules?rev=1.80
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/finger.rules?rev=1.29
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/ftp.rules?rev=1.63
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/icmp-info.rules?rev=1.25
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/icmp.rules?rev=1.27
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/imap.rules?rev=1.13.2.13
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/info.rules?rev=1.31
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/misc.rules?rev=1.59
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/local.rules?rev=1.13
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/multimedia.rules?rev=1.15
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/mysql.rules?rev=1.13
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/netbios.rules?rev=1.54
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/nntp.rules?rev=1.16
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/oracle.rules?rev=1.21
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/other-ids.rules?rev=1.12
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/p2p.rules?rev=1.20
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/pop2.rules?rev=1.14
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/pop3.rules?rev=1.26
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/porn.rules?rev=1.12
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/rservices.rules?rev=1.24
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/scan.rules?rev=1.19.2.6
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/shellcode.rules?rev=1.27
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/smtp.rules?rev=1.48
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/snmp.rules?rev=1.19
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/sql.rules?rev=1.32
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/telnet.rules?rev=1.40
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/tftp.rules?rev=1.21
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-attacks.rules?rev=1.21
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-cgi.rules?rev=1.63.2.15
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-client.rules?rev=1.27
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-coldfusion.rules?rev=1.30
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-frontpage.rules?rev=1.35
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-iis.rules?rev=1.84
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-misc.rules?rev=1.102.2.15
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/web-php.rules?rev=1.23
 http://cvs.snort.org/viewcvs.cgi/*checkout*/snort/rules/x11.rules?rev=1.20
save and exit
# wget -i download-rules.txt
it will download all the rules at once saving a lot of time.
[you can start snort by simple creating blank .rules files but remember with blank rule files snort will not detect anything]

after downloading the rules in a directory. Copy the directory on another location. Rename each file to rule-name.rules and then paste the files in /etc/snort/rules directory

change the ownership of newly created rules files

# chown snort.root /etc/snort/rules/*

Keep the original rules directory and match weekly the revision version of each rule on the site to incorporate changes accordingly.

some rules can also be download and copied in rules directory from

http://openmaniak.com/snort_bleeding.php
 then append these lines in inlude section of/etc/snort/snort.conf file for each new rule
include $RULE_PATH/bleeding.rules
 include $RULE_PATH/bleeding-attack_response.rules
 include $RULE_PATH/bleeding-dos.rules
 include $RULE_PATH/bleeding-drop.rules
 include $RULE_PATH/bleeding-dshield.rules
 include $RULE_PATH/bleeding-exploit.rules
 include $RULE_PATH/bleeding-game.rules
 include $RULE_PATH/bleeding-inappropriate.rules
 include $RULE_PATH/bleeding-malware.rules
 include $RULE_PATH/bleeding-p2p.rules
 include $RULE_PATH/bleeding-scan.rules
 include $RULE_PATH/bleeding-virus.rules
 include $RULE_PATH/bleeding-web.rules

Now its time to start snort

Start snort

# /etc/init.d/snortd start

Remember syslogd daemon must be running in order to take logs

The log file named ‘alert’ will be generated in /var/log/snort directory

now do a port scan and check the alerts in /var/log/snort/alert file, if you have defined a complete subnet in HOME_NET variable then you can do a port scan of any host but that host must be reachable by snort server
If you have some network issues then try stopping iptables and SElinux for test purpose

if you have nmap installed then simple run the command to scan ports

# nmap localhost

you can see alerts in real time by command

# tailf /var/log/snort/alert

At this time snort is running fine and generating alerts
Now lets move to an interesting section

Creating your own Snort Rules:
—————————-

you can also use Snort Rule Generator or some other utilities but its better to write a rule in a file. lets do it

We are going to create a rule for generating alerts whenever someone is trying to open youtube.com

# vim /etc/snort/rules/youtube.rules
alert tcp any any -> any any (content:"www.youtube.com"; msg:"someone is accessing youtube"; sid:1000003; rev:1;)
save and exit

now add the rule path in snort.conf

#vim /etc/snort/snort.conf
append the line in include section as
include $RULE_PATH/youtube.rules
save and exit

and restart snort daemon to take affect changes

# /etc/init.d/snortd restart

lets describe the rule:

alert tcp any any -> any any (content:"www.youtube.com"; msg:"someone is accessing youtube"; sid:1000003; rev:1;)
tcp = protocol
 1st any = Source IP (your local IP in this case)
 2nd any = Source Port
 -> = request for
 3rd any = Destination IP (youtube IP in this case)
 4th any = Destination Port (that would be 80 in this case)
 sid = snort unique id which snort uses to differentiate different alerts and their descriptions
 rev = version number
Now after creating the rule and restarting the snort daemon, lets check our new rule working status
just type

# elinks www.youtube.com

wait for youtube to be loaded

or Open a browser and type www.youtube.com

then view the alerts

# cat /var/log/snort/alert

you will see some alerts like

[**] [1:100003:1] someone is accessing youtube.com [**] [Priority: 0] {TCP} 192.168.2.5:48401 -> 64.233.169.139:80
[**] [1:100003:1] someone is accessing youtube.com [**] [Priority: 0] {TCP} 192.168.2.5:38510 -> 110.93.194.23:80

SnortSnarf:
———-

In order to get better results we will have to manage alerts in a more descriptive form

So lets install and configure SnortSnarf an html generator for snort alerts

make sure that you have installed Apache and its running

[root@snortserver ~]# /etc/init.d/httpd status
 httpd (pid 2992 2991 2990 2989 2988 2987 2986 2985 2983) is running...
 [root@snortserver ~]#

otherwise just start it

# /etc/init.d/httpd start

make sure that you have installed perl5 packages because SnortSnarf depends on perl based Time modules

lets start

download snortsnarf from http://sourceforge.net/projects/snortsnarf/

# cd /etc
# wget http://downloads.sourceforge.net/project/snortsnarf/snortsnarf/Initial%20Release%201.0/SnortSnarf-1.0.tar.gz?use_mirror=space
# tar -zxvf SnortSnarf-1.0.tar.gz

Now you have to download and install Time modules needed for snortsnarf from http://search.cpan.org/~muir/Time-modules-2006.0814/

# wget http://search.cpan.org/CPAN/authors/id/M/MU/MUIR/modules/Time-modules-2006.0814.tar.gz
# tar -zxvf Time-modules-2006.0814.tar.gz
# cd Time-modules-2006.0814

install the modules by following commands:

# perl Makefile.pl
# make
# make test
# make install

After successfull installation of Time modules, create a destination directory where snortsnarf will generate html pages as an output

# mkdir /var/www/html/ids

Now run snortsnarf

# cd /etc/SnortSnarf-1.0
# ./snortsnarf.pl -d /var/www/html/ids/ /var/log/snort/alert

where -d defines output directory and /var/log/snort/alert is the source file to take input.

The above command shows a similar output on the console

[root@snortserver SnortSnarf-1.0]# ./snortsnarf.pl -d /var/www/html/ids/ /var/log/snort/alert
 Using an array as a reference is deprecated at include/SnortSnarf/HTMLMemStorage.pm line 290.
 Using an array as a reference is deprecated at include/SnortSnarf/HTMLAnomMemStorage.pm line 266.
 SnortFileInput: input file /var/log/snort/alert exists but is length 0; skipping it
 [root@snortserver SnortSnarf-1.0]#

if your /var/log/snort/alert file is not empty then you will not see the last output line

Cron Job for SnortSnarf:
———————–

Create a cron job to automatically run the snorsnarf script to append alerts in html pages

first of all make a simple script

# cd /etc
# touch snortsnarf
# vim snortsnarf
cd /etc/SnortSnarf-1.0
 ./snortsnarf.pl -d /var/www/html/ids /var/log/snort/alert
save and exit

make the script executable

# chmod +x snortsnarf

now define a new cron job to run snortsnarf after every 5 minutes

# crontab -e
*/5 * * * * /etc/snortsnarf
save and exit

*/5 means that snortsnarf will be run after every 5 minutes, you can adjust the time accordingly.

restart the cron and Apache daemons

# /etc/init.d/crond restart
# /etc/init.d/httpd restart

now create a simple port scan and generate alerts in /var/log/snort/alert file and then open your browser and type http://localhost/ids

You can see the alerts, further click the alerts and find some more information and go on.

Now snort IDS with SnortSnarf has been successfully configured, Alhamdullilah