fastnetmon

FastNetMon - High Performance Network Load Analyzer with PCAP/ULOG2 support. But I recommends only PF_RING variant because other variants is so slow and use big amount of CPU and produce big packetloss.

What we do? We can detect hosts in OUR network with big amount of packets per second (30 000 pps in standard configuration) incoming or outgoing from certain host. And we can call external bash script which can send notify, switch off server or blackhole this client.

Why you write it? Because we can't find any software for solving this problem not in proprietary world not in open source. NetFlow based solutions is so slow and can't react on atatck with fast speed.

At now we start usage of C++11 and you can build this programm only on Debian 7 Wheezy, CentOS 6 has so old g++ compiler and can't compile it (but with CentOS 7 everything will be fine but it's not released yet).

Main programm screen image:

Install:

   # Debian 7 Wheezy
   apt-get install -y git libpcap-dev g++ gcc libboost-all-dev make

   # If you need traffic counting
   apt-get install -y libhiredis-dev

   # If you need PF_RING abilities 
   apt-get install -y libnuma-dev

   # If you need ASN/geoip stats
   apt-get install -y libgeoip-dev 

   cd /usr/src
   git clone https://github.com/FastVPSEestiOu/fastnetmon.git
   cd fastnetmon

If you want use PF_RING you should install it.

cd /usr/src
wget 'http://downloads.sourceforge.net/project/ntop/PF_RING/PF_RING-5.6.2.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fntop%2Ffiles%2FPF_RING%2F&ts=1393755620&use_mirror=kent' -OPF_RING-5.6.2.tar.gz
tar -xf PF_RING-5.6.2.tar.gz 
cd PF_RING-5.6.2
apt-get install build-essential bison flex linux-headers-$(uname -r) libnuma-dev

Build PF_RING kernel module:

cd kernel
make 
make install
modprobe pf_ring

Build lib:

cd /usr/src/PF_RING-5.6.2/userland/lib
./configure  --disable-bpf --prefix=/opt/pf_ring

You should start fastnetmon using this options:

LD_LIBRARY_PATH=/opt/pf_ring/lib/ ./fastnetmon eth3,eth4

If you want to avoid LD_LIBRARY_PATH on every call you should add pf_ring path to system:

echo "/opt/pf_ring/lib" > /etc/ld.so.conf.d/pf_ring.conf
ldconfig -v

We disabled bpf because it requires linking to PCAP.

Select backend, we use PF_RING as default, if you need PCAP/ULOG2 u must change variable ENGINE in Makefile.

Compile it:

make

Download GeoIP database to current folder:

http://dev.maxmind.com/geoip/legacy/geolite/
http://download.maxmind.com/download/geoip/database/asnum/GeoIPASNum.dat.gz
gunzip GeoIPASNum.dat.gz

Start it:

./fastnetmon

Server configuration for PCAP: no configuration needed

Server configuration for ULOG2:

iptables -A FORWARD -i br0 -j ULOG --ulog-nlgroup 1 --ulog-cprange 32 --ulog-qthreshold 45

If you use PCAP, u can set monitored interface as command line parameter (u can set 'any' as inerface name but it work not so fine):

./fastnetmon br0

Example program screen:

Below you can see all clients with more than 2000 pps

Incoming Traffic    66167 pps 88 mbps
xx.yy.zz.15         3053  pps 0  Mbps
xx.yy.zz.248        2948  pps 0  Mbps
xx.yy.zz.192        2643  pps 0  Mbps

Outgoing traffic    91676 pps 728 mbps
xx.yy.zz.15         4471  pps 40  Mbps
xx.yy.zz.248        4468  pps 40  Mbps
xx.yy.zz.192        3905  pps 32  Mbps
xx.yy.zz.157        2923  pps 24  Mbps
xx.yy.zz.169        2809  pps 24  Mbps
xx.yy.zz            2380  pps 24  Mbps
xx.yy.zz            2105  pps 16  Mbps

Internal traffic    1 pps

Other traffic       25 pps

ULOG buffer errors: 2 (0%)
ULOG packets received: 19647

Example for cpu load for Intel i7 2600 with Intel X540 NIC on 250 kpps load:

Enable programm start on server startup, please add to /etc/rc.local this lines:

cd /root/fastnetmon && screen -S fastnetmon -d -m ./fastnetmon eth3,eth4

I recommend you to disable CPU freq scaling for gain max performance (max frequency): echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

You can use this script for irq balancing on heavy loaded networks:

#!/bin/bash

# from http://habrahabr.ru/post/108240/
ncpus=`grep -ciw ^processor /proc/cpuinfo`
test "$ncpus" -gt 1 || exit 1

n=0
for irq in `cat /proc/interrupts | grep eth | awk '{print $1}' | sed s/\://g`
do
    f="/proc/irq/$irq/smp_affinity"
    test -r "$f" || continue
    cpu=$[$ncpus - ($n % $ncpus) - 1]
    if [ $cpu -ge 0 ]
            then
                mask=`printf %x $[2 ** $cpu]`
                echo "Assign SMP affinity: eth queue $n, irq $irq, cpu $cpu, mask 0x$mask"
                echo "$mask" > "$f"
                let n+=1
    fi
done

You can find more info and graphics here

Author: Pavel Odintsov pavel.odintsov at gmail.com

Obsolet install guid in CentOS 6:

   # CentOS 6
   yum install -y git libpcap-devel gcc-c++ boost-devel boost make