nfdump & nfsen

В последней версии моей любимой Vyatta (6.0) появилась возможность собирать netflow.
И ради интереса решил испробовать как это всё будет.
В Vyatta все просто:

flow-accounting {

interface eth0

interface eth1

interface eth2

netflow {

server 192.168.ччч.ччч {

port 9997

}

timeout {

expiry-interval 600

max-active-life 172800

Последнии три строки можно не писать.
Сервачок мой для опытов на котором развёрнут Zabbix (192.168.ччч.ччч),был улучшен связкой nfdump с nfsen. Первое без второго работает, но второе без первого нет.
nfdump - это коллектор netflow, прекрасно работает сам но только операции все через командную строку надо выполнять. Но раз есть такая штука как nfsen решил их слепить.
nfsen - проще описать как GUI к nfdump с возможностью запуска последнего из своего *.conf файла. На просторах Internet'а очень много ресурсов где описано как это делать, но к моему сожалению ни один до конца мне не помог.
С nfdump нет проблем он ставиться из репозитария.

sudo apt-get install nfdump

Так как на серваке у меня стоит Zabbix, то apache мне савить не надо но на всякий пож:

sudo apt-get install apache2 libapache2-mod-php5 php5-common libmailtools-perl rrdtool librrds-perl

Дальше качаем сам nfsen:

cd /home/xxx/distr/

wget http://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.3/nfsen-1.3.3.tar.gz/download

gzip -dc nfsen-1.3.2.tar.gz | tar -xf -

cd nfsen-1.3.3

Создаем папку возможно в более привычном месте:

sudo mkdir /etc/nfsen/

Копируем туда conf файл и редактируем:

cp etc/nfsen-dist.conf /etc/nfsen/nfsen.conf 

sudo vi /etc/nfsen.conf

Там правим примерно вот что:

# NfSen default layout:
# Any scripts, modules or profiles are installed by default under $BASEDIR.
# However, you may change any of these settings to fit your requested layout.

#
# Required for default layout
$BASEDIR = "/usr/local/nfsen";

#
# Where to install the NfSen binaries
$BINDIR="${BASEDIR}/bin";

#
# Where to install the NfSen Perl modules
$LIBEXECDIR="${BASEDIR}/libexec";

#
# Where to install the config files
$CONFDIR="/etc/nfsen/";

#
# NfSen html pages directory:
# All php scripts will be installed here.
# URL: Entry point for nfsen: http:///nfsen/nfsen.php
$HTMLDIR    = "/var/www/nfsen/";

#
# Where to install the docs
$DOCDIR="${HTMLDIR}/doc";

#
# Var space for NfSen
$VARDIR="/var/local/nfsen";

# directory for all pid files
# $PIDDIR="$VARDIR/run";
#
# Filter directory
# FILTERDIR="${VARDIR}/filters";
#

# FORMATDIR for custom printing formats
# FORMATDIR="${VARDIR}/fmt";
#

#
# The Profiles stat directory, where all profile information
# RRD DBs and png pictures of the profile are stored
$PROFILESTATDIR="${VARDIR}/profiles-stat";

#
# The Profiles directory, where all netflow data is stored
$PROFILEDATADIR="/var/local/nfdump";

#
# Where go all the backend plugins
$BACKEND_PLUGINDIR="${BASEDIR}/plugins";

#
# Where go all the frontend plugins
$FRONTEND_PLUGINDIR="${HTMLDIR}/plugins";

#
# nfdump tools path
$PREFIX  = "/usr/bin";

#
# nfsend communication socket
# $COMMSOCKET = "$PIDDIR/nfsen.comm";

# BASEDIR unrelated vars:
#
# Run nfcapd as this user
# This may be a different or the same uid than your web server.
# Note: This user must be in group $WWWGROUP, otherwise nfcapd
#       is not able to write data files!
$USER    = "www-data";

# user and group of the web server process
# All netflow processing will be done with this user
$WWWUSER  = "www-data";
$WWWGROUP = "www-data";

# Receive buffer size for nfcapd - see man page nfcapd(1)
$BUFFLEN = 200000;

# list of extensions for each collector. See argument -T
# for nfcapd(1) for more detailes.
# defaults to empty -> compatible to nfdump-1.5.8
# $EXTENSIONS = '';
# Example:
# $EXTENSIONS = 'all';
# $EXTENSIONS = '+3,+4';
#
# Directory sub hierarchy layout:
# Possible layouts:
#
# 0 default     no hierachy levels - flat layout - compatible with pre NfSen versions
# 1 %Y/%m/%d    year/month/day
# 2 %Y/%m/%d/%H year/month/day/hour
# 3 %Y/%W/%u    year/week_of_year/day_of_week
# 4 %Y/%W/%u/%H year/week_of_year/day_of_week/hour
# 5 %Y/%j       year/day-of-year
# 6 %Y/%j/%H    year/day-of-year/hour
# 7 %Y-%m-%d    year-month-day
# 8 %Y-%m-%d/%H year-month-day/hour
$SUBDIRLAYOUT = 7;

# Compress flows while collecting 0 or 1
$ZIPcollected    = 1;

# Compress flows in profiles 0 or 1
$ZIPprofiles     = 1;

# number of nfprofile processes to spawn during the profiling phase
# depends on how busy your system is and how many CPUs you have
# on very busy systems increase it to a higher value
$PROFILERS = 3;

# if the PROFILEDATADIR is filled up to this percentage, a warning message will be printed.
# set to 0 to disable the test
$DISKLIMIT = 98;

# number of nfprofile processes to spawn during the profiling phase
$PROFILERS = 6;

# Netflow sources
# Define an ident string, port and colour per netflow source
#
# Required parameters:
#    ident   identifies this netflow source. e.g. the router name,
#            Upstream provider name etc.
#    port    nfcapd listens on this port for netflow data for this source
#                        set port to '0' if you do not want a collector to be started
#    col     colour in nfsen graphs for this source
#
# Optional parameters
#    type    Collector type needed for this source. Can be 'netflow' or 'sflow'. Default is netflow
#        optarg  Optional args to the collector at startup
#
# Syntax:
#         'ident' => { 'port' => '', 'col' => '', 'type' => '' }
# Ident strings must be 1 to 19 characters long only, containing characters [a-zA-Z0-9_].

%sources = (
          'vyatta-vpn'   => { 'port' => '9995', 'col' => '#ff0000', 'type' => 'netflow' },
          'vyatta'  => { 'port' => '9997', 'col' => '#0000ff', 'type' => 'netflow' },
);

#
# Low water mark: When expiring files, delete files until
# size = $low_water % of max_size
# typically 90
$low_water = 90;

#
# syslog facility for periodic jobs
# nfsen uses level 'debug', 'info', 'warning' and 'err'
# Note: nfsen is very chatty for level 'debug' and 'info'
# For normal operation, you may set the logging level in syslog.conf
# to warning or error unless you want to debug NfSen
$syslog_facility = 'local3';

#
# SYSLOG mess
# Log socket type: Most *NIX such as LINUX and *BSD are fine with 'unix'
# which is the default. You need to change that to 'stream' or 'inet' for
# some Solaris version 8/9, AIX and others ..
# You may set it to undef to prevent calling Sys::Syslog::setlogsock at all
# ( works for Solaris 10 and newer Sys::Syslog module
#
# If not defined at all, 'unix' is assumed unless for Solaris, which defaults to 'stream'
# $LogSocket = 'unix';

#
# Plugins
# Plugins extend NfSen for the purpose of:
# Periodic data processing, alerting-condition and alerting-action
# For data processing a plugin may run for any profile or for a specific profile only.
#     Syntax: [ 'profile list', 'module' ]
#            profile list:  ',' separated list of profiles ( 'profilegroup/profilename' ),
#                           or '*' for any profile, '!' for no profile
#            module:        Perl Module name, equal to plugin name
# The profile list '!' make sense for plugins, which only provide alerting functions
#
# The module follows the standard Perl module conventions, with at least one
# function: Init(). See demoplugin.pm for a simple template.
#
# A file with the same name in the FRONTEND_PLUGINDIR and .php extension is automatically
# recongized as frontend plugin.
#
# Plugins are installed under
# $BACKEND_PLUGINDIR and $FRONTEND_PLUGINDIR

@plugins = (
    # profile    # module
    # [ '*',     'demoplugin' ],
);

%PluginConf = (
        # For plugin demoplugin
        demoplugin => {
                # scalar
                param2 => 42,
                # hash
                param1 => { 'key' => 'value' },
        },
        # for plugin otherplugin
        otherplugin => [
                # array
                'mary had a little lamb'
        ],
);

#
# Alert module: email alerting:
# Use this from address
$MAIL_FROM   = 'your@from.example.net';

# Use this SMTP server
$SMTP_SERVER = 'localhost';

# Use this email body:
# You may have multiple lines of text.
# Var substitution:
# @alert@               replaced by alert name
# @timeslot@    replaced by timeslot alert triggered
$MAIL_BODY       = q{
Alert '@alert@' triggered at timeslot @timeslot@
};

######################################################
#
# For the NfSen simulator include the section below.
#
######################################################
#
# Nfsen Simulator
# The simulator requires, that you have already installed
# and configured NfSen. The simulation is based on already
# pre-colleted data, which you may get from another live
# NfSen system.
#
# Steps to setup the NfSen simulator:
# 1. Configure the sources of the live profile with the
#    same names of the NfSen system, you take netflow data
#    for the simulation. Set the port for each netflow source
#    to 0 to prevent a collector to be started.
#    Install NfSen with this config in a seperate directory
# 2. Copy the pre-collected data into the appropriate
#    netflow directory of the live profile.
# 3. Configure the simulator using the parameters below
#    Enable Simulation mode => $SIMmode = 1
#    Configure the time window of the pre-collected data.
#      tstart    => Start of time window. yyyymmddhhmm
#      tbegin    => Optional parameter. Start of simulation
#                   profile exists already between tstart - tbegin
#      tend      => End of time window. yyyymmddhhmm
#      cycletime => simulation time in seconds of a 5min slot
#    Setting cycletime = 0 processes the cycles as fast as
#    possible. Please note, if you test plugings, your
#    cycletime needs to be at least the time required to
#    process all plugins.
# 4. Start nfsen: ../nfsen start
#    Simulation starts
#
# The simulator runs from tstart to tend and stops when tend
# is reached. You may stop the simulation at any given time
# using ./nfsen stop. To continue the simulation start NfSen
# again: ./nfsen start. You may reset the simulator at any
# given time using ./nfsen abort-reset. This stops the sumulation
# and rolls back to tstart. All profiles/alerts are deleted,
# so you may start from scratch again.
#
# Configure simulator parameters
#
# $SIMmode = 1;
# %sim = (
#    'tstart'     => '200707100000',    # Simulation data available from July 10th 2007 00:00
#    'tbegin'     => '200707110000',    # Simulation begins at July 11th 2007 00:00
#    'tend'       => '200707112355',    # Simulation ends at July 11th 2007 23:55
#    'cycletime'  => '30',                              # 30s per 5min slot
# );

А формат папок nfsen с сайта для тех кому важно что-где лежит:

До этого момента во всех блогах одно и тоже (ну или примерно одно и тоже)
дальше интереснее.

У меня начались сразу проблемы ))

RRDtool у меня был 1.4, с которым nfsen 1.3.3 не работает. Идём
libexec/NfSenRRD.pm и правим строку

( $rrd_version >= 1.2 && $rrd_version < 1.3 ) 

на

( $rrd_version >= 1.2 && $rrd_version < 1.5 ) 

Все вроди как OK далаем:

sudo ./install.pl /etc/nfsen/nfsen.conf 

ан нет, не все гладко.

Далее из переписки следовало, что надо заменить в install.pl все

make_path  

на

mkpath

и закомментировать строку:

CopyRecursive::dircopy("doc", "$NfConf::INSTPREFIX$NfConf::DOCDIR");

далаем:

sudo ./install.pl /etc/nfsen/nfsen.conf

cd /usr/local/nfsen/bin

sudo ./nfsen start

Все должно пройти нормально, идём на

http://ваш.сервер.com/nfsen/nfsen.php

А дальше у кого как...
У кого все в порядке, а у кого нет, потому что у некоторых стоит php5.2, а у некоторых
5.3. Для последних к коим я отношусь так же надо поколдовать дальше.
В папке web лица - /var/www/nfsen есть файлик nfsenutil.php.
В нём меняем

split 

на

preg_split

и в скобках вместо

' '

'/ /'

Тоже самое в rrdgraph.php там же.
И теперь все должно получиться. Смотрим и радуемся.

Здесь можно посмотреть nfsen во всей красе!

http://nfsen.its.monash.edu/nfsen/nfsen.php