www.delorie.com/gnu/docs/cfengine/cfengine-Anomalies_4.html   search  
 
Buy GNU books!


Anomaly detection with cfenvd and cfenvgraph

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

1.3 cfenvgraph

The `cfenvgraph' command can use used to dump a graph of averages for visual inspection of the normal state database. The format of the file is
 
t,y_1,y_2,y_3...
which can be viewed using `gnuplot' or `xgmr' or other graphical plotting program. This would allow the policy-maker to see what is likely to be a good time for such work (say 06:00 hours), and then use this time for the job, unless an anomalous load is detected.

The cfenvgraph command is used to extract data from the database used by the cfenvd environment daemon.
 
cfenvgraph -f filename.db [-r -T -t -s -e]
The command normally generates two files with format
 
t, y_1, y_2, y_3, y_4...
in a sub-directory of the current directory `cfenvgraphs-snapshot' (or `cfenvgraphs-'TIMESTAMP if `-T' is used).

The files are called
 
cfenv-average
cfenv-stddev
and contain, respectively, the weighted average values of all the recorded data and the square-root of the weighted variances with respect to the averages. Data are weighted in such a way that older values are gradually deprecated, becoming irrelevant after about two months.

Normally the vertical scale of each graph is scaled so that each line has a maximum value of 1 and a minimum value of 0, this allows all the lines to be seen in maximum detail. However, this makes it difficult to see the absolute values of the lines. With the `-n' option, no scaling is performed and true values are plotted.

The complete data span a one-week period, and the daily rhythm of the system may normally be viewed as a number of peaks, one per day.

The options are:

`--help (-h)'
List command options
`--file (-f)'
Specifiy file to plot.
`--titles (-t)'
If the `-t' option is given, comments are generated at the start of the file which describe the columns. These are in a format understood by `vvgraph' as title/label data.
`--timestamps (-T)'
If the `-T' option is given, the output filenames are time-stamped with the current time, in order to give a unique name.
`--resolution (-r)'
If the `-r' option is given then high resolution data are generated (five minute resolution), otherwise data are averaged over periods of one hour to generate simpler and smoother graphs.
`--separate (-s)'
If the `-s' option is given, cfenvgraph generates separate files for each metric, in the format
 
t,y,dy
where dy is the height of a vertical error-bar. This set of graphs combines the average with the standard-deviation. (Note that the error bars show the standard-deviation, and not the standard error of the mean i.e. stddev/sqrt(N)); the latter has no obvious meaning here. If `-e' is specified, then error bars are omitted.
`--no-error-bars (-e)'
No error bars are plotted.
`--no-scaling (-n)'
The graphs are not scaled, so that (min,max) is mapped onto the interval (0,1).

Note that the values printed for sockets always look higher than they should for highly active services. This is because even those sockets which are in CLOSE_WAIT are counted. This is the correct way to determine a normal state based on the recent past. It is a local averaging performed by the kernel. If one counts only those connections which are currently active, one gets a distorted view of activity with a 5-minute sample rate. To measure more often than this would place unacceptably high load on the system.

Graphs may be viewed in `vvgraph', `xmgr' (used in the pictures above) or `gnuplot', or other graphical viewer. These graphs are not meant for continuous viewing. The data are averages, not time-series.

For example, with gnuplot

 
host$ cfenvgraph -s
host$ gnuplot
gnuplot> plot "www-in.cfenv" with errorbars
gnuplot> plot "www-in.cfenv" with lines


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

  webmaster   donations   bookstore     delorie software   privacy  
  Copyright 2003   by The Free Software Foundation     Updated Jun 2003