Graphing Sparse Data

---

Sponsored by Catholic Health Initiatives and Immanuel Health Systems, Alegent Health is faithful to the healing ministry of Jesus Christ, providing high quality care for the body, mind and spirit of every person.

The information contained in this communication, including attachments, is confidential and private and intended only for the use of the addressees. Unauthorized use, disclosure, distribution or copying is strictly prohibited and may be unlawful. If you received this communication in error, please inform us of the erroneous delivery by return e-mail message from your computer. Additionally, although all attachments have been scanned at the source for viruses, the recipient should check any attachments for the presence of viruses before opening. Alegent Health accepts no liability for any damage caused by any virus transmitted by this e-mail. Thank you for your cooperation.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Laack,Jacob C wrote:

>I am monitoring many services with Nagios and the data ends up in
>RRDs which I graph using pnp.  (I promise this is an rrd question.)
>One type of service is only checked every 6 hours (instead of the
>normal 5 minutes) so only 4 pieces of data are put into the RRD file
>a day.  rrdgraph shows empty graphs for 4 and 24 hours since the
>data is so sparse.  Would it be possible to use CDEF (or some other
>rrdgraph magic) so that if the current plot point contains no data,
>it looks ahead into the RRD file and uses that data?

Lets clarify what you mean/want ...

Firstly, if we assume you are sampling at midnight, 6am, midday, and
6pm, and you are graphing at 3pm - what do you get ? what do you want
?

If you are getting a graph that shows some data at midnight, 6am, and
midday - but gaps in between, then that's a matter of adjusting your
rrd to allow for the sampling you are doing.

If your graph shows blocks from midnight to 6am, 6am to midday, but
you want something drawn from midday to 3pm - then you can't have it
as the period midday to 6pm has not happened yet and so cannot be
drawn.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Simon-
The RRD data is compiled in 4 hour, 24 hour, 1 week, 1 month and 1 year durations.  The data is plotted about every 6 hours rather than at specific times.  That's just how nagios works.  All of my graphs for the above durations are blank.  If I force a check (manually get a data point) a couple times, I get a small pixel or two long line.  But the next day, my graphs have no dots or lines afterwards even though 2 or 3 points of data are in there somewhere.

I thought about your last paragraph...  If my checks are 6 hours apart and my shortest RRD is 4 hours, wouldn't that pretty much keep the RRD's empty most of the time?  If that is a stupid question, my goal would be to simply draw lines between the points.

I'm going to make my polling interval 2 hours and see what kind of graphs I get.  Thanks for your insight.

-----Original Message-----
From: rrd-users-bounces@... [mailto:rrd-users-bounces@...] On Behalf Of Simon Hobson
Sent: Wednesday, May 14, 2008 10:19 AM
To: rrd-users@...
Subject: Re: [rrd-users] Graphing Sparse Data

Laack,Jacob C wrote:

>I am monitoring many services with Nagios and the data ends up in
>RRDs which I graph using pnp.  (I promise this is an rrd question.)
>One type of service is only checked every 6 hours (instead of the
>normal 5 minutes) so only 4 pieces of data are put into the RRD file
>a day.  rrdgraph shows empty graphs for 4 and 24 hours since the
>data is so sparse.  Would it be possible to use CDEF (or some other
>rrdgraph magic) so that if the current plot point contains no data,
>it looks ahead into the RRD file and uses that data?

Lets clarify what you mean/want ...

Firstly, if we assume you are sampling at midnight, 6am, midday, and
6pm, and you are graphing at 3pm - what do you get ? what do you want
?

If you are getting a graph that shows some data at midnight, 6am, and
midday - but gaps in between, then that's a matter of adjusting your
rrd to allow for the sampling you are doing.

If your graph shows blocks from midnight to 6am, 6am to midday, but
you want something drawn from midday to 3pm - then you can't have it
as the period midday to 6pm has not happened yet and so cannot be
drawn.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users


Sponsored by Catholic Health Initiatives and Immanuel Health Systems, Alegent Health is faithful to the healing ministry of Jesus Christ, providing high quality care for the body, mind and spirit of every person.

The information contained in this communication, including attachments, is confidential and private and intended only for the use of the addressees.  Unauthorized use, disclosure, distribution or copying is strictly prohibited and may be unlawful.  If you received this communication in error, please inform us of the erroneous delivery by return e-mail message from your computer.  Additionally, although all attachments have been scanned at the source for viruses, the recipient should check any attachments for the presence of viruses before opening.  Alegent Health accepts no liability for any damage caused by any virus transmitted by this e-mail.  Thank you for your cooperation.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Laack,Jacob C wrote:

>The RRD data is compiled in 4 hour, 24 hour, 1 week, 1 month and 1
>year durations.

That doesn't matter !

>The data is plotted about every 6 hours rather than at specific
>times.  That's just how nagios works.  All of my graphs for the
>above durations are blank.  If I force a check (manually get a data
>point) a couple times, I get a small pixel or two long line.  But
>the next day, my graphs have no dots or lines afterwards even though
>2 or 3 points of data are in there somewhere.

Your heartbeat setting is wrong.

>I thought about your last paragraph...  If my checks are 6 hours
>apart and my shortest RRD is 4 hours, wouldn't that pretty much keep
>the RRD's empty most of the time?

No, it doesn't matter.

How did you define your RRDs ? There are TWO particular values you
need to be aware of :

Firstly, the step setting for the whole RRD :
>  --step|-s step (default: 300 seconds)
>  Specifies the base interval in seconds with which data will be fed
>into the RRD.

This is NOT when you have to feed it data, but it is the fixed length
of the 'buckets' where the data will be stored.

Then for each DS you have a heartbeat setting :
>  DS:ds-name:GAUGE | COUNTER | DERIVE | ABSOLUTE:heartbeat:min:max

>heartbeat defines the maximum number of seconds that may pass
>between two updates of this data source before the value of the data
>source is assumed to be *UNKNOWN*.


Firstly, if we just assume heartbeat is long enough, if you only feed
data in at long periods, then rrdtool will simply normalise that data
across the time since the previous update. So for example, suppose
your step is 1 hour and your DS type is type Counter. You update at
00:00 with a value of zero, then at 04:00 with a value of 28,800
(that's 2x4x3600). The rate is simply 28,8400 / 14,400 (quantity /
time) to give you a rate of 2/second. If you look in your database
(with rrdtool fetch for example) then you will see that the values
for 01:00, 02:00, 03:00, 04:00 will all be set to 2.000. The value at
00:00 will depend on what was fed in beforehand.

If however your step is 3 hours, then after the same updates, the
value at 03:00 would still be 2.000, but the next step at 06:00 would
be NaN as it can't be calculated yet. If you update again on or after
06:00 with the same value of 28,800 then you will find that the rate
for 06:00 will now be 0.6667. This is derived from 1hour (the 03:00
to 04:00 worth of your previous update) at rate 2 plus 2 hours (04:00
to 06:00) at rate 0.

That is the normalisation part and is explained fully in the tutorials.


Now, in your case, you are updating at relatively long intervals -
this is where heartbeat comes in. I suspect that your heartbeat is
quite small - so if your updates are (say) 6 hours apart, then the
time between updates is longer than heartbeat and the program assumes
you have missing data and so the intervening period has an unknown
value. You probably need to set your heartbeat at something like 8
hours.

Be aware however that this means you could miss data and not have it
show up in the graphs.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users