|
»
»
»
importing large dataset
|
View:
New views
4 Messages
—
Rating Filter:
Alert me
|
 |
importing large dataset
by Mag Gam
::
Rate this Message:
I would like to import a very large dataset lets say 100MB text file. The file consists like this.
Date Value 2/3/03 11:00 10 2/3/03 11:01 12 2/3/03 11:02 13 2/3/03 11:03 30 2/3/03 11:04 12 2/3/03 11:05 13 It keeps going on. I can generate the epoch on the left side. However, I am unclear how to load this into rrdtool. I have followed the tutorial and I was not able to model my data properly. Can someone please show me an example for the data above? TIA _______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
|
|
Re: importing large dataset
by Alex van den Bogaerdt
::
Rate this Message:
On Sat, Apr 26, 2008 at 09:30:03AM -0400, Mag Gam wrote:
> I would like to import a very large dataset lets say 100MB text file. The > file consists like this. > > Date Value > 2/3/03 11:00 10 > 2/3/03 11:01 12 > 2/3/03 11:02 13 > 2/3/03 11:03 30 > 2/3/03 11:04 12 > 2/3/03 11:05 13 > > It keeps going on. I can generate the epoch on the left side. However, I am > unclear how to load this into rrdtool. I have followed the tutorial and I > was not able to model my data properly. Can someone please show me an > example for the data above? What do these numbers mean? When were they valid? RRDtool is not a graphing program. Use e.g. gnuplot for that. If these numbers are rates, you still need to know when these rates were valid. For instance: "12", does this mean a rate between "2/3/03 11:00" and "2/3/03 11:01" or between "2/3/03 11:01" and "2/3/03 11:02"? A small but significant difference. RRDtool uses end times (thus: updating for 2/3/03 11:01 is updating the interval upto and including 2/3/03 11:01). Are these times in some local time or in UTC? If they are in local time, make sure to compensate for this and don't forget about daylight saving (if any). You will need to convert each pair into <timestamp>:<value>, and then give those as input to rrdtool. This is a *very* basic operation, so perhaps you need to explain the problem you encounter a bit more. -- Alex van den Bogaerdt http://www.vandenbogaerdt.nl/rrdtool/ _______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
|
|
Re: importing large dataset
by Mag Gam
::
Rate this Message:
Alex,
Thanks for getting back to me. These are actually rates. It is a CPU utilization rate. These values are point in time. So, at 2/3/03 11:00 CPU utilization is at 10% Does that help for clarification? On Sat, Apr 26, 2008 at 9:57 AM, Alex van den Bogaerdt <alex@...> wrote:
_______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
|
|
Re: importing large dataset
by Alex van den Bogaerdt
::
Rate this Message:
On Sat, Apr 26, 2008 at 10:18:12AM -0400, Mag Gam wrote:
> > > I would like to import a very large dataset lets say 100MB text file. > > The > > > file consists like this. > > > > > > Date Value > > > 2/3/03 11:00 10 > > > 2/3/03 11:01 12 > > > 2/3/03 11:02 13 > > > 2/3/03 11:03 30 > > > 2/3/03 11:04 12 > > > 2/3/03 11:05 13 > > > > > > It keeps going on. I can generate the epoch on the left side. However, I > > am > > > unclear how to load this into rrdtool. I have followed the tutorial and > > I > > > was not able to model my data properly. Can someone please show me an > > > example for the data above? > > > > What do these numbers mean? > > When were they valid? > > > > > > RRDtool is not a graphing program. Use e.g. gnuplot for that. > > > > If these numbers are rates, you still need to know when these rates > > were valid. For instance: "12", does this mean a rate between > > "2/3/03 11:00" and "2/3/03 11:01" or between > > "2/3/03 11:01" and "2/3/03 11:02"? A small but significant difference. > > RRDtool uses end times (thus: updating for 2/3/03 11:01 is updating the > > interval upto and including 2/3/03 11:01). > > > > Are these times in some local time or in UTC? If they are in local time, > > make sure to compensate for this and don't forget about daylight saving > > (if any). > > > > You will need to convert each pair into <timestamp>:<value>, and then > > give those as input to rrdtool. This is a *very* basic operation, so > > perhaps you need to explain the problem you encounter a bit more. > Alex, > > Thanks for getting back to me. > > These are actually rates. It is a CPU utilization rate. These values are > point in time. So, at 2/3/03 11:00 CPU utilization is at 10% > Does that help for clarification? Let me rephrase: a rate over time, thus "in the time interval between 11:00 and 11:01, the rate was <something> units per second". This rate is thus valid between 11:00:00 and 11:00:01, between 11:00:01 and 11:00:02, ... 11:00:59 and 11:01:00 This is what RRDtool does. It doesn't do points in time, it is not a program which draws a nice picture out of an arbitrary set of numbers. If you want to pretend your data was valid during a whole minute, you are free to do so. Just setup a GAUGE data source and update the database with each of your time/value pairs. Of course you also need to setup one or more (probably more!) RRAs when you run 'rrdtool create'. Make sure you understand RRAs and consolidation, have a look at my website if the tutorial wasn't enough. HTH and AFK. -- Alex van den Bogaerdt http://www.vandenbogaerdt.nl/rrdtool/ _______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
| Free Forum Powered by Nabble | Forum Help |