« Return to Thread: Another question on data bulk loading

Another question on data bulk loading

by davidw589 :: Rate this Message:

Reply to Author | View in Thread

Hello,

 

I have researched bulk loading on

 http://pub.eigenbase.org/wiki/LucidDbFlatFileWrapper and

http://pub.eigenbase.org/wiki/LucidDbEtlTutorial#Define_Sources. As far as I see, they all need to run in lucidClient. Now I have the following files

 

subreport1-2008-06-05_09-10

subreport1-2008-06-05_09-20

subreport1-2008-06-05_09-30

subreport1-2008-06-05_09-40

subreport1-2008-06-05_09-50

subreport1-2008-06-05_10-00

subreport1-temp

 

subreport2-2008-06-05_09-00

subreport2-2008-06-05_09-10

subreport2-2008-06-05_09-20

subreport2-2008-06-05_09-30

subreport2-2008-06-05_09-40

subreport2-2008-06-05_09-50

subreport2-2008-06-05_10-00

subreport2-temp

 

subreport3-2008-06-05_09-00

subreport3-2008-06-05_09-10

subreport3-2008-06-05_09-20

subreport3-2008-06-05_09-30

subreport3-2008-06-05_09-40

subreport3-2008-06-05_09-50

subreport3-2008-06-05_10-00

subreport3-temp

 

The titles of these data file have this format "subreport#-timestamp". '#' means a number from 1 to 3 that means different types of data. Data is writing into these files on a real time basis. As the coming-in data is really huge, it is cut and saved into seperate files every ten minutes.

 

I need to load this data into corresponding tables on a real-time basis continuously at the back end. It means that no one comes to run those SQL commands anytime. All this bulk loading should be done automatically.

 

What are the steps to do this?

 

Thanks so much.

 

David

 

 

 

 

 



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Squirrel-sql-users mailing list
Squirrel-sql-users@...
https://lists.sourceforge.net/lists/listinfo/squirrel-sql-users

 « Return to Thread: Another question on data bulk loading

LightInTheBox - Buy quality products at wholesale price!