Another question on data bulk loading
|
View:
New views
4 Messages
—
Rating Filter:
Alert me
|
 |
Another question on data bulk loading
by davidw589
::
Rate this Message:
------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Squirrel-sql-users mailing list Squirrel-sql-users@... https://lists.sourceforge.net/lists/listinfo/squirrel-sql-users |
|
|
|
Re: Another question on data bulk loading
by Matthew Dahlman-2
::
Rate this Message:
David,
The functionality that you are describing isn't
something that SQuirreL is designed to do.
You'll have more success with an ETL (Extract Transform
Load) tool built for situation that you are describing. I can recommend Talend
as an open source ETL tool that I've used (http://www.talend.com/), but you'll find many
of them with a quick search.
Regards,
Matt From: squirrel-sql-users-bounces@... [mailto:squirrel-sql-users-bounces@...] On Behalf Of David W Sent: Thursday, 05 June, 2008 11:02 To: squirrel-sql-users@... Subject: [Squirrel-sql-users] Another question on data bulk loading
------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Squirrel-sql-users mailing list Squirrel-sql-users@... https://lists.sourceforge.net/lists/listinfo/squirrel-sql-users |
|
|
|
Re: Another question on data bulk loading
by shastzi
::
Rate this Message:
On Thursday 05 June 2008 2:01:48 pm David W wrote:
> Hello, > > I have researched bulk loading on > http://pub.eigenbase.org/wiki/LucidDbFlatFileWrapper and > > http://pub.eigenbase.org/wiki/LucidDbEtlTutorial#Define_Sources. As far as > I see, they all need to run in lucidClient. Now I have the following files > > subreport1-2008-06-05_09-10 > subreport1-2008-06-05_09-20 > subreport1-2008-06-05_09-30 > subreport1-2008-06-05_09-40 > subreport1-2008-06-05_09-50 > subreport1-2008-06-05_10-00 > subreport1-temp > > subreport2-2008-06-05_09-00 > subreport2-2008-06-05_09-10 > subreport2-2008-06-05_09-20 > subreport2-2008-06-05_09-30 > subreport2-2008-06-05_09-40 > subreport2-2008-06-05_09-50 > subreport2-2008-06-05_10-00 > subreport2-temp > > subreport3-2008-06-05_09-00 > subreport3-2008-06-05_09-10 > subreport3-2008-06-05_09-20 > subreport3-2008-06-05_09-30 > subreport3-2008-06-05_09-40 > subreport3-2008-06-05_09-50 > subreport3-2008-06-05_10-00 > subreport3-temp > > The titles of these data file have this format "subreport#-timestamp". '#' > means a number from 1 to 3 that means different types of data. Data is > writing into these files on a real time basis. As the coming-in data is > really huge, it is cut and saved into seperate files every ten minutes. > > I need to load this data into corresponding tables on a real-time basis > continuously at the back end. It means that no one comes to run those SQL > commands anytime. All this bulk loading should be done automatically. > > What are the steps to do this? > > Thanks so much. > > David > > > > > For things like that I use a little python program. KW ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Squirrel-sql-users mailing list Squirrel-sql-users@... https://lists.sourceforge.net/lists/listinfo/squirrel-sql-users |
|
|
|
Re: Another question on data bulk loading
by David Durham, Jr.
::
Rate this Message:
On Thu, Jun 5, 2008 at 12:01 PM, David W <davidw589@...> wrote:
> Hello, > > I have researched bulk loading on > > http://pub.eigenbase.org/wiki/LucidDbFlatFileWrapper and > > http://pub.eigenbase.org/wiki/LucidDbEtlTutorial#Define_Sources. As far as I > see, they all need to run in lucidClient. Now I have the following files If you know Java and something like Hibernate, I wrote something that could be useful to you.. It will produce Java Beans from flat files. In fact, if you construct the csv, or excel file properly, you don't have to write much code at all. Here's an example: http://code.google.com/p/beanfiles/wiki/Examples -Dave ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Squirrel-sql-users mailing list Squirrel-sql-users@... https://lists.sourceforge.net/lists/listinfo/squirrel-sql-users |
| Free Forum Powered by Nabble | Forum Help |