large database files

[Sean Allen-5]

i'm starting to do some serious playing around with zodb.

step 1, move about 10% of our data over to get an idea about speed of  
searching 1.5 million entries etc.
standard file storage... the csv file i dumped all the info to is  
about 190 megs, the resulting file
storage file is 750 megs ( 1.8 gigs before packing ). Does this mean,  
that I should expect the entire database
to close in at something like 7.5 gigs? If yes, are there problems  
that can arise for a database of that size?

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev

[Benji York-2]

On Wed, Jul 23, 2008 at 2:01 AM, Sean Allen <sean@...> wrote:
> step 1, move about 10% of our data over to get an idea about speed of
> searching 1.5 million entries etc.
> standard file storage... the csv file i dumped all the info to is about 190
> megs, the resulting file
> storage file is 750 megs ( 1.8 gigs before packing ). Does this mean, that I
> should expect the entire database
> to close in at something like 7.5 gigs?

Not a bad guess.

> If yes, are there problems that can arise for a database of that size?

You should be fine.  That's a small database by some standards.
--
Benji York
Senior Software Engineer
Zope Corporation
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev

[Benji York-2]

On Wed, Jul 23, 2008 at 2:01 AM, Sean Allen <sean@...> wrote:
> i'm starting to do some serious playing around with zodb.
>
> step 1, move about 10% of our data over to get an idea about speed of
> searching 1.5 million entries etc.
> standard file storage... the csv file i dumped all the info to is about 190
> megs, the resulting file
> storage file is 750 megs ( 1.8 gigs before packing ). Does this mean, that I
> should expect the entire database

(After reading your other message about indexing); if you aren't doing
any indexing yet, then you should expect a larger database once the
indexes are added.

Some advocate keeping indexes in a separate database, especially if they
are a significant fraction of the total database size.  That way you can
have different packing schedules, backup plans, storage devices, etc.
--
Benji York
Senior Software Engineer
Zope Corporation
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev

[Sean Allen-5]

On Jul 23, 2008, at 8:02 AM, Benji York wrote:

> (After reading your other message about indexing);

got any advice on that one?

are the pages i've found for things released for zodb 3.3 really
the ones i want to be using? are there newer indexing tools
i should be looking at?

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev