Advice for people with large mailboxers

[Jones B.]

Hi all,

I'm responding back about a thread Tomas and I had going, titled:

> Subject: Re: [SM-USERS] How many messages is squirrelmail designed to handle?
> Dates:
  Tue, 29 Apr 2008 04:11:25 +0000 (UTC)
+ 28 april.


Anyhow, Tomas recommended a few things with people who have a large mailbox and who access it via squirrelmail. You may refer to the original thread, which lists some fantastic ideas, mainly:
 

* enable sort mailbox to load larger messages.
** Use date/size sorting if you can [compared to subject and from/to]


I wonder:

Would moving messages from the inbox folder to another folder be a piece of advice one should follow? Some users I know time out via imap squirrelmail webmail and I have seen the suggestion that one should move mail to another squirrelmail folder and just view new messages in the inbox. Well, while that allows a person to view their inbox all swell and dandy when new messages come in, if they ever view that archive folder where they moved mail to, won't they experience the same problems of timing out when viewing it. Right?

I'm also concerned that even if they move mail messages to another folder, they still won't be able to do search queries on their messages, since those queries on that large collection will still time out since the folder is big.

--
______________________________________________
IndiaInfo Mail - the free e-mail service with a difference! www.indiainfo.com
Check out our value-added Premium features, such as an extra 20MB for mail storage, POP3, e-mail forwarding, and ads-free mailboxes!

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
-----
squirrelmail-users mailing list
Posting guidelines: http://squirrelmail.org/postingguidelines
List address: squirrelmail-users@...
List archives: http://news.gmane.org/gmane.mail.squirrelmail.user
List info (subscribe/unsubscribe/change options): https://lists.sourceforge.net/lists/listinfo/squirrelmail-users

[Fredrik Jervfors-4]

> I'm responding back about a thread Tomas and I had going, titled:
>
>> Subject: Re: [SM-USERS] How many messages is squirrelmail designed to
>> handle? Dates:
>>
> Tue, 29 Apr 2008 04:11:25 +0000 (UTC)
> + 28 april.
>
> Anyhow, Tomas recommended a few things with people who have a large
> mailbox and who access it via squirrelmail. You may refer to the original
> thread, which lists some fantastic ideas, mainly:
>
> * enable sort mailbox to load larger messages.
> ** Use date/size sorting if you can [compared to subject and from/to]
>
> I wonder:
>
> Would moving messages from the inbox folder to another folder be a piece
> of advice one should follow? Some users I know time out via imap
> squirrelmail webmail and I have seen the suggestion that one should move
> mail to another squirrelmail folder and just view new messages in the
> inbox.

Moving mails out of the inbox is generally a good advice. Depending on how
the IMAP server stores the mails, a large inbox might even be an
invitation to mailbox corruption and data loss, and/or slow mail delivery
down.

Compare with how you handle paper mail: you're not likely to keep all your
mails in a big box. People usually throw unneeded mails in the trash and
sort the important mails into binders/folders/drawers. With a good folder
structure you're more likely to find what you're looking for without
having to do a system-wide search, just by manually browser through your
folders.

I'm sure there's plenty on advice on how to structure your folders, so use
your favourite search engine and read up on the subject.

> Well, while that allows a person to view their inbox all swell and
> dandy when new messages come in, if they ever view that archive folder
> where they moved mail to, won't they experience the same problems of
> timing out when viewing it. Right?

Time-outs are usually related to the size of the mailbox, so just make
sure that every folder doesn't grow too much. It's hard to recommend a
limit though, since that depends on your system. You might want to do some
trial and error testing to find out what works on your system. Also keep
in mind that server side sorting usually is more efficient than client
side sorting when using web clients such as SquirrelMail.

> I'm also concerned that even if they move mail messages to another
> folder, they still won't be able to do search queries on their messages,
> since those queries on that large collection will still time out since
> the folder is big.

You have the same problem even if you keep all the mails in the inbox, so
there's no difference, except that if you have mails in folders you can
search the folders one by one (if needed). And it's possible to do a
search in all folders at once if you want to (with the caveat previously
mentioned).

Another tip is to use a IMAP server with good indexing. That will speed
searches up and reduce the risks for time-outs.

Sincerely,
Fredrik

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
-----
squirrelmail-users mailing list
Posting guidelines: http://squirrelmail.org/postingguidelines
List address: squirrelmail-users@...
List archives: http://news.gmane.org/gmane.mail.squirrelmail.user
List info (subscribe/unsubscribe/change options): https://lists.sourceforge.net/lists/listinfo/squirrelmail-users

[Paul Lesniewski]

On Wed, Jun 11, 2008 at 9:05 AM, Fredrik Jervfors
<jervfors@...> wrote:

>> I'm responding back about a thread Tomas and I had going, titled:
>>
>>> Subject: Re: [SM-USERS] How many messages is squirrelmail designed to
>>> handle? Dates:
>>>
>> Tue, 29 Apr 2008 04:11:25 +0000 (UTC)
>> + 28 april.
>>
>> Anyhow, Tomas recommended a few things with people who have a large
>> mailbox and who access it via squirrelmail. You may refer to the original
>> thread, which lists some fantastic ideas, mainly:
>>
>> * enable sort mailbox to load larger messages.
>> ** Use date/size sorting if you can [compared to subject and from/to]
>>
>> I wonder:
>>
>> Would moving messages from the inbox folder to another folder be a piece
>> of advice one should follow? Some users I know time out via imap
>> squirrelmail webmail and I have seen the suggestion that one should move
>> mail to another squirrelmail folder and just view new messages in the
>> inbox.
>
> Moving mails out of the inbox is generally a good advice. Depending on how
> the IMAP server stores the mails, a large inbox might even be an
> invitation to mailbox corruption and data loss, and/or slow mail delivery
> down.
>
> Compare with how you handle paper mail: you're not likely to keep all your
> mails in a big box. People usually throw unneeded mails in the trash and
> sort the important mails into binders/folders/drawers. With a good folder
> structure you're more likely to find what you're looking for without
> having to do a system-wide search, just by manually browser through your
> folders.
>
> I'm sure there's plenty on advice on how to structure your folders, so use
> your favourite search engine and read up on the subject.

I couldn't agree more.  If you have a user keeping all their mails in
the INBOX, then they need to be educated about how to organize them
better.  That aside, note that in our development branch, login times
are reduced noticeably due to a better header-caching algorithm.

>> Well, while that allows a person to view their inbox all swell and
>> dandy when new messages come in, if they ever view that archive folder
>> where they moved mail to, won't they experience the same problems of
>> timing out when viewing it. Right?
>
> Time-outs are usually related to the size of the mailbox, so just make
> sure that every folder doesn't grow too much. It's hard to recommend a
> limit though, since that depends on your system. You might want to do some
> trial and error testing to find out what works on your system. Also keep
> in mind that server side sorting usually is more efficient than client
> side sorting when using web clients such as SquirrelMail.
>
>> I'm also concerned that even if they move mail messages to another
>> folder, they still won't be able to do search queries on their messages,
>> since those queries on that large collection will still time out since
>> the folder is big.
>
> You have the same problem even if you keep all the mails in the inbox, so
> there's no difference, except that if you have mails in folders you can
> search the folders one by one (if needed). And it's possible to do a
> search in all folders at once if you want to (with the caveat previously
> mentioned).
>
> Another tip is to use a IMAP server with good indexing. That will speed
> searches up and reduce the risks for time-outs.
>

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
-----
squirrelmail-users mailing list
Posting guidelines: http://squirrelmail.org/postingguidelines
List address: squirrelmail-users@...
List archives: http://news.gmane.org/gmane.mail.squirrelmail.user
List info (subscribe/unsubscribe/change options): https://lists.sourceforge.net/lists/listinfo/squirrelmail-users

[Tomas Kuliavas]

Jones B. wrote:

Hi all,

I'm responding back about a thread Tomas and I had going, titled:

> Subject: Re: [SM-USERS] How many messages is squirrelmail designed to handle?
> Dates:
  Tue, 29 Apr 2008 04:11:25 +0000 (UTC)
+ 28 april.


Anyhow, Tomas recommended a few things with people who have a large mailbox and who access it via squirrelmail. You may refer to the original thread, which lists some fantastic ideas, mainly:
 

* enable sort mailbox to load larger messages.
** Use date/size sorting if you can [compared to subject and from/to]


I wonder:

Would moving messages from the inbox folder to another folder be a piece of advice one should follow? Some users I know time out via imap squirrelmail webmail and I have seen the suggestion that one should move mail to another squirrelmail folder and just view new messages in the inbox. Well, while that allows a person to view their inbox all swell and dandy when new messages come in, if they ever view that archive folder where they moved mail to, won't they experience the same problems of timing out when viewing it. Right?

I'm also concerned that even if they move mail messages to another folder, they still won't be able to do search queries on their messages, since those queries on that large collection will still time out since the folder is big.

One more advice - if you have large mailboxes, use cyrus imap. Not courier and not UW IMAP. It is possible that dovecot caching might be faster than Courier maildirs or UW IMAP mbx/unix mailboxes, but I haven't tried to profile dovecot yet.

If you use UW IMAP and have large mailboxes, change server before complaining that mailbox access is slow. UW has worst IMAP SELECT time on 50k messages/1.6GB mailbox.

[Jones B.]

Thanks Tomas, Paul , and Fredrik

Please keep the suggestions coming, but in the meantime I would like to contribute back somehow.

Is it possible some of these and future suggestions can be maintained here:

SquirrelMailPerformance
http://squirrelmail.org/wiki/SquirrelMailPerformance
-- and maybe the subthreads like, SpeedWithUW, SpeedWithDBMail, SpeedWithCyrus, etc.

BrowseProblemsByComplaint
http://squirrelmail.org/wiki/BrowseProblemsByComplaint
-- which had a nice section on "speed"

Some concepts like server side sort, etc I already noticed when researching this issue before and have seen it in the forum, however the recommendation about folder sizes and keeping the inbox small might be worthwhile to make.

I found when Fredrik made the claim that:

> a large inbox might even be an invitation to mailbox corruption and data loss, and/or slow mail delivery down.

the data retention portion (if true and verifiable, though possibly no fault to squirrelmail) caught my mind and probably should be reflected on some doc on the links above somewhere, as kind of a caveat for people to be proactive. I have seen it recommended to keep the inbox small and offload to folders, but I read this before as just a recommendation... now that data loss is a confirmed possibility, I think this recommendation carries a little bit more weight and should be mentioned here...

I also think maybe the comment about cyrus imap for large boxes could be made there too. Maybe even a linkback to this thread <shrug>.

I could help and edit the wiki, I just want to know if there is consensus on some of these suggestions before I or anyone starts editing away...

--
______________________________________________
IndiaInfo Mail - the free e-mail service with a difference! www.indiainfo.com
Check out our value-added Premium features, such as an extra 20MB for mail storage, POP3, e-mail forwarding, and ads-free mailboxes!

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
-----
squirrelmail-users mailing list
Posting guidelines: http://squirrelmail.org/postingguidelines
List address: squirrelmail-users@...
List archives: http://news.gmane.org/gmane.mail.squirrelmail.user
List info (subscribe/unsubscribe/change options): https://lists.sourceforge.net/lists/listinfo/squirrelmail-users

[Paul Lesniewski]

On Wed, Jun 11, 2008 at 10:52 PM, Jones B. <jonesb@...> wrote:

> Thanks Tomas, Paul , and Fredrik
>
> Please keep the suggestions coming, but in the meantime I would like to contribute back somehow.
>
> Is it possible some of these and future suggestions can be maintained here:
>
> SquirrelMailPerformance
> http://squirrelmail.org/wiki/SquirrelMailPerformance
> -- and maybe the subthreads like, SpeedWithUW, SpeedWithDBMail, SpeedWithCyrus, etc.
>
> BrowseProblemsByComplaint
> http://squirrelmail.org/wiki/BrowseProblemsByComplaint
> -- which had a nice section on "speed"

The whole idea of a wiki is for the community to help keep it up to
date, so please be our guest...

> Some concepts like server side sort, etc I already noticed when researching this issue before and have seen it in the forum, however the recommendation about folder sizes and keeping the inbox small might be worthwhile to make.
>
> I found when Fredrik made the claim that:
>
>> a large inbox might even be an invitation to mailbox corruption and data loss, and/or slow mail delivery down.
>
> the data retention portion (if true and verifiable, though possibly no fault to squirrelmail) caught my mind and probably should be reflected on some doc on the links above somewhere, as kind of a caveat for people to be proactive. I have seen it recommended to keep the inbox small and offload to folders, but I read this before as just a recommendation... now that data loss is a confirmed possibility, I think this recommendation carries a little bit more weight and should be mentioned here...
>
> I also think maybe the comment about cyrus imap for large boxes could be made there too. Maybe even a linkback to this thread <shrug>.
>
> I could help and edit the wiki, I just want to know if there is consensus on some of these suggestions before I or anyone starts editing away...

All the suggestions seem to make sense to me, go ahead and add what
you think is appropriate.  The data loss issue Fredrik mentions is
probably somewhat rare on a well-configured system, but "weaker" IMAP
servers with absolutely HUGE mailboxes probably are subject to such
risks, perhaps as much because of write failures in the O/S or
hardware as because of bugs in the software.... that is, I'm not sure
you'll find any hard and fast evidence of such issues, but it's common
sense that slower, less well-designed software will present more risks
when you stretch it beyond its limits.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
-----
squirrelmail-users mailing list
Posting guidelines: http://squirrelmail.org/postingguidelines
List address: squirrelmail-users@...
List archives: http://news.gmane.org/gmane.mail.squirrelmail.user
List info (subscribe/unsubscribe/change options): https://lists.sourceforge.net/lists/listinfo/squirrelmail-users