Memory consumption and network performance questions when opening large folders

[chubbard]

I'm opening large folders of email, in the gigabyte ballpark, and I'm
processing those messages.  However, the memory consumption never goes
down because Folder holds a hard reference to those messages.  Once I
close the folder of course that memory is released.  But, I really
just want to process the message and have the memory reclaimed.  I
don't want the folder to cache it for me.  Is there a way to tell the
folder implementation to not to cache MimeMessages for me?

Also is there a way to improve the performance of pulling down the
email?  I've noticed that since things are lazy email only comes down
once I touch a single MimeMessage.  I'd like to pull down the email in
bulk in larger chunks to reduce network latency and trips back and
forth.  I saw the fetch() method with FetchProfile, but it seemed to
indicate that this method isn't guaraneteed to work across all
providers.  What are the limitations with the Sun providers Pop3 and
IMap?   Can you fetch a N number of messages at a time and download
the full contents of the messages?

Charlie

===========================================================================
To unsubscribe, send email to listserv@... and include in the body
of the message "signoff JAVAMAIL-INTEREST".  For general help, send email to
listserv@... and include in the body of the message "help".

[Bill Shannon]

Charlie Hubbard wrote:

> I'm opening large folders of email, in the gigabyte ballpark, and I'm
> processing those messages.  However, the memory consumption never goes
> down because Folder holds a hard reference to those messages.  Once I
> close the folder of course that memory is released.  But, I really
> just want to process the message and have the memory reclaimed.  I
> don't want the folder to cache it for me.  Is there a way to tell the
> folder implementation to not to cache MimeMessages for me?
>
> Also is there a way to improve the performance of pulling down the
> email?  I've noticed that since things are lazy email only comes down
> once I touch a single MimeMessage.  I'd like to pull down the email in
> bulk in larger chunks to reduce network latency and trips back and
> forth.  I saw the fetch() method with FetchProfile, but it seemed to
> indicate that this method isn't guaraneteed to work across all
> providers.  What are the limitations with the Sun providers Pop3 and
> IMap?   Can you fetch a N number of messages at a time and download
> the full contents of the messages?

The answers are different for the different providers:

For POP3, the entire message is fetched the first time any of the message
content is needed, and is cached for future access to the message content.
To invalidate the cache and allow the memory to be garbage collected, call
POP3Message.invalidate(boolean invalidateHeaders).
http://java.sun.com/products/javamail/javadocs/com/sun/mail/pop3/POP3Message.html#invalidate(boolean)

For IMAP, none of the message content is ever cached.  It's all fetched on
demand each time it's needed.  The headers are cached, and you can invalidate
the cache by calling IMAPMessage.invalidateHeaders().  You can also control
the size of the chunks used when fetching content by setting the
mail.imap.fetchsize property.
http://java.sun.com/products/javamail/javadocs/com/sun/mail/imap/package-summary.html

The fetch method and FetchProfile allows prefetching the header information
in bulk.  It can't be used to prefetch content.

Hope that helps.

===========================================================================
To unsubscribe, send email to listserv@... and include in the body
of the message "signoff JAVAMAIL-INTEREST".  For general help, send email to
listserv@... and include in the body of the message "help".