[mime4j] EOLConvertingInputStream is evil

[olegk]

Folks,

I am pretty to close to completing the refactoring of the
MimeTokenStream and related classes. However, before I can proceed I
would like to know whether it would be possible to change the way mime4j
handles line delimiters. I can't help feeling the current approach based
on EOLConvertingInputStream is wrong

(1) EOLConvertingInputStream is a MAJOR performance killer. Presently it
always reads one byte at a time thus completely eliminating all
performance gains at the MimeBoundaryInputStream level. Even if it was
changed to read more data at a time it would still cause significant
performance degradation due to the double buffering of the input data.

(2) EOLConvertingInputStream is superfluous. In real world situations
one cannot know beforehand whether incoming messages are standard
compliant or not. Basically any reasonable application would always need
to be lenient about line delimiters especially when dealing with MIME
content sent over HTTP.

(3) And finally EOLConvertingInputStream is broken. It touches every
single occurrence of the line delimiter regardless of their position
within the MIME stream. Multipart messages may contain raw binary data
(application/octet-stream). This is especially common when uploading
multipart encoded content over HTTP. EOLConvertingInputStream would
simply corrupt such messages.

Would anyone object if I implemented lenient line delimiter handling
only when parsing header fields / MIME boundary and got rid of
EOLConvertingInputStream?

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@...
For additional commands, e-mail: server-dev-help@...

[robert burrell donkin-2]

On Sat, Jul 5, 2008 at 5:04 PM, Oleg Kalnichevski <olegk@...> wrote:
> Folks,
>
> I am pretty to close to completing the refactoring of the
> MimeTokenStream and related classes. However, before I can proceed I
> would like to know whether it would be possible to change the way mime4j
> handles line delimiters. I can't help feeling the current approach based
> on EOLConvertingInputStream is wrong

i've never really liked the way it's currently used but i don't
understand the reason why this design choice was made. anyone
remember?

> (1) EOLConvertingInputStream is a MAJOR performance killer. Presently it
> always reads one byte at a time thus completely eliminating all
> performance gains at the MimeBoundaryInputStream level. Even if it was
> changed to read more data at a time it would still cause significant
> performance degradation due to the double buffering of the input data.

+1

> (2) EOLConvertingInputStream is superfluous. In real world situations
> one cannot know beforehand whether incoming messages are standard
> compliant or not. Basically any reasonable application would always need
> to be lenient about line delimiters especially when dealing with MIME
> content sent over HTTP.

with MIME over HTTP, i agree. with mail, it's more complex. i think
that there are better ways to deal with this, though.

> (3) And finally EOLConvertingInputStream is broken. It touches every
> single occurrence of the line delimiter regardless of their position
> within the MIME stream. Multipart messages may contain raw binary data
> (application/octet-stream). This is especially common when uploading
> multipart encoded content over HTTP. EOLConvertingInputStream would
> simply corrupt such messages.

IMHO EOLConvertingInputStream is not broken but the current usage of it

> Would anyone object if I implemented lenient line delimiter handling
> only when parsing header fields / MIME boundary and got rid of
> EOLConvertingInputStream?

EOLConvertingInputStream is reasonably useful for mail and is not out
of scope. so i'd prefer to keep the implementation.

switching to lenient hanlding of line delimiters is fine by me. may
need to offer an event hook for anyone who needs strict compliance.

- robert

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@...
For additional commands, e-mail: server-dev-help@...