DO NOT REPLY [Bug 45023] New: DEFLATE preventing 304 NOT MODIFIED response

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023

           Summary: DEFLATE preventing 304 NOT MODIFIED response
           Product: Apache httpd-2
           Version: 2.2.8
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mod_deflate
        AssignedTo: bugs@...
        ReportedBy: noah@...


On my server, I recently added the DEFLATE filter for all
application/x-javascript files, and noticed that apache has stopped sending 304
NOT MODIFIED on ALL my js files *always*.

Observe this header trace:

-----------------------------------------
GET /main.js HTTP/1.1
Host: xxxxxxxxxxxxx
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14)
Gecko/20080404 Firefox/2.0.0.14
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
If-Modified-Since: Mon, 12 May 2008 16:09:00 GMT
If-None-Match: "44d0ac3fd1f00"-gzip
Cache-Control: max-age=0


HTTP/1.x 200 OK
Date: Sat, 17 May 2008 06:09:25 GMT
Server: Apache
Last-Modified: Mon, 12 May 2008 16:09:00 GMT
Etag: "44d0ac3fd1f00"-gzip
Accept-Ranges: bytes
Cache-Control: max-age=86400, must-revalidate, private
Expires: Sun, 18 May 2008 06:09:25 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 9797
Keep-Alive: timeout=2, max=299
Connection: Keep-Alive
Content-Type: application/x-javascript
-----------------------------------------

It seems to me, that a 304 NOT MODIFIED would have been the appropriate
response here.

Before I added DEFLATE to javascript files, the above response was always 304
NOT MODIFIED (except the first one).
Now with deflate, it is *always* 200 OK.

Is this by design, or is this a bug?

One thing to mention is - I have both mod_deflate and mod_expires statically
compiled into httpd.


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


Takashi Sato <takashi@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |takashi@...
           Severity|normal                      |major




--- Comment #1 from Takashi Sato <takashi@...>  2008-05-17 00:22:47 PST ---
If we remove "-gzip" from Etag for If-None-Match, a server returns 304.
mod_deflate adds "-gzip" and sends it to us, but condition check of
If-None-Match runs before mod_deflate ...???


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


Nick Kew <nick@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|major                       |normal
          Component|mod_deflate                 |Core




--- Comment #2 from Nick Kew <nick@...>  2008-05-17 06:47:55 PST ---
This is actually a fix of Bug 39727.  Yes, we know the core code testing for
conditional responses still needs work.


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


John Siracusa <siracusa@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |siracusa@...




--- Comment #3 from John Siracusa <siracusa@...>  2008-06-24 07:32:39 PST ---
This bug nearly defeats the purpose of mod_deflate.  I turned mod_deflate on
with the intent of saving bandwidth when sending JavaScript and CSS files.
Instead, I'm using *more* bandwidth for every request but the first.
(Previously, subsequent requests would result in small, no-content 304
responses.  Now the compressed JS and CSS files are sent every time.)

BTW, the change that appends "-gzip" to the etag (revision 607219, I think) is
a bit wacky in its own right in that it appends outside the double quotes.
IOW, if the original header is:

    Etag: "5954c6-10f4-449d11713aac0"

the modified header ends up as:

    Etag: "5954c6-10f4-449d11713aac0"-gzip

when it probably should be:

    Etag: "5954c6-10f4-449d11713aac0-gzip"

with the "-gzip" inside the quotes.  I altered the code to do this, but it had
no affect on this 304 bug.  I just wanted to note it here so the situation is
avoided when this bug is fixed for real.


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


rahul <rahul@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |RFC




--- Comment #4 from rahul <rahul@...>  2008-08-19 05:36:44 PST ---
There is a workaround,
If you are serving from a location say /js
You can use a configuration like below to switch ETag: $1-gzip to $1

<Location /js>
RequestHeader  edit "If-None-Match" "^(.*)-gzip$" "$1"
Header  edit "ETag" "^(.*[^g][^z][^i][^p])$" "$1-gzip"
</Location>

(Perhaps this should be done from mod_deflate as input filter
with the condition that if Client requests with Accept-Encoding: gzip
only then modify If*match headers to their un-gzip values.)

(Also the current value of ETag (+gzip) is not RFC compliant, since RFC
mandates
a quoted string as the value of ETag. As the noted by john, -gzip should be
inside quotes.)


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #5 from rahul <rahul@...>  2008-08-19 06:41:51 PST ---
Created an attachment (id=22453)
 --> (https://issues.apache.org/bugzilla/attachment.cgi?id=22453)
A POC slightly better than the above conf lines,

Adds an input filter ETAG that can be used like below, along with DEFLATE
to let apache recognize ($1)-gzip as $1

AddInputFilter ETAG .txt
AddOutputFilterByType DEFLATE text/plain


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #6 from John Siracusa <siracusa@...>  2008-08-19 07:47:26 PST ---
Both the workaround in comment #4 and the patch in comment #5 appear to work.
Neither are real "fixes" for the bug, however.  (But I'm glad I have something
to work with until a real fix arrives, so thanks rahul!)


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


rahul <rahul@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #22453|0                           |1
        is obsolete|                            |




--- Comment #7 from rahul <rahul@...>  2008-08-21 08:13:07 PST ---
Created an attachment (id=22468)
 --> (https://issues.apache.org/bugzilla/attachment.cgi?id=22468)
use ap_hook_post_read_request instead of input filter

Incorporate suggested changes by Nick


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


Franklin Tse <peaceable_whale@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |peaceable_whale@...




--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


Maxime Ritter <airmax@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |airmax@...




--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


Sidharth Kshatriya <sid.kshatriya@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sid.kshatriya@...




--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #8 from Sidharth Kshatriya <sid.kshatriya@...>  2008-09-18 08:23:02 PST ---
Hello!

I'm also facing the same problem -- I had been scratching my head for sometime
before I found this bug report. I have to reduce my page size urgently and not
having gzip on is not really an option for me.

I'm not really comfortable having a customized compiled version of apache2 on
my system and the regular expression "hack" will also slow things down (will
it?).

Any alternative ideas? When might this bug be fixed?

Thanks,

Sidharth


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #9 from Sidharth Kshatriya <sid.kshatriya@...>  2008-09-18 08:27:57 PST ---
I had a look at ap_hook_post_read_request attachment...seems like I can get
away with just compiling mod_deflate.c

I'm using Apache 2.2.8. Can anyone help me on where I can get the correct
source for the corresponding mod_deflate and how I can compile mod_deflate?

Thanks for your help!

Sidharth


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #10 from Sidharth Kshatriya <sid.kshatriya@...>  2008-09-18 09:19:28 PST ---
Hi,

Sorry for flooding this mailing list.

In this thread there is Rahul's mod_deflate patch and there are some patches in
this bug thread:

https://issues.apache.org/bugzilla/show_bug.cgi?id=39727
(mostly by Nick Kew)

Which one is the best solution?

Thanks,

Sidharth


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #11 from rahul <rahul@...>  2008-09-18 10:27:15 PST ---
You will need both. This patch expects the other to have been applied (this is
already applied on trunk).
Also please use the dev@... for questions.


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #12 from Sidharth Kshatriya <sid.kshatriya@...>  2008-09-18 11:03:49 PST ---
Thanks for your response Rahul.

Unfortunately I am new to all of this. I applied Nick's patch to my
mod_deflate.c but I got patching errors...his patch is assuming a slightly
different mod_deflate.c from mine.

Would really appreciate if you were able to direct me to the place (or even
better just gave me the patched mod_deflate.c version) that I could just
compile.

[ASIDE: I am using apxs2 -c mod_deflate.c

with LDFLAGS="-lz" set in /usr/bin/aprconfig

I notice that my original mod_deflate.so depends on libpthread. When I apply
your patch to my file and compile, I don't get dependency with libpthread.]

Please tell me if I would need to do any additional transformations to the file
you would provide me (assuming you can :-) ).

Thanks a Ton!

Sidharth
[Attaching the mod_deflate that I have. This I obtained by issuing
sudo apt-get install apache2-src
on ubuntu. This is version 2.2.8 Apache]


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023





--- Comment #13 from Sidharth Kshatriya <sid.kshatriya@...>  2008-09-18 11:10:32 PST ---
Created an attachment (id=22605)
 --> (https://issues.apache.org/bugzilla/attachment.cgi?id=22605)
mod_deflate as found in apache2-src package on ubuntu version 2.2.8-1ubuntu0.3


--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@...
For additional commands, e-mail: bugs-help@...

https://issues.apache.org/bugzilla/show_bug.cgi?id=45023


Franklin Tse <peaceable_whale@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depen