Forward proxies and aliases

[Tavian Barnes]

Is it possible to only proxy part of a site, and use local files for
the rest?  I want most requests to be passed through the proxy, but
some to be aliased to a local file.  I tried

Alias http://www.google.com/ig /home/httpd/html/ig

and similar things, and nothing worked.  Can this be done?

--
Tavian Barnes

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[Wilda, Jet]

You can do that for example by

ProxyPass /ig !

ProxyPass / http://server:port/
ProxyPassReverse / http:// server:port/

This would cause Apache to look for /ig in the DocumentRoot.

~Jet
       

-----Original Message-----
From: Tavian Barnes [mailto:tavianator@...]
Sent: Tuesday, July 08, 2008 2:23 PM
To: users@...
Subject: [users@httpd] Forward proxies and aliases

Is it possible to only proxy part of a site, and use local files for
the rest?  I want most requests to be passed through the proxy, but
some to be aliased to a local file.  I tried

Alias http://www.google.com/ig /home/httpd/html/ig

and similar things, and nothing worked.  Can this be done?

--
Tavian Barnes

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server
Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...
----------
Learn more about Chase Paymentech Solutions,LLC payment processing services at www.chasepaymentech.com.

THIS MESSAGE IS CONFIDENTIAL.  This e-mail message and any attachments are proprietary and confidential information intended only for the use of the recipient(s) named above.  If you are not the intended recipient, you may not print, distribute, or copy this message or any attachments.  If you have received this communication in error, please notify the sender by return e-mail and delete this message and any attachments from your computer.


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[Tavian Barnes]

But that's a reverse proxy, not a forward one.  I'm attempting to use
an Apache server as a proxy for my web browser, that uses a local copy
of certain files from certain sites.  As such, it has to be a forward
proxy, because I'm using it for arbitrary web sites.

On Tue, Jul 8, 2008 at 1:13 PM, Wilda, Jet
<Jet.Wilda@...> wrote:

> You can do that for example by
>
> ProxyPass /ig !
>
> ProxyPass / http://server:port/
> ProxyPassReverse / http:// server:port/
>
> This would cause Apache to look for /ig in the DocumentRoot.
>
> ~Jet
>
>
> -----Original Message-----
> From: Tavian Barnes [mailto:tavianator@...]
> Sent: Tuesday, July 08, 2008 2:23 PM
> To: users@...
> Subject: [users@httpd] Forward proxies and aliases
>
> Is it possible to only proxy part of a site, and use local files for
> the rest?  I want most requests to be passed through the proxy, but
> some to be aliased to a local file.  I tried
>
> Alias http://www.google.com/ig /home/httpd/html/ig
>
> and similar things, and nothing worked.  Can this be done?
>
> --
> Tavian Barnes
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server
> Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@...
>   "   from the digest: users-digest-unsubscribe@...
> For additional commands, e-mail: users-help@...
> ----------
> Learn more about Chase Paymentech Solutions,LLC payment processing services at www.chasepaymentech.com.
>
> THIS MESSAGE IS CONFIDENTIAL.  This e-mail message and any attachments are proprietary and confidential information intended only for the use of the recipient(s) named above.  If you are not the intended recipient, you may not print, distribute, or copy this message or any attachments.  If you have received this communication in error, please notify the sender by return e-mail and delete this message and any attachments from your computer.
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@...
>   "   from the digest: users-digest-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

--
Tavian Barnes

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[Nick Kew]

On Tue, 8 Jul 2008 13:48:54 -0600
"Tavian Barnes" <tavianator@...> wrote:

> But that's a reverse proxy, not a forward one.  I'm attempting to use
> an Apache server as a proxy for my web browser, that uses a local copy
> of certain files from certain sites.  As such, it has to be a forward
> proxy, because I'm using it for arbitrary web sites.

That's not what you originally asked.

Anyway, three solutions, from the most sensible
to the least:
  * Set up proxy exceptions in your browser preferences
  * Use mod_cache for your local copies
  * Use mod_rewrite - probably a rewritemap.

Unless of course your real purpose is to block crap like
animated images, in which case maybe the rewritemap makes
good sense (I use my ADSL router to do that).

--
Nick Kew

Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[christian.folini]

On Tue, 8 Jul 2008 22:04:54 "Nick Kew" <nick@...> wrote:
>> But that's a reverse proxy, not a forward one.  I'm attempting to use

>> an Apache server as a proxy for my web browser, that uses a local
copy
>> of certain files from certain sites.  As such, it has to be a forward

>> proxy, because I'm using it for arbitrary web sites.
>
>That's not what you originally asked.

The subject reads "Forward proxies and aliases". So where is the
problem?

Otherwise, it sounds like a case for mod_cache or a nifty rewrite map
for me too.

christian


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[Tavian Barnes]

On Tue, Jul 8, 2008 at 2:03 PM, Nick Kew <nick@...> wrote:
> On Tue, 8 Jul 2008 13:48:54 -0600
> "Tavian Barnes" <tavianator@...> wrote:
>
>> But that's a reverse proxy, not a forward one.  I'm attempting to use
>> an Apache server as a proxy for my web browser, that uses a local copy
>> of certain files from certain sites.  As such, it has to be a forward
>> proxy, because I'm using it for arbitrary web sites.
>
> That's not what you originally asked.

It's what I attempted to ask, by referring to "forward proxies" in the
subject line.

> Anyway, three solutions, from the most sensible
> to the least:
>  * Set up proxy exceptions in your browser preferences

I know of nothing that can direct particular web addresses to local
copies of files; all the tools I've seen can just selectively proxy
addresses.

>  * Use mod_cache for your local copies

My local copies are intended to differ from the copies on the web.
Can mod_cache or mod_file_cache really be set to use a particular
local  version of a file, rather than automatically generating the
cache?  I realise that I didn't clarify this earlier.

>  * Use mod_rewrite - probably a rewritemap.

Is there any reason I'd need to use a rewrite map?  Reading through
the mod_rewrite documentation, I came up with this,

RewriteCond %{HTTP_HOST} ^www\.google\.com
RewriteRule ^/ig /ig [R]

which unfortunately doesn't work.  What am I missing?

> Unless of course your real purpose is to block crap like
> animated images, in which case maybe the rewritemap makes
> good sense (I use my ADSL router to do that).
>
> --
> Nick Kew
>
> Application Development with Apache - the Apache Modules Book
> http://www.apachetutor.org/
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@...
>   "   from the digest: users-digest-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

--
Tavian Barnes

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[awarnier]

Just an idea from a guy who really knows next to nothing about proxies :
If
- a proxy configuration allows you to selectively forward some requests
to a selection of sites, but not to a local file
- but what you want to do is to redirect some URLs to a local file

then can you not set up a local virtual host under some name, and
forward ditto "local" requests to that local virtual host, whose
pleasure it would be to serve the local stuff in question ?

André


Tavian Barnes wrote:

> On Tue, Jul 8, 2008 at 2:03 PM, Nick Kew <nick@...> wrote:
>> On Tue, 8 Jul 2008 13:48:54 -0600
>> "Tavian Barnes" <tavianator@...> wrote:
>>
>>> But that's a reverse proxy, not a forward one.  I'm attempting to use
>>> an Apache server as a proxy for my web browser, that uses a local copy
>>> of certain files from certain sites.  As such, it has to be a forward
>>> proxy, because I'm using it for arbitrary web sites.
>> That's not what you originally asked.
>
> It's what I attempted to ask, by referring to "forward proxies" in the
> subject line.
>
>> Anyway, three solutions, from the most sensible
>> to the least:
>>  * Set up proxy exceptions in your browser preferences
>
> I know of nothing that can direct particular web addresses to local
> copies of files; all the tools I've seen can just selectively proxy
> addresses.
>
>>  * Use mod_cache for your local copies
>
> My local copies are intended to differ from the copies on the web.
> Can mod_cache or mod_file_cache really be set to use a particular
> local  version of a file, rather than automatically generating the
> cache?  I realise that I didn't clarify this earlier.
>
>>  * Use mod_rewrite - probably a rewritemap.
>
> Is there any reason I'd need to use a rewrite map?  Reading through
> the mod_rewrite documentation, I came up with this,
>
> RewriteCond %{HTTP_HOST} ^www\.google\.com
> RewriteRule ^/ig /ig [R]
>
> which unfortunately doesn't work.  What am I missing?
>
>> Unless of course your real purpose is to block crap like
>> animated images, in which case maybe the rewritemap makes
>> good sense (I use my ADSL router to do that).
>>
>> --
>> Nick Kew
>>
>> Application Development with Apache - the Apache Modules Book
>> http://www.apachetutor.org/
>>
>> ---------------------------------------------------------------------
>> The official User-To-User support forum of the Apache HTTP Server Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe@...
>>   "   from the digest: users-digest-unsubscribe@...
>> For additional commands, e-mail: users-help@...
>>
>>
>
>
>

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[Tavian Barnes]

> Just an idea from a guy who really knows next to nothing about proxies :
> If
> - a proxy configuration allows you to selectively forward some requests to a
> selection of sites, but not to a local file
> - but what you want to do is to redirect some URLs to a local file
>
> then can you not set up a local virtual host under some name, and forward
> ditto "local" requests to that local virtual host, whose pleasure it would
> be to serve the local stuff in question ?
>
> André
>

The thing is, when the browser sends the request to the local virtual
host, it's treating it like a proxy server.  So, the local host would
have to be able to recognise a proxy request for a certain page, and
return a local file instead, which is the same thing I'm trying to do
now.

--
Tavian Barnes

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[awarnier]

Tavian Barnes wrote:

>> Just an idea from a guy who really knows next to nothing about proxies :
>> If
>> - a proxy configuration allows you to selectively forward some requests to a
>> selection of sites, but not to a local file
>> - but what you want to do is to redirect some URLs to a local file
>>
>> then can you not set up a local virtual host under some name, and forward
>> ditto "local" requests to that local virtual host, whose pleasure it would
>> be to serve the local stuff in question ?
>>
>> André
>>
>
> The thing is, when the browser sends the request to the local virtual
> host, it's treating it like a proxy server.  So, the local host would
> have to be able to recognise a proxy request for a certain page, and
> return a local file instead, which is the same thing I'm trying to do
> now.
>

It's probably me in this case, but I believe one of us is not
understanding this right.

At the browser level, you are requesting items from www.google.com, and
the browser is configured to use serverA as a http proxy, right ?

And what you want is that the proxy on serverA would get some stuff
really from www.google.com, but some other stuff it should deliver from
local files instead, still right ?
But that is something you cannot configure easily, because there is no
way in the proxy to tell it to forward some requests to a http server,
but serve some other ones from local files.  Still right ?

So, what I mean is that on the same host hosting serverA, (which you use
as a http proxy), you also configure a separate serverB (virtual), which
can deliver some "local" files (if a suitable request URL requests
them). (or you set up serverB on a totally separate local host, whatever).

The browser requests are all directed to www.google.com, but still all
requests go through serverA, because it is the configured http proxy.
And serverA, by virtue of its proxy functionality would normally forward
these requests to www.google.com (or serve them from its cache of
www.google.com items).

Now in serverA, you set up some early URL filter which will rewrite
(modify) all requests which you do not really want to go to
www.google.com, and substitute the appropriate URL so that this item
will be instead "proxied" to serverB instead of www.google.com.
(In other words e.g., in all requests ending in ".gif", you change to
hostname to "serverB").
So for some requests, www.google.com delivers the content, while from
some others it will be serverB.  But all responses eventually go back
through the serverA proxy, which delivers them to the browser.

I believe the browser wouldn't know the difference, because for him all
requests are to www.google.com (while using the proxy serverA), and
that's where it thinks it gets the responses from.  I don't think that
the browser "matches" the responses back with its requests, to check
inside if they really come from the same place.
Because if it did, then the whole proxy mechanism wouldn't work in the
first place.

Or maybe I'm totally wrong with this, but it seems at least plausible, no ?

André


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[Tavian Barnes]

> Tavian Barnes wrote:
>>>
>>> Just an idea from a guy who really knows next to nothing about proxies :
>>> If
>>> - a proxy configuration allows you to selectively forward some requests
>>> to a
>>> selection of sites, but not to a local file
>>> - but what you want to do is to redirect some URLs to a local file
>>>
>>> then can you not set up a local virtual host under some name, and forward
>>> ditto "local" requests to that local virtual host, whose pleasure it
>>> would
>>> be to serve the local stuff in question ?
>>>
>>> André
>>>
>>
>> The thing is, when the browser sends the request to the local virtual
>> host, it's treating it like a proxy server.  So, the local host would
>> have to be able to recognise a proxy request for a certain page, and
>> return a local file instead, which is the same thing I'm trying to do
>> now.
>>
> It's probably me in this case, but I believe one of us is not understanding
> this right.
>
> At the browser level, you are requesting items from www.google.com, and the
> browser is configured to use serverA as a http proxy, right ?

Right.

> And what you want is that the proxy on serverA would get some stuff really
> from www.google.com, but some other stuff it should deliver from local files
> instead, still right ?

Right.

> But that is something you cannot configure easily, because there is no way
> in the proxy to tell it to forward some requests to a http server, but serve
> some other ones from local files.  Still right ?

As far as I can tell.

> So, what I mean is that on the same host hosting serverA, (which you use as
> a http proxy), you also configure a separate serverB (virtual), which can
> deliver some "local" files (if a suitable request URL requests them). (or
> you set up serverB on a totally separate local host, whatever).
>
> The browser requests are all directed to www.google.com, but still all
> requests go through serverA, because it is the configured http proxy.
> And serverA, by virtue of its proxy functionality would normally forward
> these requests to www.google.com (or serve them from its cache of
> www.google.com items).
>
> Now in serverA, you set up some early URL filter which will rewrite (modify)
> all requests which you do not really want to go to www.google.com, and
> substitute the appropriate URL so that this item will be instead "proxied"
> to serverB instead of www.google.com.
> (In other words e.g., in all requests ending in ".gif", you change to
> hostname to "serverB").
> So for some requests, www.google.com delivers the content, while from some
> others it will be serverB.  But all responses eventually go back through the
> serverA proxy, which delivers them to the browser.

Actually, it would be fine to just rewrite the url to a location on my
local web server.  I scratched my head figuring out how to do this,
until I noticed that my RewriteRule ^/ig wasn't matching, but .*ig.*
was.  I figured that the rewrite module must handle proxied urls
differently, so I wrote a script to use as a RewriteMap that simply
copied standard input to a file, and to standard output again.
Reading the file, it turns out (is this documented somewhere?) that
the url comes to the RewriteRule in the form of
"proxy:http://www.google.com/ig" which I can now rewrite to
http://localhost/ig.  So, problem solved.

> I believe the browser wouldn't know the difference, because for him all
> requests are to www.google.com (while using the proxy serverA), and that's
> where it thinks it gets the responses from.  I don't think that the browser
> "matches" the responses back with its requests, to check inside if they
> really come from the same place.
> Because if it did, then the whole proxy mechanism wouldn't work in the first
> place.

When behind a proxy, the browser sends request to the proxy in the
form of GET http://www.google.com/ig, rather than sending GET /ig to
www.google.com.  It has no idea if the proxy server did some tricky
things behind its back.

> Or maybe I'm totally wrong with this, but it seems at least plausible, no ?

Your idea is more complicated than I needed, but I think it should work too.

Thanks Andre, and everyone else; I've got it working.

> André
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@...
>  "   from the digest: users-digest-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

--
Tavian Barnes

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...

[awarnier]

Tavian Barnes wrote:
>
> When behind a proxy, the browser sends request to the proxy in the
> form of GET http://www.google.com/ig, rather than sending GET /ig to
> www.google.com.  It has no idea if the proxy server did some tricky
> things behind its back.
>
Thank you for reminding me of that.  I remember reading that somewhere,
but I had forgotten.  I though that the browser just sent the request to
the IP of the proxy as "GET /ig", with a "Host: www.google.com" header.
But now I realise the error of my ways.
Thanks.

André



---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@...
   "   from the digest: users-digest-unsubscribe@...
For additional commands, e-mail: users-help@...