Page handlers and proxy errors

[Bill Gribble]

We have a CherryPy 3.0.0 server behind Apache via mod_proxy.  There's a
page handler that has guts basically like this:

   @exposed
   def index(self):
     dbc = psycopg.connect(self.dsn)
     try:
        curs = dbc.cursor()
        curs.execute("a query that occasionally takes a long time")
        res = curs.fetchall()
        return self.format_results_to_html(res)
     finally:
        dbc.commit()                
        dbc.close()

On the occasions that the query takes a really long time to execute
(during middle-of-the-night vacuum jobs, for instance) sometimes Apache
will return a 502 Proxy Error to the client, with timeout being the
reason.  When this happens, the finally: clause above is apparently not
executed, and the Postgresql connection continues to exist in the "IDLE
in transaction" state until the backend is manually killed.

This is really screwing with our Slony database replication, which
cannot stand "IDLE in transaction" connections.

Any ideas as to why our finally: handler is not being called, and what
we might be able to do to address this problem?

Thanks,
Bill Gribble

!DSPAM:4846bc30117725565028729!



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-users@...
To unsubscribe from this group, send email to cherrypy-users-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---

[Robert Brewer-4]

Bill Gribble wrote:
> We have a CherryPy 3.0.0 server behind Apache via mod_proxy.  There's
a

> page handler that has guts basically like this:
>
>    @exposed
>    def index(self):
>      dbc = psycopg.connect(self.dsn)
>      try:
>         curs = dbc.cursor()
>         curs.execute("a query that occasionally takes a long time")
>         res = curs.fetchall()
>         return self.format_results_to_html(res)
>      finally:
>         dbc.commit()
>         dbc.close()
>
> On the occasions that the query takes a really long time to execute
> (during middle-of-the-night vacuum jobs, for instance) sometimes

Apache
> will return a 502 Proxy Error to the client, with timeout being the
> reason.  When this happens, the finally: clause above is apparently
not
> executed, and the Postgresql connection continues to exist in the
"IDLE
> in transaction" state until the backend is manually killed.
>
> This is really screwing with our Slony database replication, which
> cannot stand "IDLE in transaction" connections.
>
> Any ideas as to why our finally: handler is not being called, and what
> we might be able to do to address this problem?

Are you sure it's mod_proxy? <:) I know Apache is pretty aggressive
about forcefully killing off timed out subprocesses, but it shouldn't be
killing a standalone CherryPy process! Maybe it's an OOM-killer...?

If it's truly being killed off, you'll have to move it to a different
process somehow. If it's not being killed off (or to avoid being
killed), you might try moving the work to a new thread and either:

 1. return "202 Accepted" right away, along with a caching scheme so
that future requests return either 202 or the cached answer once the
work is done. I also return a Retry-After header in these cases.
 2. return "303 See Other" right away, along with a Location header
specifying the URI of the answer. Clients can poll that URI until they
get 200 instead of 204 No Content.


Robert Brewer
fumanchu@...


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-users@...
To unsubscribe from this group, send email to cherrypy-users-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---

[Bill Gribble]

"Robert Brewer" <fumanchu@...> writes:
> Are you sure it's mod_proxy? <:) I know Apache is pretty aggressive
> about forcefully killing off timed out subprocesses, but it shouldn't be
> killing a standalone CherryPy process! Maybe it's an OOM-killer...?

There's plenty of memory, I don't think it would be an OOM issue.  The
CherryPy process keeps running; AFAICT it's just the request thread that
terminates abruptly.

>  1. return "202 Accepted" right away, along with a caching scheme so
> that future requests return either 202 or the cached answer once the
> work is done. I also return a Retry-After header in these cases.
>  2. return "303 See Other" right away, along with a Location header
> specifying the URI of the answer. Clients can poll that URI until they
> get 200 instead of 204 No Content.


We do a variant of 1 on some pages known to take a long time, so I
guess it's the approach that makes sense.  It's just irritating to have
apache mistreating my innocent CherryPy server!

Thanks for the tip..

Bill Gribble

!DSPAM:4846c78a117724481081033!



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-users@...
To unsubscribe from this group, send email to cherrypy-users-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---