Http Connection Pool

[Davide Taibi]

Dear All,
I need to build a connection pool by using the IOUtils library.

I have a website set to download (around 5000) and I need to parallelize the
connection by launching from 10 to 20 connection per time.

I've tried a multithread strategy but it doesn't work soundly.

I'd like to implement something like a queue of connection and I tought
about the apache commons pool libraries.
Is there any implementation of http connection pool?

Thanks in advance

Davide Taibi
http://www.taibi.it

[Olivier Lamy]

Hi,

Have a look at :
http://hc.apache.org/httpcomponents-client/httpclient/apidocs/org/apache/http/impl/conn/tsccm/ThreadSafeClientConnManager.html

--
Olivier


2008/4/30 Davide Taibi <davide@...>:

> Dear All,
>  I need to build a connection pool by using the IOUtils library.
>
>  I have a website set to download (around 5000) and I need to parallelize the
>  connection by launching from 10 to 20 connection per time.
>
>  I've tried a multithread strategy but it doesn't work soundly.
>
>  I'd like to implement something like a queue of connection and I tought
>  about the apache commons pool libraries.
>  Is there any implementation of http connection pool?
>
>  Thanks in advance
>
>  Davide Taibi
>  http://www.taibi.it
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Adam Retter-4]

The Apache Commons HTTPClient supports multiple HTTP Connections in
the manner you indicate -

http://hc.apache.org/httpclient-3.x/performance.html#Concurrent_execution_of_HTTP_methods


On Wed, Apr 30, 2008 at 10:50 AM, Davide Taibi <davide@...> wrote:

> Dear All,
>  I need to build a connection pool by using the IOUtils library.
>
>  I have a website set to download (around 5000) and I need to parallelize the
>  connection by launching from 10 to 20 connection per time.
>
>  I've tried a multithread strategy but it doesn't work soundly.
>
>  I'd like to implement something like a queue of connection and I tought
>  about the apache commons pool libraries.
>  Is there any implementation of http connection pool?
>
>  Thanks in advance
>
>  Davide Taibi
>  http://www.taibi.it
>

--
Adam Retter

Software Pimp Extraordinaire

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[rajesh.balamohan]

>>> This class is derived from MultiThreadedHttpConnectionManager in
HttpClient 3

Any specific reason for renaming it?. Do we have any performance
improvements. Currently we are using HttpClient 3.x. Is it ok to take
HttpClient 4.0 for our production system (if there are any significant
performance improvements?)

On Wed, Apr 30, 2008 at 3:26 PM, Adam Retter <adam.retter@...>
wrote:

> The Apache Commons HTTPClient supports multiple HTTP Connections in
> the manner you indicate -
>
>
> http://hc.apache.org/httpclient-3.x/performance.html#Concurrent_execution_of_HTTP_methods
>
>
> On Wed, Apr 30, 2008 at 10:50 AM, Davide Taibi <davide@...> wrote:
> > Dear All,
> >  I need to build a connection pool by using the IOUtils library.
> >
> >  I have a website set to download (around 5000) and I need to
> parallelize the
> >  connection by launching from 10 to 20 connection per time.
> >
> >  I've tried a multithread strategy but it doesn't work soundly.
> >
> >  I'd like to implement something like a queue of connection and I tought
> >  about the apache commons pool libraries.
> >  Is there any implementation of http connection pool?
> >
> >  Thanks in advance
> >
> >  Davide Taibi
> >  http://www.taibi.it
> >
>
>
>
> --
> Adam Retter
>
> Software Pimp Extraordinaire
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@...
> For additional commands, e-mail: user-help@...
>
>

--
~Rajesh.B

[Olivier Lamy]

As I have understand, you can now use NIO instead of classic IO.
But for more questions you have to ask the hc ml [1].

--
Olivier

[1] http://hc.apache.org/mail-lists.html

2008/4/30 Rajesh Balamohan <rajesh.balamohan@...>:

> >>> This class is derived from MultiThreadedHttpConnectionManager in
>  HttpClient 3
>
>  Any specific reason for renaming it?. Do we have any performance
>  improvements. Currently we are using HttpClient 3.x. Is it ok to take
>  HttpClient 4.0 for our production system (if there are any significant
>  performance improvements?)
>
>  On Wed, Apr 30, 2008 at 3:26 PM, Adam Retter <adam.retter@...>
>  wrote:
>
>
>
>  > The Apache Commons HTTPClient supports multiple HTTP Connections in
>  > the manner you indicate -
>  >
>  >
>  > http://hc.apache.org/httpclient-3.x/performance.html#Concurrent_execution_of_HTTP_methods
>  >
>  >
>  > On Wed, Apr 30, 2008 at 10:50 AM, Davide Taibi <davide@...> wrote:
>  > > Dear All,
>  > >  I need to build a connection pool by using the IOUtils library.
>  > >
>  > >  I have a website set to download (around 5000) and I need to
>  > parallelize the
>  > >  connection by launching from 10 to 20 connection per time.
>  > >
>  > >  I've tried a multithread strategy but it doesn't work soundly.
>  > >
>  > >  I'd like to implement something like a queue of connection and I tought
>  > >  about the apache commons pool libraries.
>  > >  Is there any implementation of http connection pool?
>  > >
>  > >  Thanks in advance
>  > >
>  > >  Davide Taibi
>  > >  http://www.taibi.it
>  > >
>  >
>  >
>  >
>  > --
>  > Adam Retter
>  >
>  > Software Pimp Extraordinaire
>  >
>  > ---------------------------------------------------------------------
>  > To unsubscribe, e-mail: user-unsubscribe@...
>  > For additional commands, e-mail: user-help@...
>  >
>  >
>
>
>  --
>  ~Rajesh.B
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...