ivy:resolve & latest.status

[Hans Lund]

I'm trying to resolve dependencies using specified a specified status

having the following in settings.

--------
<statuses default="integration">
        <status name="release" integration="false"/>
  <status name="DEV" integration="true"/>
  <status name="integration" integration="true"/>
</statuses>
---------

Now in the ivy file I have some dependencies like the following (where
${revision.status} is set in ant depending on source-branch.

------------
<dependency org="com.multisupport"
name="commons"rev="latest.${revision.status}" conf="compile->default(*)
"/>
------------


Running:

<ivy:resolve conf="runtime" />
As long as it runs this works as expected.

But: After a short while running on a CI (hudson) server, where the ivy
repository also is placed, resolving stops working due to connection
problems in httpClient.
---

[ivy:resolve] 01-07-2008 13:16:24
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: Retrying request
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: Retrying request
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: Retrying request
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: Retrying request
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
[ivy:resolve] 01-07-2008 13:16:24 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
[ivy:resolve] INFO: Retrying request ...............  and so on:


which in the end results in unresolved dependencies.

It looks like that the problem might be that every dependency is downloaded (ivy, checksum, artifacts) and after ~
all connections on the server are used.
So in effect <ivy:resolve> performs a dos against the repository.

Does any one had any similar experiences, and have a solution?
-- what I can come up with is to clean up the dev repository, but I find that somewhat a no-solution



Should this be considered an ivy bug?

Regards
Hans Lund

















 

[Xavier Hanin]

On Tue, Jul 1, 2008 at 5:58 PM, Hans Lund <ha.lund@...> wrote:

> I'm trying to resolve dependencies using specified a specified status
>
> having the following in settings.
>
> --------
> <statuses default="integration">
>        <status name="release" integration="false"/>
>        <status name="DEV" integration="true"/>
>        <status name="integration" integration="true"/>
> </statuses>
> ---------
>
> Now in the ivy file I have some dependencies like the following (where
> ${revision.status} is set in ant depending on source-branch.
>
> ------------
> <dependency org="com.multisupport"
> name="commons"rev="latest.${revision.status}" conf="compile->default(*)
> "/>
> ------------
>
>
> Running:
>
> <ivy:resolve conf="runtime" />
> As long as it runs this works as expected.
>
> But: After a short while running on a CI (hudson) server, where the ivy
> repository also is placed, resolving stops working due to connection
> problems in httpClient.
> ---
>
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> processing request: Address already in use: connect
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: Retrying request
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> processing request: Address already in use: connect
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: Retrying request
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> processing request: Address already in use: connect
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: Retrying request
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> processing request: Address already in use: connect
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: Retrying request
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> processing request: Address already in use: connect
> [ivy:resolve] 01-07-2008 13:16:24
> org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> [ivy:resolve] INFO: Retrying request ...............  and so on:
>
>
> which in the end results in unresolved dependencies.
>
> It looks like that the problem might be that every dependency is downloaded
> (ivy, checksum, artifacts) and after ~
> all connections on the server are used.
> So in effect <ivy:resolve> performs a dos against the repository.
>
> Does any one had any similar experiences, and have a solution?
> -- what I can come up with is to clean up the dev repository, but I find
> that somewhat a no-solution
>
>
>
> Should this be considered an ivy bug?

Absolutely, please open a JIRA. Could you test also without httpclient, to
see if it makes any difference?

Xavier

>
>
> Regards
> Hans Lund
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/

[Hans Lund]

On Thu, 2008-07-03 at 21:22 +0200, Xavier Hanin wrote:

> On Tue, Jul 1, 2008 at 5:58 PM, Hans Lund <ha.lund@...> wrote:
>
> > I'm trying to resolve dependencies using specified a specified status
> >
> > having the following in settings.
> >
> > --------
> > <statuses default="integration">
> >        <status name="release" integration="false"/>
> >        <status name="DEV" integration="true"/>
> >        <status name="integration" integration="true"/>
> > </statuses>
> > ---------
> >
> > Now in the ivy file I have some dependencies like the following (where
> > ${revision.status} is set in ant depending on source-branch.
> >
> > ------------
> > <dependency org="com.multisupport"
> > name="commons"rev="latest.${revision.status}" conf="compile->default(*)
> > "/>
> > ------------
> >
> >
> > Running:
> >
> > <ivy:resolve conf="runtime" />
> > As long as it runs this works as expected.
> >
> > But: After a short while running on a CI (hudson) server, where the ivy
> > repository also is placed, resolving stops working due to connection
> > problems in httpClient.
> > ---
> >
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > processing request: Address already in use: connect
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: Retrying request
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > processing request: Address already in use: connect
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: Retrying request
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > processing request: Address already in use: connect
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: Retrying request
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > processing request: Address already in use: connect
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: Retrying request
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > processing request: Address already in use: connect
> > [ivy:resolve] 01-07-2008 13:16:24
> > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > [ivy:resolve] INFO: Retrying request ...............  and so on:
> >
> >
> > which in the end results in unresolved dependencies.
> >
> > It looks like that the problem might be that every dependency is downloaded
> > (ivy, checksum, artifacts) and after ~
> > all connections on the server are used.
> > So in effect <ivy:resolve> performs a dos against the repository.
> >
> > Does any one had any similar experiences, and have a solution?
> > -- what I can come up with is to clean up the dev repository, but I find
> > that somewhat a no-solution
> >
> >
> >
> > Should this be considered an ivy bug?
>
> Absolutely, please open a JIRA. Could you test also without httpclient, to
> see if it makes any difference?
>
> Xavier

OK, I've created a JIRA bug report
http://issues.apache.org/jira/browse/IVY-854

I think that further discussion should be on ivy-dev ?

/Hans Lund

[Xavier Hanin]

On Fri, Jul 4, 2008 at 11:50 AM, Hans Lund <ha.lund@...> wrote:

> On Thu, 2008-07-03 at 21:22 +0200, Xavier Hanin wrote:
> > On Tue, Jul 1, 2008 at 5:58 PM, Hans Lund <ha.lund@...> wrote:
> >
> > > I'm trying to resolve dependencies using specified a specified status
> > >
> > > having the following in settings.
> > >
> > > --------
> > > <statuses default="integration">
> > >        <status name="release" integration="false"/>
> > >        <status name="DEV" integration="true"/>
> > >        <status name="integration" integration="true"/>
> > > </statuses>
> > > ---------
> > >
> > > Now in the ivy file I have some dependencies like the following (where
> > > ${revision.status} is set in ant depending on source-branch.
> > >
> > > ------------
> > > <dependency org="com.multisupport"
> > > name="commons"rev="latest.${revision.status}" conf="compile->default(*)
> > > "/>
> > > ------------
> > >
> > >
> > > Running:
> > >
> > > <ivy:resolve conf="runtime" />
> > > As long as it runs this works as expected.
> > >
> > > But: After a short while running on a CI (hudson) server, where the ivy
> > > repository also is placed, resolving stops working due to connection
> > > problems in httpClient.
> > > ---
> > >
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > processing request: Address already in use: connect
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: Retrying request
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > processing request: Address already in use: connect
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: Retrying request
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > processing request: Address already in use: connect
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: Retrying request
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > processing request: Address already in use: connect
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: Retrying request
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > processing request: Address already in use: connect
> > > [ivy:resolve] 01-07-2008 13:16:24
> > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > [ivy:resolve] INFO: Retrying request ...............  and so on:
> > >
> > >
> > > which in the end results in unresolved dependencies.
> > >
> > > It looks like that the problem might be that every dependency is
> downloaded
> > > (ivy, checksum, artifacts) and after ~
> > > all connections on the server are used.
> > > So in effect <ivy:resolve> performs a dos against the repository.
> > >
> > > Does any one had any similar experiences, and have a solution?
> > > -- what I can come up with is to clean up the dev repository, but I
> find
> > > that somewhat a no-solution
> > >
> > >
> > >
> > > Should this be considered an ivy bug?
> >
> > Absolutely, please open a JIRA. Could you test also without httpclient,
> to
> > see if it makes any difference?
> >
> > Xavier
>
> OK, I've created a JIRA bug report
> http://issues.apache.org/jira/browse/IVY-854
>
> I think that further discussion should be on ivy-dev ?

Yes, probably (and ivy-dev is actually ant-dev). Not sure if there is much
people from Ivy team currently checking their e-mails though.

Xavier


>
> /Hans Lund
>
>
>


--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/

[Hans Lund]

On Fri, 2008-07-04 at 11:58 +0200, Xavier Hanin wrote:

> On Fri, Jul 4, 2008 at 11:50 AM, Hans Lund <ha.lund@...> wrote:
>
> > On Thu, 2008-07-03 at 21:22 +0200, Xavier Hanin wrote:
> > > On Tue, Jul 1, 2008 at 5:58 PM, Hans Lund <ha.lund@...> wrote:
> > >
> > > > I'm trying to resolve dependencies using specified a specified status
> > > >
> > > > having the following in settings.
> > > >
> > > > --------
> > > > <statuses default="integration">
> > > >        <status name="release" integration="false"/>
> > > >        <status name="DEV" integration="true"/>
> > > >        <status name="integration" integration="true"/>
> > > > </statuses>
> > > > ---------
> > > >
> > > > Now in the ivy file I have some dependencies like the following (where
> > > > ${revision.status} is set in ant depending on source-branch.
> > > >
> > > > ------------
> > > > <dependency org="com.multisupport"
> > > > name="commons"rev="latest.${revision.status}" conf="compile->default(*)
> > > > "/>
> > > > ------------
> > > >
> > > >
> > > > Running:
> > > >
> > > > <ivy:resolve conf="runtime" />
> > > > As long as it runs this works as expected.
> > > >
> > > > But: After a short while running on a CI (hudson) server, where the ivy
> > > > repository also is placed, resolving stops working due to connection
> > > > problems in httpClient.
> > > > ---
> > > >
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > > processing request: Address already in use: connect
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: Retrying request
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > > processing request: Address already in use: connect
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: Retrying request
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > > processing request: Address already in use: connect
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: Retrying request
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > > processing request: Address already in use: connect
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: Retrying request
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: I/O exception (java.net.BindException) caught when
> > > > processing request: Address already in use: connect
> > > > [ivy:resolve] 01-07-2008 13:16:24
> > > > org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
> > > > [ivy:resolve] INFO: Retrying request ...............  and so on:
> > > >
> > > >
> > > > which in the end results in unresolved dependencies.
> > > >
> > > > It looks like that the problem might be that every dependency is
> > downloaded
> > > > (ivy, checksum, artifacts) and after ~
> > > > all connections on the server are used.
> > > > So in effect <ivy:resolve> performs a dos against the repository.
> > > >
> > > > Does any one had any similar experiences, and have a solution?
> > > > -- what I can come up with is to clean up the dev repository, but I
> > find
> > > > that somewhat a no-solution
> > > >
> > > >
> > > >
> > > > Should this be considered an ivy bug?
> > >
> > > Absolutely, please open a JIRA. Could you test also without httpclient,
> > to
> > > see if it makes any difference?
> > >
> > > Xavier
> >
> > OK, I've created a JIRA bug report
> > http://issues.apache.org/jira/browse/IVY-854
> >
> > I think that further discussion should be on ivy-dev ?
>
> Yes, probably (and ivy-dev is actually ant-dev). Not sure if there is much
> people from Ivy team currently checking their e-mails though.
>
> Xavier

OK, I've subscribed to ant-dev, and wrote some ideas there. But here my
current 'fix' if anyone should encounter something similar:

Be sure to scale the tcp stack on the hosting http-server (your
repository), and most importantly clean-up the repository for old not in
use published revisions. For every revision you delete you will save 3
http connections when resolving on latest.status!

Hans

 
>
>
> >
> > /Hans Lund
> >
> >
> >
>
>