Ivy RoundUp Repository - feedback requested

[Archie Cobbs]

Hello fellow Ivy users,

I'd like to announce a new little project I've started, and ask for your
feedback (and help, if interested).

This project has two basic parts...

   1. *Builder Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
   a new Ivy resolver that accesses ivy files and "build instructions" from an
   online "builder" repository. "Builder" repositories contain ivy.xml files
   but no artifacts. To get the artifacts, the build instructions are
   downloaded from the repository and executed locally. These instructions
   specify additional resource(s) to download and how to build the artifacts
   from them, for example, by downloading a project's original distribution
   archive directly from their web site and extracting the desired artifacts.
   2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
   online, open-source community "Builder" repository for all Ivy users.

Please click the links for more info and documentation.

I am lobbying to get the builder resolver added into Ivy itself; right now
it's still in patch form (you can download a pre-built ivy.jar from the
project website).

Some motivations for starting this project:

   1. Ivyrep is no longer maintained, but we need a decent community Ivy
   repository that everyone can share
   2. Hosting hundreds of large files that are just copies of the same
   files available elsewhere is expensive and redundant, so let's avoid doing
   that
   3. 99% of projects out there do not publish ivy.xml files, so we need
   a community project that focuses on developing and maintaining them
   4. To get the most out of Ivy, there needs to be a consistent set of
   guidelines for creating ivy.xml files: how to choose organization names,
   philosophy for defining configurations, etc. A community project supported
   by Ivy users can provide this.

What I want to do is gauge interest in this idea and ask for any volunteers
who'd like to start adding and maintaining meta-data for their favorite
projects. The Ivy RoundUp repository is online now, though only as a
proof-of-concept (it only contains a few modules so far). Take a look and
you should be able to get the general idea:
http://ivyroundup.googlecode.com/

In the worst case, if nobody else is interested, I will just use this for
myself -- it's already working better than what I was doing (i.e., checking
in giant ZIP files into Subversion and creating a project for every one to
publish into our private Ivy repository), and in any case the work of
setting it up is already done. Note also anyone could create their own
private builder repository using this project as well.

In the best case, we'll put together a piece of infrastructure that all Ivy
users can really benefit from.

Let me know what you think.

Thanks,
-Archie

--
Archie L. Cobbs

[jonathan doklovic]

Interesting idea.

I really think there is a need for a stable public repo as well as some
standards for conf naming and such.

I'm not sure I'm a fan of the build instructions though. It seems that
pulling artifacts from the creators websites has some problems. Off the
top of my head...  it could add a lot of time to resolve since a lot of
OS sites are slow. (including sourceforge).  Also, as time goes by,
sites have a tendency to archive artifacts and sometimes even remove
them altogether, which would then in turn break the build instructions
in either case.  Just seems like it might be more maintenance than it's
worth.

In my opinion, disk space is cheap and it would be more reliable to
store copies of the artifacts in the public repo.

Cool work though.

- Jonathan


On Tue, 2008-04-15 at 10:06 -0500, Archie Cobbs wrote:

> Hello fellow Ivy users,
>
> I'd like to announce a new little project I've started, and ask for your
> feedback (and help, if interested).
>
> This project has two basic parts...
>
>    1. *Builder Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
>    a new Ivy resolver that accesses ivy files and "build instructions" from an
>    online "builder" repository. "Builder" repositories contain ivy.xml files
>    but no artifacts. To get the artifacts, the build instructions are
>    downloaded from the repository and executed locally. These instructions
>    specify additional resource(s) to download and how to build the artifacts
>    from them, for example, by downloading a project's original distribution
>    archive directly from their web site and extracting the desired artifacts.
>    2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
>    online, open-source community "Builder" repository for all Ivy users.
>
> Please click the links for more info and documentation.
>
> I am lobbying to get the builder resolver added into Ivy itself; right now
> it's still in patch form (you can download a pre-built ivy.jar from the
> project website).
>
> Some motivations for starting this project:
>
>    1. Ivyrep is no longer maintained, but we need a decent community Ivy
>    repository that everyone can share
>    2. Hosting hundreds of large files that are just copies of the same
>    files available elsewhere is expensive and redundant, so let's avoid doing
>    that
>    3. 99% of projects out there do not publish ivy.xml files, so we need
>    a community project that focuses on developing and maintaining them
>    4. To get the most out of Ivy, there needs to be a consistent set of
>    guidelines for creating ivy.xml files: how to choose organization names,
>    philosophy for defining configurations, etc. A community project supported
>    by Ivy users can provide this.
>
> What I want to do is gauge interest in this idea and ask for any volunteers
> who'd like to start adding and maintaining meta-data for their favorite
> projects. The Ivy RoundUp repository is online now, though only as a
> proof-of-concept (it only contains a few modules so far). Take a look and
> you should be able to get the general idea:
> http://ivyroundup.googlecode.com/
>
> In the worst case, if nobody else is interested, I will just use this for
> myself -- it's already working better than what I was doing (i.e., checking
> in giant ZIP files into Subversion and creating a project for every one to
> publish into our private Ivy repository), and in any case the work of
> setting it up is already done. Note also anyone could create their own
> private builder repository using this project as well.
>
> In the best case, we'll put together a piece of infrastructure that all Ivy
> users can really benefit from.
>
> Let me know what you think.
>
> Thanks,
> -Archie
>

[Archie Cobbs]

On Tue, Apr 15, 2008 at 10:22 AM, jonathan doklovic <jdoklovic@...>
wrote:

> I'm not sure I'm a fan of the build instructions though. It seems that
> pulling artifacts from the creators websites has some problems. Off the
> top of my head...  it could add a lot of time to resolve since a lot of
> OS sites are slow. (including sourceforge).  Also, as time goes by,
> sites have a tendency to archive artifacts and sometimes even remove
> them altogether, which would then in turn break the build instructions
> in either case.  Just seems like it might be more maintenance than it's
> worth.
>
>

My thinking is that generally you'd try to avoid "building". Instead, use
Ivy's cache, and/or the builder resolver's resource cache. So if the
download is slow, at least it's only slow the first time... this helps but
doesn't eliminate the problem.

As for the problem of the original distributions disappearing, that is also
a valid point. It can be treated as an orthogonal problem however. For
example, there's no reason the build instructions can't point to resources
hosted on some "permanent mirror" site set up just for that purpose, instead
of the original projects' site, etc. Perhaps that can be a follow-on project
to this one to add robustness.

A way to address both of your comments would be to allow configuration of a
"resource URL override" pattern: when configured, the resource URLs in the
build instructions would be ignored, and instead all resources would be
downloaded using the override URL pattern to create the URL. Then you could
keep your own private mirror. Hmm, that should be easy to add.

Thanks for the comments.

-Archie

--
Archie L. Cobbs

[Archie Cobbs]

On Tue, Apr 15, 2008 at 10:48 AM, Archie Cobbs <archie@...> wrote:

> A way to address both of your comments would be to allow configuration of
> a "resource URL override" pattern: when configured, the resource URLs in the
> build instructions would be ignored, and instead all resources would be
> downloaded using the override URL pattern to create the URL. Then you could
> keep your own private mirror. Hmm, that should be easy to add.
>
>
I've added this capability using a new "resourceURL" attribute and updated
the patch, ivy.jar, and documentation on the website.

Thanks,
-Archie

--
Archie L. Cobbs

[Xavier Hanin]

On Tue, Apr 15, 2008 at 5:22 PM, jonathan doklovic <jdoklovic@...>
wrote:

> Interesting idea.
>
> I really think there is a need for a stable public repo as well as some
> standards for conf naming and such.
>
> I'm not sure I'm a fan of the build instructions though. It seems that
> pulling artifacts from the creators websites has some problems. Off the
> top of my head...  it could add a lot of time to resolve since a lot of
> OS sites are slow. (including sourceforge).  Also, as time goes by,
> sites have a tendency to archive artifacts and sometimes even remove
> them altogether, which would then in turn break the build instructions
> in either case.  Just seems like it might be more maintenance than it's
> worth.
>
> In my opinion, disk space is cheap and it would be more reliable to
> store copies of the artifacts in the public repo.

I agree with your arguments, but I do think storing build instructions is a
very good idea. By storing the build instructions, you get an easier import
of new versions from the project: let's say you have already written build
instructions for foo 1.1, when foo 1.2 is released, there is good chance
that your build instructions will only require minor adjustments. If
downloading, unzipping, repackaging sources and javadocs were done by hand,
you would have to reproduce the same steps each time.

Now I see this approach as an intermediary step: once you have a repository
with build instructions, it's super easy to setup a repository with actual
artifacts, built automatically from the build instructions. And if enough
people use the builder approach, I guess there will be enough interest to
get money to setup this repository.

That being said, I see a problem with hosting the repository in google code
svn repo: access seems to be very slow sometimes, and subversion doesn't
help. Maybe it would be interesting to have a public file server launching
an svn up in a cron? I have a small box which I use for xoocode.org hosting,
I could set this up if you think it's interesting.

Xavier

>
>
> Cool work though.
>
> - Jonathan
>
>
> On Tue, 2008-04-15 at 10:06 -0500, Archie Cobbs wrote:
> > Hello fellow Ivy users,
> >
> > I'd like to announce a new little project I've started, and ask for your
> > feedback (and help, if interested).
> >
> > This project has two basic parts...
> >
> >    1. *Builder Resolver*<
> http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> >    a new Ivy resolver that accesses ivy files and "build instructions"
> from an
> >    online "builder" repository. "Builder" repositories contain ivy.xml
> files
> >    but no artifacts. To get the artifacts, the build instructions are
> >    downloaded from the repository and executed locally. These
> instructions
> >    specify additional resource(s) to download and how to build the
> artifacts
> >    from them, for example, by downloading a project's original
> distribution
> >    archive directly from their web site and extracting the desired
> artifacts.
> >    2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
> >    online, open-source community "Builder" repository for all Ivy users.
> >
> > Please click the links for more info and documentation.
> >
> > I am lobbying to get the builder resolver added into Ivy itself; right
> now
> > it's still in patch form (you can download a pre-built ivy.jar from the
> > project website).
> >
> > Some motivations for starting this project:
> >
> >    1. Ivyrep is no longer maintained, but we need a decent community Ivy
> >    repository that everyone can share
> >    2. Hosting hundreds of large files that are just copies of the same
> >    files available elsewhere is expensive and redundant, so let's avoid
> doing
> >    that
> >    3. 99% of projects out there do not publish ivy.xml files, so we need
> >    a community project that focuses on developing and maintaining them
> >    4. To get the most out of Ivy, there needs to be a consistent set of
> >    guidelines for creating ivy.xml files: how to choose organization
> names,
> >    philosophy for defining configurations, etc. A community project
> supported
> >    by Ivy users can provide this.
> >
> > What I want to do is gauge interest in this idea and ask for any
> volunteers
> > who'd like to start adding and maintaining meta-data for their favorite
> > projects. The Ivy RoundUp repository is online now, though only as a
> > proof-of-concept (it only contains a few modules so far). Take a look
> and
> > you should be able to get the general idea:
> > http://ivyroundup.googlecode.com/
> >
> > In the worst case, if nobody else is interested, I will just use this
> for
> > myself -- it's already working better than what I was doing (i.e.,
> checking
> > in giant ZIP files into Subversion and creating a project for every one
> to
> > publish into our private Ivy repository), and in any case the work of
> > setting it up is already done. Note also anyone could create their own
> > private builder repository using this project as well.
> >
> > In the best case, we'll put together a piece of infrastructure that all
> Ivy
> > users can really benefit from.
> >
> > Let me know what you think.
> >
> > Thanks,
> > -Archie
> >
>

--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/

[Archie Cobbs]

On Wed, Apr 16, 2008 at 2:41 AM, Xavier Hanin <xavier.hanin@...>
wrote:

> That being said, I see a problem with hosting the repository in google
> code
> svn repo: access seems to be very slow sometimes, and subversion doesn't
> help. Maybe it would be interesting to have a public file server launching
> an svn up in a cron? I have a small box which I use for xoocode.orghosting,
> I could set this up if you think it's interesting.


Yes, that would be easy. If you checkout the project, you'll see that when
you run "ant" it builds the entire repository as a directory tree under
repo/. Then, upon successful review, the developer would commit that repo/
subtree into SVN and it automatically becomes available via HTTP from Google
SVN.

Of course there's no reason you couldn't take repo/ and copy it to some
faster web server if you wanted, e.g. using svn update as you suggest.

I haven't experienced slowness with Google SVN, and the files being
downloaded are small, but on the other hand I haven't done any performance
testing.

Sure, let's try it. Here's the URL to checkout:
http://ivyroundup.googlecode.com/svn/trunk/repo

-Archie

--
Archie L. Cobbs

[Xavier Hanin]

On Wed, Apr 16, 2008 at 4:42 PM, Archie Cobbs <archie@...> wrote:

> On Wed, Apr 16, 2008 at 2:41 AM, Xavier Hanin <xavier.hanin@...>
> wrote:
>
> > That being said, I see a problem with hosting the repository in google
> > code
> > svn repo: access seems to be very slow sometimes, and subversion doesn't
> > help. Maybe it would be interesting to have a public file server
> launching
> > an svn up in a cron? I have a small box which I use for
> xoocode.orghosting,
> > I could set this up if you think it's interesting.
>
>
> Yes, that would be easy. If you checkout the project, you'll see that when
> you run "ant" it builds the entire repository as a directory tree under
> repo/. Then, upon successful review, the developer would commit that repo/
> subtree into SVN and it automatically becomes available via HTTP from
> Google
> SVN.
>
> Of course there's no reason you couldn't take repo/ and copy it to some
> faster web server if you wanted, e.g. using svn update as you suggest.
>
> I haven't experienced slowness with Google SVN, and the files being
> downloaded are small, but on the other hand I haven't done any performance
> testing.

This morning the repository was very slow, with even bad responses (empty
pages). But it was the first time I experienced this... So maybe it's not
worth trying to use something else ATM.

Xavier

>
>
> Sure, let's try it. Here's the URL to checkout:
> http://ivyroundup.googlecode.com/svn/trunk/repo
>
> -Archie
>
> --
> Archie L. Cobbs
>

--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/

[Jing Xue-2]

Quoting Archie Cobbs <archie@...>:

> Hello fellow Ivy users,
>
> I'd like to announce a new little project I've started, and ask for your
> feedback (and help, if interested).
>
> This project has two basic parts...
>
>    1. *Builder  
> Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
>    a new Ivy resolver that accesses ivy files and "build  
> instructions" from an
>    online "builder" repository. "Builder" repositories contain ivy.xml files
>    but no artifacts. To get the artifacts, the build instructions are
>    downloaded from the repository and executed locally. These instructions
>    specify additional resource(s) to download and how to build the artifacts
>    from them, for example, by downloading a project's original distribution
>    archive directly from their web site and extracting the desired artifacts.
>    2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
>    online, open-source community "Builder" repository for all Ivy users.

[snip]

I read the thread on the dev list, and was thoroughly interested in  
the idea of a community-maintained public ivy.xml repository, because  
a carefully defined ivy.xml provides a lot more value than the ones  
generated from the pom, and would be something worth sharing.

On the other hand, I'm not sure I see the value in the builder  
resolver. I guess I'm having a hard time imagining a use case where I  
would prefer building a 3rd party artifact locally rather than simply  
downloading it through an ibiblio resolver, or from a company-level  
proxy repository.

Cheers.
--
Jing Xue

[Archie Cobbs]

On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <jingxue@...> wrote:

> I read the thread on the dev list, and was thoroughly interested in the
> idea of a community-maintained public ivy.xml repository, because a
> carefully defined ivy.xml provides a lot more value than the ones generated
> from the pom, and would be something worth sharing.
>
> On the other hand, I'm not sure I see the value in the builder resolver. I
> guess I'm having a hard time imagining a use case where I would prefer
> building a 3rd party artifact locally rather than simply downloading it
> through an ibiblio resolver, or from a company-level proxy repository.


The problem is that not all artifacts are available as URLs. Many times,
they are packaged up inside a tar.gz file and the only way to get at them is
to extract them.

So without builder resolver, someone has to:

   1. Build and maintain an online repository that has huge disk space
   (and bandwidth) requirements
   2. Perform the manual download, extract, and publish steps every time
   a software has a new release

With builder resolver, #1 goes away entirely and #2 is reduced to updating a
SHA1 checksum (common case).

If you don't think #1 and #2 matter, then why aren't you volunteering to do
them for us? :-)

-Archie

--
Archie L. Cobbs

[Xavier Hanin]

On Wed, Apr 16, 2008 at 8:54 PM, Archie Cobbs <archie@...> wrote:

> On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <jingxue@...>
> wrote:
>
> > I read the thread on the dev list, and was thoroughly interested in the
> > idea of a community-maintained public ivy.xml repository, because a
> > carefully defined ivy.xml provides a lot more value than the ones
> generated
> > from the pom, and would be something worth sharing.
> >
> > On the other hand, I'm not sure I see the value in the builder resolver.
> I
> > guess I'm having a hard time imagining a use case where I would prefer
> > building a 3rd party artifact locally rather than simply downloading it
> > through an ibiblio resolver, or from a company-level proxy repository.
>
>
> The problem is that not all artifacts are available as URLs. Many times,
> they are packaged up inside a tar.gz file and the only way to get at them
> is
> to extract them.

I think the problem is in the intention and the meaning of "building"
artifacts. If I understand correctly, Jing think that the builder resolver
will actually compile and package the artifacts, which wouldn't make a lot
of sense. What the "builder" actually do is simply download the artifacts
from where they are over the net, and when the jar is actually contained in
an archive, do the extract. For some artifacts like javadoc and sources
which are not always provided as an artifact usable by IDE, the builder can
actually extract the zip where the source files can be obtained, and
repackage the source files in a zip which can be used by an IDE. For
instance if you want to create a module for Ivy 1.4.1, you could get the
zips from <http://www.jaya.free.fr/downloads/ivy/1.4.1/>, then unpack them
to get the main Ivy jars, and repack the sources and javadocs as usable
artifacts for IDEs.

The big advantage to do this automatically instead of manually is that the
work needed when a new minor version is released is usually minor: bump up
the version and update the sha1 and if the packaging hasn't changed you're
done! Then you can concentrate on metadata.

Xavier

>
> So without builder resolver, someone has to:
>
>   1. Build and maintain an online repository that has huge disk space
>   (and bandwidth) requirements
>   2. Perform the manual download, extract, and publish steps every time
>   a software has a new release
>
> With builder resolver, #1 goes away entirely and #2 is reduced to updating
> a
> SHA1 checksum (common case).
>
> If you don't think #1 and #2 matter, then why aren't you volunteering to
> do
> them for us? :-)
>
> -Archie
>
> --
> Archie L. Cobbs
>

--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/

[Gilles Scokart]

Jing was not the only one to misunderstand the "building" term... I did also.

Actually now the ivy roundup resolver as a resolver that allow you to
take the meta-data from the roundup repository and take the artefact
directly from the project distribution site!

And then, I'm suddently much more enthousiast about it!

It make me think to a thread that I read some time ago on the jmock
mailing list [1].
Initially, the jmock team didn't wanted to distribute themself their
library in a maven repository.  Their resistance was understandable
and had a right justification.  They would preffer someone else to
redistribute. They finaly did it themself (after some external
contributions) because of the pression of the maven users and because
it is better for the user to receive the library directly from the
team that build it.

With the ivy roundup aproach the user has to trust the ivy roundup
community to setup the right meta-data (as a user, I would have a to
look at how this community work, but It is possible that I could trust
them), and then I would have to trust the author of the library (and
if I use their library, I guess that I trust them.

With a maven2 redistribution aproach, I need to to trust the maven2
community for both the meta-data and the binaries.  It looks to me
more "risky" to trust.


All that to say : you should remove the image of "building".  The
resolver is a tool to get dependencies from their original
distributions site.  That should be reflected in the term you use if
you want to keep me enthousiastic :-)

[1] http://article.gmane.org/gmane.comp.java.jmock.user/972/match=maven+downstream+release

On 17/04/2008, Xavier Hanin <xavier.hanin@...> wrote:

> On Wed, Apr 16, 2008 at 8:54 PM, Archie Cobbs <archie@...> wrote:
>
>  > On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <jingxue@...>
>  > wrote:
>  >
>  > > I read the thread on the dev list, and was thoroughly interested in the
>  > > idea of a community-maintained public ivy.xml repository, because a
>  > > carefully defined ivy.xml provides a lot more value than the ones
>  > generated
>  > > from the pom, and would be something worth sharing.
>  > >
>  > > On the other hand, I'm not sure I see the value in the builder resolver.
>  > I
>  > > guess I'm having a hard time imagining a use case where I would prefer
>  > > building a 3rd party artifact locally rather than simply downloading it
>  > > through an ibiblio resolver, or from a company-level proxy repository.
>  >
>  >
>  > The problem is that not all artifacts are available as URLs. Many times,
>  > they are packaged up inside a tar.gz file and the only way to get at them
>  > is
>  > to extract them.
>
>
> I think the problem is in the intention and the meaning of "building"
>  artifacts. If I understand correctly, Jing think that the builder resolver
>  will actually compile and package the artifacts, which wouldn't make a lot
>  of sense. What the "builder" actually do is simply download the artifacts
>  from where they are over the net, and when the jar is actually contained in
>  an archive, do the extract. For some artifacts like javadoc and sources
>  which are not always provided as an artifact usable by IDE, the builder can
>  actually extract the zip where the source files can be obtained, and
>  repackage the source files in a zip which can be used by an IDE. For
>  instance if you want to create a module for Ivy 1.4.1, you could get the
>  zips from <http://www.jaya.free.fr/downloads/ivy/1.4.1/>, then unpack them
>  to get the main Ivy jars, and repack the sources and javadocs as usable
>  artifacts for IDEs.
>
>  The big advantage to do this automatically instead of manually is that the
>  work needed when a new minor version is released is usually minor: bump up
>  the version and update the sha1 and if the packaging hasn't changed you're
>  done! Then you can concentrate on metadata.
>
>
>  Xavier
>
>
>
>  >
>  > So without builder resolver, someone has to:
>  >
>  >   1. Build and maintain an online repository that has huge disk space
>  >   (and bandwidth) requirements
>  >   2. Perform the manual download, extract, and publish steps every time
>  >   a software has a new release
>  >
>  > With builder resolver, #1 goes away entirely and #2 is reduced to updating
>  > a
>  > SHA1 checksum (common case).
>  >
>  > If you don't think #1 and #2 matter, then why aren't you volunteering to
>  > do
>  > them for us? :-)
>  >
>  > -Archie
>  >
>  > --
>  > Archie L. Cobbs
>  >
>
>
>
>
> --
>  Xavier Hanin - Independent Java Consultant
>  http://xhab.blogspot.com/
>  http://ant.apache.org/ivy/
>  http://www.xoocode.org/
>

--
Gilles Scokart

[Nicolas Lalevée-2]

Le jeudi 17 avril 2008, Gilles Scokart a écrit :

> Jing was not the only one to misunderstand the "building" term... I did
> also.
>
> Actually now the ivy roundup resolver as a resolver that allow you to
> take the meta-data from the roundup repository and take the artefact
> directly from the project distribution site!
>
> And then, I'm suddently much more enthousiast about it!
>
> It make me think to a thread that I read some time ago on the jmock
> mailing list [1].
> Initially, the jmock team didn't wanted to distribute themself their
> library in a maven repository.  Their resistance was understandable
> and had a right justification.  They would preffer someone else to
> redistribute. They finaly did it themself (after some external
> contributions) because of the pression of the maven users and because
> it is better for the user to receive the library directly from the
> team that build it.
>
> With the ivy roundup aproach the user has to trust the ivy roundup
> community to setup the right meta-data (as a user, I would have a to
> look at how this community work, but It is possible that I could trust
> them), and then I would have to trust the author of the library (and
> if I use their library, I guess that I trust them.
>
> With a maven2 redistribution aproach, I need to to trust the maven2
> community for both the meta-data and the binaries.  It looks to me
> more "risky" to trust.
>
>
> All that to say : you should remove the image of "building".  The
> resolver is a tool to get dependencies from their original
> distributions site.  That should be reflected in the term you use if
> you want to keep me enthousiastic :-)
>
> [1]
> http://article.gmane.org/gmane.comp.java.jmock.user/972/match=maven+downstr
>eam+release

This idea behind this repository is getting more clear to me too. :)
Then I cannot not compare it to Linux distributions. It reminds me an article
about debian packager, about the different responsabilities about developping
and packaging.

Then I wouldn't make a difference in trust between the maven community and an
ivy round up community. Their main job is about packaging, so we have to
trust both for that. If I can trust the ivy roundup community to produce a
reproductible build, then I would trust the maven community to execute a
reproductible build.
This make me think of gentoo VS ubuntu. The first one is just packaging build
script, the other one is packaging build. It sometimes can make sense in the
native world, but not in a jvm world.
On the other hand, maintaing build scripts is definitively a good idea. Just
like the debian/ubuntu community is maintaing their build scripts. As said
Xavier it helps the community to maintain a module along with the upgrading
versions. I would consider the Ivy RoundUp Repository being a repository for
maintainers more than for Ivy users. I think that Ivy users should only see
jars.

That discussion makes me also think of the Orbit community. There are building
and releasing entire osgi bundle repositories. See for instance:
http://download.eclipse.org/tools/orbit/committers/drops/I20080416020842
Thought I don't know how practical it is.

Nicolas

>
> On 17/04/2008, Xavier Hanin <xavier.hanin@...> wrote:
> > On Wed, Apr 16, 2008 at 8:54 PM, Archie Cobbs <archie@...> wrote:
> >  > On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <jingxue@...>
> >  >
> >  > wrote:
> >  > > I read the thread on the dev list, and was thoroughly interested in
> >  > > the idea of a community-maintained public ivy.xml repository,
> >  > > because a carefully defined ivy.xml provides a lot more value than
> >  > > the ones
> >  >
> >  > generated
> >  >
> >  > > from the pom, and would be something worth sharing.
> >  > >
> >  > > On the other hand, I'm not sure I see the value in the builder
> >  > > resolver.
> >  >
> >  > I
> >  >
> >  > > guess I'm having a hard time imagining a use case where I would
> >  > > prefer building a 3rd party artifact locally rather than simply
> >  > > downloading it through an ibiblio resolver, or from a company-level
> >  > > proxy repository.
> >  >
> >  > The problem is that not all artifacts are available as URLs. Many
> >  > times, they are packaged up inside a tar.gz file and the only way to
> >  > get at them is
> >  > to extract them.
> >
> > I think the problem is in the intention and the meaning of "building"
> >  artifacts. If I understand correctly, Jing think that the builder
> > resolver will actually compile and package the artifacts, which wouldn't
> > make a lot of sense. What the "builder" actually do is simply download
> > the artifacts from where they are over the net, and when the jar is
> > actually contained in an archive, do the extract. For some artifacts like
> > javadoc and sources which are not always provided as an artifact usable
> > by IDE, the builder can actually extract the zip where the source files
> > can be obtained, and repackage the source files in a zip which can be
> > used by an IDE. For instance if you want to create a module for Ivy
> > 1.4.1, you could get the zips from
> > <http://www.jaya.free.fr/downloads/ivy/1.4.1/>, then unpack them to get
> > the main Ivy jars, and repack the sources and javadocs as usable
> > artifacts for IDEs.
> >
> >  The big advantage to do this automatically instead of manually is that
> > the work needed when a new minor version is released is usually minor:
> > bump up the version and update the sha1 and if the packaging hasn't
> > changed you're done! Then you can concentrate on metadata.
> >
> >
> >  Xavier
> >
> >  > So without builder resolver, someone has to:
> >  >
> >  >   1. Build and maintain an online repository that has huge disk space
> >  >   (and bandwidth) requirements
> >  >   2. Perform the manual download, extract, and publish steps every
> >  > time a software has a new release
> >  >
> >  > With builder resolver, #1 goes away entirely and #2 is reduced to
> >  > updating a
> >  > SHA1 checksum (common case).
> >  >
> >  > If you don't think #1 and #2 matter, then why aren't you volunteering
> >  > to do
> >  > them for us? :-)
> >  >
> >  > -Archie
> >  >
> >  > --
> >  > Archie L. Cobbs
> >
> > --
> >  Xavier Hanin - Independent Java Consultant
> >  http://xhab.blogspot.com/
> >  http://ant.apache.org/ivy/
> >  http://www.xoocode.org/

--
Nicolas LALEVÉE
ANYWARE TECHNOLOGIES
Tel : [phone]
Fax : [phone]
http://www.anyware-tech.com

[Gilles Scokart]

A few more feed back:

1. I like the fact that the ivy.xml and build.xml contains a license
boilerplate.  I think it should a rule, and that the AL should be
applied systematically for all meta-data info stored in ivy-rep
2. As I said before, I don't like the term 'build'.  Something like
'download', 'install', 'get', etc... would be much better.
3. Signatures should be considered, and all meta-data files should be signed.
4. You should consider versioning of the meta-data.   ivyroundup will
be a downstream distributor, you can make errors that you will have to
fix, without asking a new release from your upstream distributor, and
still allowing build reproduceability.
(by build reproduceability, I mean that the user must be able to
define with high precision what he want to change).
5. A dependency repository needs to :
                        - Notify users in case of a new version of a module that he use.
                        - Provides release notes links.
6. You might encounter hosting issues


On 15/04/2008, Archie Cobbs <archie@...> wrote:

> Hello fellow Ivy users,
>
>  I'd like to announce a new little project I've started, and ask for your
>  feedback (and help, if interested).
>
>  This project has two basic parts...
>
>    1. *Builder Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
>    a new Ivy resolver that accesses ivy files and "build instructions" from an
>    online "builder" repository. "Builder" repositories contain ivy.xml files
>    but no artifacts. To get the artifacts, the build instructions are
>    downloaded from the repository and executed locally. These instructions
>    specify additional resource(s) to download and how to build the artifacts
>    from them, for example, by downloading a project's original distribution
>    archive directly from their web site and extracting the desired artifacts.
>    2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
>    online, open-source community "Builder" repository for all Ivy users.
>
>  Please click the links for more info and documentation.
>
>  I am lobbying to get the builder resolver added into Ivy itself; right now
>  it's still in patch form (you can download a pre-built ivy.jar from the
>  project website).
>
>  Some motivations for starting this project:
>
>    1. Ivyrep is no longer maintained, but we need a decent community Ivy
>    repository that everyone can share
>    2. Hosting hundreds of large files that are just copies of the same
>    files available elsewhere is expensive and redundant, so let's avoid doing
>    that
>    3. 99% of projects out there do not publish ivy.xml files, so we need
>    a community project that focuses on developing and maintaining them
>    4. To get the most out of Ivy, there needs to be a consistent set of
>    guidelines for creating ivy.xml files: how to choose organization names,
>    philosophy for defining configurations, etc. A community project supported
>    by Ivy users can provide this.
>
>  What I want to do is gauge interest in this idea and ask for any volunteers
>  who'd like to start adding and maintaining meta-data for their favorite
>  projects. The Ivy RoundUp repository is online now, though only as a
>  proof-of-concept (it only contains a few modules so far). Take a look and
>  you should be able to get the general idea:
>  http://ivyroundup.googlecode.com/
>
>  In the worst case, if nobody else is interested, I will just use this for
>  myself -- it's already working better than what I was doing (i.e., checking
>  in giant ZIP files into Subversion and creating a project for every one to
>  publish into our private Ivy repository), and in any case the work of
>  setting it up is already done. Note also anyone could create their own
>  private builder repository using this project as well.
>
>  In the best case, we'll put together a piece of infrastructure that all Ivy
>  users can really benefit from.
>
>  Let me know what you think.
>
>  Thanks,
>  -Archie
>
>
>  --
>  Archie L. Cobbs
>

--
Gilles Scokart