Generic build system?

[Jean Louis BOUDART]

Hi All,

I've seen a few time ago on this mailing list a thread about "easy ant",
but actually there is no implementation of.
I'm currently working on a generic build-system for my  company, and i  
want to share my experience.

History :
Each time we create a new project we have to write a specific build
script. One of the main issue could be dependencies of the project.
That's why we use a dependency manager :). Looking for that kind of
stuff on
the web you can find Maven and  Ivy.

What i like in maven is "conventions" of commands, every projects you'll
find that use maven uses the same "conventions", so you're never lost.
But what i dislike in maven is that every time you want to do "complex"
things you have to create "ant" hooks. Maven repository are as cleaner
as my bedroom :), and quite all the time asking a repository is long
especially when you consider you are downloading two thousand millions
of plug ins that you even don't know any existence of.

Hum maybe maven is not the solution that i'm looking for.

That's why I chose to use ivy which is really more powerful and
flexible. But the main problem is that when one is using  Ant + Ivy
there is no "conventions",  maybe  i'll name a target called build and
my neighborhood  will call this target "compile".

I don't want to loose time on writing a new build.xml that use my repo
on every new project of my company.
That's why i've made a generic build.xml that have all the common task
we use in "standard" project.
By "common task" i mean :
   - clean
   - build
   - resolve
   - dist (making jar / war / ear)
   - lauch tests
   - generate javadoc
   - jetty-run "a la maven"
   - have dependencies report
   - have a way to add artifiact on my repo
   - generate javadoc
   - generate a skeleton of project (build.xml + build.properties +
ivy.xml + empty src directory)
   - etc... :p

Every project import the common-build system and can use or override
properties or targets
Exemple: on every web project i can run "ant jetty-run" that
automatically :
    - resolve dependencies
    - build the application
    - make war
    - download jetty (if it wasn't downloaded before)
    - deploy the war in jetty

And wow miracle i can access my webapp quickly with my favorite webbrowser.

Exemple2: I want to build an signed applet , i'll can override the
"dist" target to use sign-jar which is specific to "my-applet" project

Exemple3: I have a strange project that use a "non official" repo, maybe
i'll want to use "common repo" (given by my build-system) and add the
specific repo only on my project.

The code is still in early beta but you can find it here :
http://ejbca-fr.org/svn/build-system/trunk/

For the moment the code is rather specific my company but can be quickly
opened and become a really open source project, i'm writing the
documentation (actually only in french).

So if you're interested by this project, or if you want to give your
point of view or advices feel free to contact me.

Cheers

[Jim White]

Jean Louis BOUDART wrote:

> I've seen a few time ago on this mailing list a thread about "easy ant",
> but actually there is no implementation of.

Well, I'm sure Xavier may see that differently.  ;-)

Another Ant + Ivy build template system that I like is the one used by
Spring WebFlow.

http://springframework.svn.sourceforge.net/viewvc/springframework/spring-webflow/trunk/


> I'm currently working on a generic build-system for my  company, and i  
> want to share my experience.
> ...

Excellent work.

> So if you're interested by this project, or if you want to give your
> point of view or advices feel free to contact me.

I'm interested in all Ant-based build systems and one of the projects I
started is called AntAnywhere which takes any Ant script (and its
associated files) and packages it as JNLP (Java Web Start) so it can be
"clickable".  The application being for WORA applications more so than
application building per se.

http://www.ifcx.org/wiki/AntAnywhere.html

I also do a lot with Groovy and one of its good features is making Ant
tasks easily used as functions.  There are also several new build tools
based on that.

http://www.google.com/search?q=groovy+build+tools

Jim

[chrishane]

You might look at: http://gradle.org/

It's based on groovy and looks very easy to create build scripts.

Chris....

Jean Louis BOUDART wrote:

> Hi All,
>
> I've seen a few time ago on this mailing list a thread about "easy ant",
> but actually there is no implementation of.
> I'm currently working on a generic build-system for my  company, and i  
> want to share my experience.
>
> History :
> Each time we create a new project we have to write a specific build
> script. One of the main issue could be dependencies of the project.
> That's why we use a dependency manager :). Looking for that kind of
> stuff on
> the web you can find Maven and  Ivy.
>
> What i like in maven is "conventions" of commands, every projects you'll
> find that use maven uses the same "conventions", so you're never lost.
> But what i dislike in maven is that every time you want to do "complex"
> things you have to create "ant" hooks. Maven repository are as cleaner
> as my bedroom :), and quite all the time asking a repository is long
> especially when you consider you are downloading two thousand millions
> of plug ins that you even don't know any existence of.
>
> Hum maybe maven is not the solution that i'm looking for.
>
> That's why I chose to use ivy which is really more powerful and
> flexible. But the main problem is that when one is using  Ant + Ivy
> there is no "conventions",  maybe  i'll name a target called build and
> my neighborhood  will call this target "compile".
>
> I don't want to loose time on writing a new build.xml that use my repo
> on every new project of my company.
> That's why i've made a generic build.xml that have all the common task
> we use in "standard" project.
> By "common task" i mean :
>   - clean
>   - build
>   - resolve
>   - dist (making jar / war / ear)
>   - lauch tests
>   - generate javadoc
>   - jetty-run "a la maven"
>   - have dependencies report
>   - have a way to add artifiact on my repo
>   - generate javadoc
>   - generate a skeleton of project (build.xml + build.properties +
> ivy.xml + empty src directory)
>   - etc... :p
>
> Every project import the common-build system and can use or override
> properties or targets
> Exemple: on every web project i can run "ant jetty-run" that
> automatically :
>    - resolve dependencies
>    - build the application
>    - make war
>    - download jetty (if it wasn't downloaded before)
>    - deploy the war in jetty
>
> And wow miracle i can access my webapp quickly with my favorite webbrowser.
>
> Exemple2: I want to build an signed applet , i'll can override the
> "dist" target to use sign-jar which is specific to "my-applet" project
>
> Exemple3: I have a strange project that use a "non official" repo, maybe
> i'll want to use "common repo" (given by my build-system) and add the
> specific repo only on my project.
>
> The code is still in early beta but you can find it here :
> http://ejbca-fr.org/svn/build-system/trunk/
>
> For the moment the code is rather specific my company but can be quickly
> opened and become a really open source project, i'm writing the
> documentation (actually only in french).
>
> So if you're interested by this project, or if you want to give your
> point of view or advices feel free to contact me.
>
> Cheers
>
>
>
>
>

[Xavier Hanin]

On Tue, Jul 8, 2008 at 9:52 PM, Jim White <jim@...> wrote:

> Jean Louis BOUDART wrote:
>
> I've seen a few time ago on this mailing list a thread about "easy ant",
>> but actually there is no implementation of.
>>
>
> Well, I'm sure Xavier may see that differently.  ;-)

I only have a prototype, and hasn't had much feedback on it, except from
Stefan Bodewig. So for the moment I do not devote any more time to the
project. But I'm pleased to see this kind of initiative... it would really
be nice if the Ant + Ivy community could come up with a good out of the box
solution.

Xavier

>
>
> Another Ant + Ivy build template system that I like is the one used by
> Spring WebFlow.
>
>
> http://springframework.svn.sourceforge.net/viewvc/springframework/spring-webflow/trunk/
>
>
>  I'm currently working on a generic build-system for my  company, and i
>>  want to share my experience.
>> ...
>>
>
> Excellent work.
>
> So if you're interested by this project, or if you want to give your point
>> of view or advices feel free to contact me.
>>
>
> I'm interested in all Ant-based build systems and one of the projects I
> started is called AntAnywhere which takes any Ant script (and its associated
> files) and packages it as JNLP (Java Web Start) so it can be "clickable".
>  The application being for WORA applications more so than application
> building per se.
>
> http://www.ifcx.org/wiki/AntAnywhere.html
>
> I also do a lot with Groovy and one of its good features is making Ant
> tasks easily used as functions.  There are also several new build tools
> based on that.
>
> http://www.google.com/search?q=groovy+build+tools
>
> Jim
>
>

--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/

[Russel Winder-4]

On Tue, 2008-07-08 at 12:52 -0700, Jim White wrote:

> I also do a lot with Groovy and one of its good features is making Ant
> tasks easily used as functions.  There are also several new build tools
> based on that.
>
> http://www.google.com/search?q=groovy+build+tools

Being more explict:

Groovy has the AntBuilder which makes programming Ant tasks with Groovy
very straightforward.

Gant is a lightweight framework around AntBuilder and Groovy to provide
the infrastructure that everyone would replicate if they had only Groovy
and AntBuilder.  Gant's goal is to make Ant task scripting using Groovy
really easy.   It has various little toolkits and target sets, including
an Ivy tool.

Gradle is a complete convention over configuration build system using
Groovy and ivy.

Given these, hopefully the days of having to handcraft XML files are
numbered :-)

--
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: [phone], [phone]
41 Buckmaster Road,              f: [phone]
London SW11 1EN, UK.             m: [phone]

[Jean Louis BOUDART]

Using a script language like groovy for build system sounds interesting.
Doing complexe things will be easier, using a groovy than doing it in
Ant-XML.

What i doesn't like in gradle is the "new syntax" for dependencies
declaration, 'cause you can't use all the stuff behind ivy (like the
eclipse plugin IvyDe) or you must configure application tu use  ivy.xml
generated by  gradle. :/
What about integration with continuous integration?

I think that easyant seems to be more closed to what i'm looking for, even
if it's only a "POC"  version.

Being more explicits about what i like in  easyant :
    It seems to be really easy to override or extends existing targets and
path.
    There is different kind of "build module". build-plugins are separated
of phases (inspired by the same concept in maven).
    On basic project you only need an ivy.xml file on your project.

I'll try to merge my own build-system with easy ant i'll give you a
feedback in few days.

Cheers
Jean-Louis




Russel Winder a écrit :

> On Tue, 2008-07-08 at 12:52 -0700, Jim White wrote:
>
>> I also do a lot with Groovy and one of its good features is making Ant
>> tasks easily used as functions.  There are also several new build tools
>> based on that.
>>
>> http://www.google.com/search?q=groovy+build+tools
>
> Being more explict:
>
> Groovy has the AntBuilder which makes programming Ant tasks with Groovy
> very straightforward.
>
> Gant is a lightweight framework around AntBuilder and Groovy to provide
> the infrastructure that everyone would replicate if they had only Groovy
> and AntBuilder.  Gant's goal is to make Ant task scripting using Groovy
> really easy.   It has various little toolkits and target sets, including
> an Ivy tool.
>
> Gradle is a complete convention over configuration build system using
> Groovy and ivy.
>
> Given these, hopefully the days of having to handcraft XML files are
> numbered :-)
>

[hdockter]

Hi Jean-Louis,

On Jul 12, 2008, at 12:44 PM, Jean-Louis Boudart wrote:

> Using a script language like groovy for build system sounds  
> interesting.
> Doing complexe things will be easier, using a groovy than doing it in
> Ant-XML.
>
> What i doesn't like in gradle is the "new syntax" for dependencies
> declaration, 'cause you can't use all the stuff behind ivy (like the
> eclipse plugin IvyDe) or you must configure application tu use  
> ivy.xml
> generated by  gradle. :/

The whole point of Gradle is to offer more power and freedom.  
Configuring the dependencies in Groovy offers more possibilities. One  
use case from the Gradle build itself:

Gradle ships with its own version of Groovy which is part of the  
dependency declaration. On our CI server we have one build  
configuration were we build and test Gradle with the latest Groovy.  
To do this we just pass an argument with the URL to the Groovy jar we  
are interested. If this argument is set, we download the jar as part  
of the build script and add it to our dependencies instead of the  
default jar. This is very quick solution to a problem we had.  
Additionally we have introduced new semantic concepts on top of Ivy  
like client modules. You could not use them via XML.

But of course, as you have mentioned, it also has disadvantages not  
to use the ivy.xml. It would be pretty easy to enhance Gradle to also  
accept ivy.xml's and ivysettings.xml's. It should even be easy to mix  
both forms of declarations. There are just other things which have a  
higher priority for us at the moment. If this is your biggest worry:  
Why not contributing this to Gradle ;) It would definitely take less  
time than creating an Ant based build by convention approach.

I think one big problem with Ant is that is had to introduce many  
general purpose language elements in its XML DSL to deal with complex  
builds. Those constructs are Ant specific, therefore have a high  
learning curve. And yet they are a pale imitation (thanks for this  
phrase, Russel ;)) of what a full blown OO language has to offer.

> What about integration with continuous integration?

This is no problem for Gradle. With the help of the Gradle Wrapper  
you can run Gradle builds on any CI server without the need to have  
Gradle installed there. See the Gradle user's guide.

- Hans

--
Hans Dockter
Gradle Project lead
http://www.gradle.org

[Russel Winder-4]

On Sat, 2008-07-12 at 12:44 +0200, Jean-Louis Boudart wrote:
> Using a script language like groovy for build system sounds interesting.
> Doing complexe things will be easier, using a groovy than doing it in
> Ant-XML.

Very much so.  The two systems we have at the moment are Gant and
Gradle.  Gant is just a "script Ant tasks with Groovy" system whereas
Gradle is a fully-fledged convention-over-configuration build system.

> What i doesn't like in gradle is the "new syntax" for dependencies
> declaration, 'cause you can't use all the stuff behind ivy (like the
> eclipse plugin IvyDe) or you must configure application tu use  ivy.xml
> generated by  gradle. :/

I'll leave this one to Hans.  I am sure though that there must be a
solution.  Whatever Ant and Maven can do, Gant and Gradle can almost
certainly do more easily since they are using Groovy instead of XML.

> What about integration with continuous integration?

Hudson has seen the light and can launch Gant directly.  Buildbot can
launch with any system, so working with Gant and Gradle is trivial.
Other systems such as Bamboo and CruiseControl are moderately less
up-to-date and so only offer Ant and Maven.  Gant offers an Ant task to
allow these systems to use Ant to launch a Gant build.  I would guess
that the Gradle wrapper would allow Gradle builds to be started from
systems that do not know about Gradle.
 
> I think that easyant seems to be more closed to what i'm looking for, even
> if it's only a "POC"  version.

I'll have to pass on that, I have not tried it.

> Being more explicits about what i like in  easyant :
>     It seems to be really easy to override or extends existing targets and
> path.
>     There is different kind of "build module". build-plugins are separated
> of phases (inspired by the same concept in maven).
>     On basic project you only need an ivy.xml file on your project.
>
> I'll try to merge my own build-system with easy ant i'll give you a
> feedback in few days.
>
> Cheers
> Jean-Louis

--
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: [phone], [phone]
41 Buckmaster Road,              f: [phone]
London SW11 1EN, UK.             m: [phone]