« Return to Thread: Gant 0.4 - Creating a powerful DSL for building Java projects

Re: Gant 0.4 - Creating a powerful DSL for building Java projects

by hdockter :: Rate this Message:

Hi Jochen,

On May 22, 2007, at 12:19 AM, Jochen Theodorou wrote:

Hans Dockter schrieb:

Hi,
For Gant 0.4 we want to add abstractions as known from Maven, like project layout, build lifecycle, dependencies, etc ...

As I was encouraging a build tool written in Groovy but never had the time to really help with Gant I feel a bit guilty ;) Anyway I had of course some "fantasies" about how such a tool should be and some maven things don't fit that. One of that is the project layout and plugins. I very much dislike the maven way of forcing people to make additional directories for plugins that are used in the build and additional directories to separate what the developer thought belongs together. I guess it is no problem to help if the developer keeps a certain layout, but he should be able to change that very easily.

See later for directory layout. Re your second point. For me it is clear: No plugins! This artificial separation makes no point for our approach. Often it makes the build less expressive and a paint to maintain. We don't need plugins, we offer good old tasks (targets) in the build script. Groovy offers all that is necessary if one wants to decouple build logic . We don't have to care about that.

My basic idea of a lifecycle that time was to have simple method calls:

def build() {
init()
compile()
test()
jar()
deploy()
}

where each method can be overwritten and add things to the lifecycle... Now I know that will not fit, since these lifecycly methods do need additional informations like descriptions and "task" dependencies. I guess one diea back then was to have a set of fixed top level methods for that like

def compile(){
meta {
description "call javac task from ant to compiles all java files"
compilation from:".java", to:".class"
}
action {
ant.javac(...)
}
}

well, something like that. meta and action would be special methods activated depending on the phase the build is. Other methods would be allowed too, marking normal dependencies for the task. Well that was my idea back then.. of course with some defaults for each method so that someone could write a class like that:

class CleanAlwaysBuild extends DefaultBuild {
def build() {
clean()
super.build()
}
}

anyway... it just got the impression that a complete rework of Gant is on the way, so I thought I mention this again.. just ignore it if you don't like it ;)

I think I would like to move away from the Maven concept of an external lifecycle object were you attach actions to. I'd rather have lifecycle taks that depend on each other. Those dependencies set up an implicit lifecycle. See my comments on lifecycle in my original mail.

I don't understand the concept of the meta and action closure. Is meta for configuration of the action?

some comments on your points:

[...]

2. We use a few exemplar projects as foci for discussing what the Gant
scripts should look like.

I like this idea very much. The respective projects should be complex enough. Anybody reading this, who is responsible for a build of some open source project and would like to join? A friend of mine knows the nutch search engine project very well including its build. He says they have a complex ant based build but does not think there is any interest from the nutch commiters in changing there build. But as a guinea pig, why not. We could at least have a look at it.

My experience with build systems is, that people are interested in it as long as they need one, and then very fast try to forget about it ;)

3. All ideas from SCons, Waf, Rake, Rant, CMake, Maven, Ant, Buildr
that are useful should be considered worth "stealing".

This is what open source is about, isn't it ;)

but maybe you guys should point out some things that are good. In any case you should point out what Gant can do and what not.. I mean there must be a reason why 0.4 should be so different... or should it be not so different and I misunderstood something?

So far Gant is no DSL for building Java projects. There is no terminology for source folder, test folder, lifecycle, packaging, .... We want to introduce this. I think the 'why' and the advantages of such a build DSL are pretty obvious. So Gant 0.4 will hopefully offer many more things than Gant 0.3. This build DSL is the most prominent goal for 0.4. But there is more that can be done. What Rake offers is very appealing to me. Rake offers no concrete tasks like compile. It is completely agnostic whether you use if for Java, C or whatever. It is a base technology for any build or build system. See for example http://www.martinfowler.com/articles/rake.html

I guess Russel can make a lot of points here :)

Some more of my thoughts on the issue:
*Project Layout:*
I like the project layout used by Maven2. It is defined here: http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
One thing that we might add to this, is an additional folder for integration tests (This is also supposed to be added with Maven 2.1).

yes, that is exactly what I have a problem with... why a java sub directory? There is more than just java. why a filters directory? I enver really figured out what that is useful for. having src/main and src/test is ok... even though I would not name that directory main... anyway.

I was pretty slack to refer to this link. What I mean with using the Maven project layout is assuming the following folder structure:

src/main/java

src/main/resources

src/test/java

src/test/resources

src/integtest/java

src/integtest/resources

src/webapps

That is it. We said that we provide a build DSL for Java projects. Actually I think we should provide a build DSL for Java/Groovy projects. What structure should we offer?

Just try to make it not too complicated to have mixed content and a different directory structure, then all should be fine.

Not too complicated is not good enough. Such simple things should be done simple, otherwise we have failed :)

[...]

*Dependency handling:*

use Ivy configured from the build script (no or optional XML)

3.) Transitive dependencies

is a must I would say.

Why?

a.) Even if we support transitive dependencies* we definitely should offer a mechanism for disabling them.

why?

[...]
> If we don't support transitive dependencies, just for

getting/putting dependencies to/from the repository, it might be simple enough not to rely on an external tool for this.

I think Ivy is not very big.

The transitive dependency issue deserves a wiki page on its own. But I think the importance of the issue should not be overestimated.

well... it will certainly work without that feature too, you will then have to define all jars instead of the jars you use directly.

Regarding the whole transitive dependency issue. For me there is only one must. The build script must be able to make it clear what are first level dependencies and what are not. If I wanted to add hibernate to my dependencies in Maven1, I had to declare a list like the following:

hibernate...

ehcache

cglib

asm

....

other non hibernate stuff

If someone decided not to use hibernate any longer, he or she had a good time figuring out which dependencies to remove from the list. How do you know, that asm is not a runtime dependency of another non hibernate dependency. To maintain the dependencies with Maven1 was a nightmare. You had to add a lot of comments. To have more expressiveness in regard to this, was for me initially one of the main advantages of the Maven2 dependency handling. I still think it is better than Maven1, but ...

Finally, some goodies from the Maven dependency handling:

1.) Let's say you want to use dom4j. You want to use the xpath functionality of dom4j. Dom4j needs Jaxen for supporting this. As not everybody uses the xpath functionality, Jaxen is an optional dependency of dom4j.

See: http://repo1.maven.org/maven2/dom4j/dom4j/1.6.1/dom4j-1.6.1.pom

What do you have to do in the pom.xml to get it into your classpath?

...

</dependency>

<groupId>jaxen</groupId>

<artifactId>jaxen</artifactId>

</dependency>

You have to declare Jaxen as a first level dependency and the build expresses no longer, that Jaxen is used in this project only as a dependency of dom4j. Yak!!!! Welcome back in the jar hell!

2.) Let's say you want to use dom4j and some other project. Let's say you want to use an XPP reader with dom4j. As this is an optional dependency, you have to do the same as above. Now imagine the other project has a non optional dependency to xpp3_min. Either you end up with xpp3 and xpp3_min (a subset of xpp3) in your classpath. Or you have to do say:

...

</dependency>

</dependency>

<artifactId>other project</artifactId>

</exclusion>

</exclusions>

</dependency>

The latter says on first sight. The other project does not need xpp3. Which is not the case. How to understand the dependencies in a larger build if you have those kind of mechanisms?

This is a common problem. For example cglib and cglib_full, spring and spring-context, spring-....

I feel very strong about not having anything like this in our build system. For me the Maven dependency mechanism is a no go! Ivy might be an alternative. But I don't know it good enough, to be confident not to run into similar issues. So at least we should be able to disable any transitive dependency handling.!

Now what about this one:

MY_CACHE = "org.cache:mycache:jar:1.0"

HIBERNATE = ["org.hibernate:hibernate:jar:3.2.0.ga", MY_CACHE, ....]

OTHER = ["...", MY_CACHE, ....]

...

compile.with HIBERNATE

This is so simple and expressive.

(stolen from buildr)

4.) Snapshots and scope
Maven introduces the concept of snapshots and scopes of dependencies. I would leave them out for the first iteration.

I think the idea of having scopes is a good one.. but I am sure it is not needed at first. Of course I wouldn't do that completely like maven. I do not like the idea of having a fixed lifecycle and using the names and scopes form such a fixed cycle doesn't fit a lifecycle that is not fixed. For example the idea I described above a scope could be defined by the name of the method we are in or a list consisting of all lifecycle method names... if a list, then you could setup a kind of inheritance between scopes and if you need only one scope you define it for the top level method, "build" then.

[...]

Why not:

JMOCK = group("jmock", "jmock-cglib", :under=>"jmock", :version=>"1.1.0")

JUNIT = "junit:junit:jar:3.8.1"

TEST_DEPS = [JMOCK, JUNIT]

...

compile.with COMPILE_DEPS
test.with TEST_DEPS

(stolen from buildr ;) )

*C**onclusions*
My idea would be to implement a build system with a Maven2 project layout (including multiple subprojects depending on each other) and a a lifecycle task handling based on Gant tasks. I think having a good DSL for those issues would be the most important thing. On top of that a simple dependency handling for the first release might do. That would be good enough to convince many people to migrate from Ant and Maven. Another question is whether we should start to implement a Groovy Rake as a base for our build system.

why depending so much on one tool on one hand and then try to unify to totally different tools on the other?

Because Ant and Rake have no intersection. Except that if Rake were available for Java, you could have based Ant on Rake to do things more elegantly. We might want to offer this Rake elegance to our users for there custom tasks. And use it internally for some of our stuff.

I really recommend having a look at buildr.

- Hans

« Return to Thread: Gant 0.4 - Creating a powerful DSL for building Java projects