[ANN] ocamlbuild-ctools and symbiosis meta build engine

[Mikkel Fahnøe Jørgensen]

I just dusted off some code used for internal builds and made it public.


---- cppinclude ----

http://git.dvide.com/pub/cppinclude/

A very simple but fast dependency scanner needed by ocamlbuild-ctools.
Just compile the cppinclude.c file to cppi binary and put it in the
executable path.


---- ocamlbuild-ctools ----

http://git.dvide.com/pub/ocamlbuild-ctools
http://git.dvide.com/pub/ocamlbuild-ctools/tree/README


An ocamlbuild plugin that supports compiling C and C++ cross platform
using ocamlbuild only.

Variant builds - can build many different variants from the same
source tree. For example static libraries, test builds, debug,
optimization, linking or not linking with some other code, different
platform builds etc.

Automic recursive header file generation - needed for pulling source
into _build directory.
It used to be able to rebuild on header file change, but it has
regressed - it is possible that .stamp files do not hold signatures of
dependencies of dependencies or something.

It has not been tested on Windows, but build rules have been copied
from old build scripts so it should not be that hard to get going.


---- Symbiosis ----

http://git.dvide.com/pub/symbiosis
http://git.dvide.com/pub/symbiosis/tree/README

This is a meta build system.
It is implemented as an ocamlbuild plugin.

It checks out source code from repositories and starts builds of
individual components.

Because ocamlbuild doesn't support linking additional files, a lot of
different code is collected in a single file. This includes a JSON
parser for configuration files.

Symbiosis tries to avoid the problem of collecting all source
dependencies in one large tree, or alternatively installing
dependencies on the system. The former makes it difficult to change
dependent component versions. The latter makes it difficult to quickly
test different build version complicates developer setup.

Correctly configured, just call ocamlbuild with a target for the top
most component to build. All the source should be appear in the
working directory and build.
Initially it can be complex to set up, but once done, it is rather
easy to add new components with interesting dependencies.

The README explains it, but it really lacks examples since it was
extracted from my own build infrastructure.

Each component is viewed as a source tree with actions that can depend
on other actions in other components. Typically checkout and build
actions.
Actions are implemented through agents - for example to execute make,
ocamlbuild or check out source code.
Actions receive parameters and return result arguments. This makes it
possible to find tool locations, libraries etc.

Future:

Currently agents need to be added to the ocamlbuild plugin. This is
easy - for example adding a new scm agent. But it would be nice to add
agents as external scripts. Symbiosis is designed to support this, but
it has not been implemented.

I have looked into integration with continuos build systems, but it
appears to be easier to integrate this into symbiosis also, since it
already does some of the work such a build system requires - but is
definitely is also missing some parts.


Mikkel

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[Mikkel Fahnøe Jørgensen]

2008/9/27 Mikkel Fahnøe Jørgensen <mikkel@...>:

Regarding Symbiosis:

> http://git.dvide.com/pub/symbiosis
> http://git.dvide.com/pub/symbiosis/tree/README

Some clarification:

Symbiosis is a standalone tool that, once compiled, does not require
ocaml nor ocamlbuild to be installed at all. (Because ocambuild and
the symbiosis plugin are compiled together to a new executable).

Thus Symbiosis can be distributed as a binary package for driving all
kinds of high-level builds. This means developers can download a
prebuilt Symbiosis executable and a standard configuration file to
their local machine from a project file server and basically have a
build going by typing:

  symbiosis "myproject"

This assumes available components are listed in proxy files in a
dedicated repository that the config files can identify.


I hope to put up an example project eventually, but at least you have
the source code now.


Relation to git submodule:

Lars Hjemli who has been working on git submodules pointed out some
similarity. This is correct. To some extend symbiosis tries to solve
the submodule problem. At the time didn't know git submodules (but
sort of knew about svn externals). But I do not want a system that is
tied into a specific source control tool, and nothing prevents
symbiosis from compiling projects that use git submodules or
equivalent where this is the preferred solution for some component
integration.

Git submodule does not, for good reasons, in itself handle the problem
of how to compile a submodule. Symbiosis does this.

Symbiosis does not (yet) make it easy to snapshot a specific revision
without some manual work (updating checkout revision id in proxies).
Git submodules does.

Eventually I hope to generate a build report where the exact checkout
revision of each component is listed along with the compile flags that
went in - symbiosis should then be able to rebuild a project given
such a report. So if such build reports are checked into source
control, it should be easier to rebuild past versions of projects.

Mikkel

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[David Teller-3]

On Sun, 2008-09-28 at 18:39 +0200, Mikkel Fahnøe Jørgensen wrote:

> 2008/9/27 Mikkel Fahnøe Jørgensen <mikkel@...>:
> This means developers can download a
> prebuilt Symbiosis executable and a standard configuration file to
> their local machine from a project file server and basically have a
> build going by typing:
>
>   symbiosis "myproject"
>
> This assumes available components are listed in proxy files in a
> dedicated repository that the config files can identify.
>
>
> I hope to put up an example project eventually, but at least you have
> the source code now.

Symbiosis looks quite interesting. I'm really looking forward to seeing
an example project!

Cheers,
 David

--
David Teller-Rajchenbach
 Security of Distributed Systems
  http://www.univ-orleans.fr/lifo/Members/David.Teller
 Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[Mikkel Fahnøe Jørgensen]

2008/9/28 David Teller <David.Teller@...>:

>> I hope to put up an example project eventually, but at least you have
>> the source code now.
>
> Symbiosis looks quite interesting. I'm really looking forward to seeing
> an example project!

Batteries also looks interesting - it seems we are attacking similar
problems from different angles :-)

There is a tutorial online (and in the source) now and two examples in
the source tree.

http://dvide.com/labs/symbiosis/tutorial/
http://git.dvide.com/pub/symbiosis/tree/examples

The examples are created during the tutorial.

The tutorial also creates a more complex setup with source control
which is not possible include in the example tree due to the
configuration involved.

symbiosis v0.1.1 adds the tutorial and basic git and wget supported -
both used in the tutorial.

The three tools now have a home page:

http://dvide.com/labs/symbiosis/
http://dvide.com/labs/ocamlbuild-ctools/
http://dvide.com/labs/cppinclude/

or just

http://dvide.com/labs/

Regards,
Mikkel

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs