Eclipse Build Systems in Perspective

It is easy to get confused over the "build system options" available when developing with Eclipse - there is JDT and ANT, PDE (interactive), and PDE/Build (headless) - to start with the "classics". Then there is Athena, an elaboration on the also classic 'releng' system used to build Eclipse itself, then the newer Buckminster. The two latest additions to the family of build related projects at Eclipse are Eclipse b3 (2009), and Tycho (proposal 2010).

With this blog post I want to put eclipse build technology in perspective.

So, what does all these technologies do?

The classics

JDT - builds java source, interactively, under control of the project and workspace preferences.

PDE - builds OSGi components and the Eclipse specific extensions (features, products, etc.) under the control of PDE preferences. PDE consists of both the interactive parts (incremental builds, export commands, dialogs etc.), and the headless logic that performs the actual work. This logic is also made available as ANT tasks.

PDE/Build - generates ANT scripts that are then executed in headless fashion to build. The generated ANT scripts makes use of the same ANT tasks used by the interactive PDE.

If we stop there for a moment - this was the level of support for building provided by Eclipse a few years back. Although PDE does a very good job of building things interactively, it is fair to say that the headless PDE/Build has been the source of much pain and frustration. To complete the picture; at this time there was also the "releng" system in use at Eclipse, which only ran on the Eclipse servers.

Improving on PDE

While PDE/Build has more or less remained unchanged there has been many different approaches to how to provide good support for headless builds. Some took the scripting route - improving on releng base builder (Athena), some wrote their own scripts, other started modeling PDE like this project in PDE incubator, and some wrote better/easier to use script generators like pluginbuilder.

Eclipse Buckminster

When the Eclipse Buckminster project was introduced in 2005 it focused on additional things — when using generated scripts and technologies external to the IDE there are often issues caused by differences in how things are built i.e. it works just fine in the IDE but breaks in the build. An important goal for Buckminster was (and still is) to provide exactly the same build interactively as on the servers. This means that Buckminster has a tight integration with the builders running in a workspace made available in an efficient packaging for headless execution. Another important design decision was to use the existing information (i.e. the meta data used by PDE in the IDE) without generation of scripts and without roundtrip engineering, instead using advice/decoration to modify discovered metadata when the original information is not enough.

In addition to the important philosophical difference, Buckminster also provides unique support for materialization of a workspace, automatic provisioning of a target platform, running JUnit tests, EclEmma code coverage, Hudson integration, and much more.

Note that when Buckminster builds PDE related material (Buckminster can build other things as well), it calls on the same PDE logic that is used when building with interactive PDE.

Tycho

Tycho is a set of Maven plugins that provide building of OSGi and Eclipse related components (features, plugins, RCP products, etc.) and is an alternative to PDE suitable for those that have a Maven centric setup. Tycho does not use the original PDE logic.

Eclipse b3

The b3 project is about making it easy to work with build systems — discovering and modernizing or integrating existing build systems should be just as easy as building meta data rich components interactively or in continuos integration fashion. Eclipse b3 starts at the very other end of the spectrum than "which compiler to use" or which meta data dialect is used to describe components.

Eclipse b3 does this by providing EMF based models for build (i.e. components, their relationships, versions, types, etc.), expressions (i.e. the processing in form of tasks, builders, compilers, etc.), and p2 (with support for aggregation, re-categorization, mirroring, maven meta data publishing, and more), as well as a concrete syntax in form of a DSL implemented with Xtext that provides a rich text editing environment, and an evaluator that makes it possible to run b3 scripts.

As an example - there is nothing in Eclipse b3 that restrict it to using the original PDE logic to build the PDE related artifacts, it can just as well make use of Tycho's alternate way of building the same things.

The very first way Eclipse b3 will be building things is to use Buckminster as the execution engine. A small, and easy to understand b3 script will drive the entire build — combining the ease of use in the b3 DSL with the proven stable builds provided by Buckminster.

Eclipse b3 - a success at Eclipsecon

As you may have seen, the Eclipse b3 project is about creating a new generation of Eclipse technology to simplify software build and assembly. There has been lots of activity since the project was created, and there was much interest in b3 at Eclipsecon - so, here is a status update.

In case you did not know; a seminar on b3 was held at Eclipse Summit Europe 09 where the initial ideas were presented and discussed. A lot of new ideas about how people want to work with builds were generated, and there was lots of positive feedback on the original ideas. As always, some darlings were also completely killed in the process (the message that XPath queries are anything but easy to understand was received).

The feedback told us that these things are important:

• Ease of use
• Based on Modeling
• Debuggable
• Flexible / Extensible

Armed with all that input the time between ESE and Eclipsecon 2010 was devoted to developing a version of b3 that demonstrates the ideas, with a focus on Ease of Use, while not sacrificing flexibility or capability of dealing with real world complexities when building. At Eclipsecon we reached the first milestone of b3 consisting of:

• A build ecore model
• A process/expression ecore model
• A p2 ecore model with aggregation and rewrite support
• A concrete syntax implemented with XText (i.e. a feature rich eclipse editor and much more).
• An evaluator (i.e. making it possible to run the build scripts).
• Documentation of the concrete syntax
• A website with links to all b3 related information (documentation, for developers, etc).

The b3 interest at Eclipsecon 2010 was huge - the room was packed, not everyone could get in, and of those that did, close to 80% liked the presentation (i.e. voted +1). To the two individuals who voted -1; I am hoping you put some comments on your votes so I know what you did not like (it simply has to be 'lack of chairs' :)).

For the next milestone (around Helios release) we are adding concrete things to b3:

• Use a b3 script to drive a buckminster build
• Publish b3 build units to a p2 repository - i.e. author installable units

As always - love to hear your questions and comments. I have already received quite a few questions regarding the relationship between b3 and other eclipse related build technologies (buckminster, athena, PDE build, and now the Tycho proposal), and that will be the topic of my next blog post about b3.