Best practices for generator development

This page is about development of the generator, not for building applications with the generated code.

The best practices shown here aim on illustrating several aspects with regard to structure, design and constraints. The following content is based on practical experience and building blocks. Further topics are the MDSD development process as well as organisational aspects.

Development process

An approach suggested often for MDSD projects is an iterative-incremental and especially agile development process. This process is divided into two threads:

• domain architecture development
• application development Regarding the first one we are splitting work on the tool chain into two parts again:
• one team from start of the toolchain: dsl design, meta models, validation constraints, tooling
• one team from end of the toolchain: core/platform development, generator improvements Bidirectional communication flows between both threads are essential
• upcoming dsl evolvements and lifecycles
• reference implementation snippets + required/desired language evolvements
• agreement about lifecycles and priorities/order of bigger amendments
• regular meetings

Development of Zikula platform for MDSD

Simple guidelines for the programming model:

• Always develop against interfaces, not against implementations
• Never create objects for yourself, but always use factories
• Use also factories for accessing resources (like database connections)
• State-less design is a good idea in enterprise systems
• Separate concerns: ensure that an artefact does one thing, not five

Release management

While one team works on architecture and generator, the other thread creates modules with it. New generator versions must be introduced to the application thread at well-defined points. For both products (module as well as generator) a release planning is required - and both must fit to each other. This coordination is the real challenge when doing big projects on application side, but there is also the chance for continuous feedback and improvement. Thus we aim on continuous deployment of the generator cartridges. When new bugs or feature requests come up, a quick response is needed so that application development thread can continue without having to wait. For example quick workarounds can be contributed as generator patches.

Versioning

What is being versioned?

• DSLs (meta models, constraints, concrete notations)
• Templates and transformations
• The MDSD platform / core
• The applications / modules (including models, specifications and hand-written code)

Projects and dependencies

Ideally all generated code is not versioned, as it would be only overhead. The goal is a structural separation of manual and generated code on physical leven (that is, the file system). Only the manually implemented code parts are checked into SVN. It is important to manage dependencies of those projects, including their versions. It is essential for an application project to define on ''which version'' of the domain architecture it is based. When the platform/core evolves, the domain architecture has to grow as well, and possibly even the application projects. Regarding the structure of a module project an application's SVN repository contains the models and the manually-written code. The generator creates the generated code, after that the application will be generated with the help of a build script (which will also be generated).

DSL evolution

Typically the DSL will mature iteratively. Knowledge and understanding of the domain increase so that the DSL can evolve. It is essential to ensure that the DSL keeps compatible to old versions. So the generator configuration should be changed to support different versions of DSL and meta model. This should especially be considered if module developers switch to a new version. We need migration pathes with as less model changes as possible. The new DSL features should be offered to the developers, not enforced. Old features can also become deprecated. The generator could output warnings if such features are used.

For example we can put a version number into the models which determines how the generator understands a model or which code it generates (see also this feature request). We can also implement implicite rules: if a certain model attribute does not exist, a default value is used..

If the DSLs are developed in this way, that old features are kept and new ones are added, their evolution is quite easy. Naturally this approach increases the generator complexity. Thus one should also ensure that old features get removed with time. A controlled usage of ''deprecated'' allows sorting out old features. The generator can create a file which logs all features still being used so that they can removed from generator officially then.

Generator development

People like beauty code

Generator code must follow the same coding guidelines and design pattern as usual. It should be understandable for the devs working with it. There are beautifier components, but as far as I know there does not exist one for PHP yet.

Managing generator complexity

To keep the transformation gap small and to reduce the generator's complexity we can often help at the platform (core). For example we can use the Doctrine framework instead of generating tons of own persistance code. So when applicable we should encapsulate complexities not in every application, but centrally. For example with three inheritance steps:

• the core provides an abstract base class
• the generator creates a subclass which is also abstract
• the concrete class inherits from the second one If the transition gap becomes too huge we can split up the whole transformation into two ones. A model-to-model transformation could help reducing complexity in the actual generation step.

Testing generator cartridges with reference models

All artifacts of the domain achitecture, like code generators, model validation and model transformations must be tested automatically, too. To ensure that the generated code behaves like required we can create unit tests for it. Therefore we have to create reference models using the DSLs concept as example. Ideally each element is named according to the aspect it is representing, in order to reach full coverage of all aspects and variabilities. Within later iterations we could also build new reference models especially for describing cases where bugs have appeared.

To test generation results we have to predefine what should be generated. For this purpose we should use one or two test models reflecting and covering all existing language concepts. Then we can use automatic diffs to ensure that the generated results are always equal to the predefined ones. This is the recommended way for testing generator cartridges.