Alternative to mako templating language

[cevdetta]

Switching to another templating language might help us to tidy up the dependencies and make the build faster.
I propose to use a rust based solution like tera which is used by zola.
There are 51 files (.mako.rs, .mako) that are need to change.
We can drop python in the build process and fully focus on rust.

[nicoburns]

I think this is a good idea, but it's quite a big task. It's not only the .mako files. All of the Python files in style/properties would also need to be ported.

I actually started on this, so I've put my progress up as a draft PR (#241). I'm not actively working on it, so feel free to pick it up if you are interested.

I propose to use a rust based solution like tera

Tera is generally good, but we may wish to consider optimising for compile times and using something like https://lib.rs/crates/tinytemplate.

In general, the current codegen code puts a lot of logic into the templates. And I think if we're rewriting it we may wish to do so with a cleaner architecture with a clearer separation between:

• The data model (Rust structs)
• The generation logic
• String templating (which could hopefully be relatively simple)

[cevdetta]

I propose to use a rust based solution like tera

Tera is generally good, but we may wish to consider optimising for compile times and using something like https://lib.rs/crates/tinytemplate.

askama-rs/template-benchmark github

There are 2 choices we can make; compiled or interpreted:
For compiled: vy which is faster than maud
For interpreted: tinytemplate but it's not maintained for 3 years, is old and has serde_json as a dependency which is unnecessary; that's why we can choose tera if we won't choose compiled version(s).

In general, the current codegen code puts a lot of logic into the templates. And I think if we're rewriting it we may wish to do so with a cleaner architecture with a clearer separation between:

* The data model (Rust structs)

* The generation logic

* String templating (which could hopefully be relatively simple)

Agreed.

[nicoburns]

There are 2 choices we can make; compiled or interpreted

I think interpreted probably makes most sense in this. The templates are small, so compile times are likely to dominate runtime costs. In any case, I don't think either vy or maud make sense becuase they are both HTML-centric, but we're not generating HTML.

Tera may be a good choice, but I would note that it also depends on serde_json and overall has a lot more dependencies than tinytemplate. minijinja also looks like it might be a decent option.

[cevdetta]

There are 2 choices we can make; compiled or interpreted

I think interpreted probably makes most sense in this. The templates are small, so compile times are likely to dominate runtime costs. In any case, I don't think either vy or maud make sense becuase they are both HTML-centric, but we're not generating HTML.

Tera may be a good choice, but I would note that it also depends on serde_json and overall has a lot more dependencies than tinytemplate. minijinja also looks like it might be a decent option.

It's true, and also tinytemplate isn't updated as I've stated and it'd be good if we could've updated (edition: 2024, dependencies and all), remove serde_json as dependency, then it'd be a solid choice in every aspect. Dependency wise indeed tera seem to have more, also minininja is fine and serde is the only dependency and serde_json is for devdependency, so might figure out what we could do?
Also I'm not expert in Rust but I'm fine with templating languages (had familarity with tera because of zola).

[Loirooriol]

This needs to be coordinated with upstream Firefox. @emilio

[emilio]

Fwiw it'd be great to avoid introducing new dependencies for this.

Also, is mako really slow? When I measured this in the past mako takes just milliseconds (specially after we optimized it to do a single pass), and most of the build time is just compiling the generated rust code.

Some unsorted thoughts since I'm on PTO this week:

Whatever system we were to choose needs to be able to run independently of the style crate build, at least to generate lists of properties / structs and so on. That means we probably want to split property metadata apart from code generation (both are done using mako right now). It'd be great for that format to be readable from python easily (because the Firefox build system generally uses python), but it's technically feasible to work around it.

In the past I (half-jokingly) suggested using css as the language for defining the property metadata. We don't need anything fancy, mostly key value pairs I think? Some of the properties are generated in loops right now (see border radius), but I wouldn't be too sad if we unrolled those. I think some common metadata format (toml? json? yaml? or css, hah) probably should be how we define the css property metadata.

For codegen on top to support shorthands and generate the various types we need, we could do whatever, but I'd rather use something dumb / simple, maybe just use some manual rust code (or using the quote crate), which feels like it'd be enough for our needs.

In general I'd like to limit the generated rust code to the minimum long term, since anything we do is going to interact really poorly with LSPs and most other stuff... Since we already use quote for all the custom macros, it seems it'd be nice to reuse it.

But anyways, killing mako long term seems fine, but I'd approach it incrementally:

• it seems a good first step here would be to move the property metadata outside of mako, into some format tbd. Then we just use json.loads() / or whatever to feed it into mako.
• Then we can migrate anything that currently relies on the mako output outside of the style crate build (I think the property docs in servo if those still exist, and the various Firefox build scripts that generate headers) to use the metadata rather than mako.
• Then we replace mako with whatever we see fit for the rust codegen.

Thoughts? I'm open to alternative plans of course, but just switching templating systems for the sake of it seems weird, specially if we have no evidence of mako being slow?

[mrobinson]

This sounds like a great plan to me and I like the incremental approach.

[nicoburns]

Also, is mako really slow? When I measured this in the past mako takes just milliseconds (specially after we optimized it to do a single pass), and most of the build time is just compiling the generated rust code.

Agreed. Runtime of the templating engine is a non-issue, and I suspect it would be very hard to beat it with a Rust engine (given that it would need to be compiled before it can be run).

---

For me the motivation for doing this is two-fold:

• It would be nice to eliminate Python as a build dependency of Stylo. For Blitz as a Rust crate, the expectation is that a Rust toolchain (and perhaps the C toolchain) is all you need and cargo build will just work. We don't have that experience right now (specifically on Windows where Python doesn't come preinstalled).

  (this could also be fulfilled by running the codegen "offline" and committing the generated code)

• I feel like the Stylo codegen could do with some cleanup / refactoring / documentation to do things like "split property metadata apart from code generation", document the available configuration options: what they do, valid syntax, etc. That would enable us to modify that part of the codebase with more confidence, and all the usual benefits of stricter typing and more documentation.

I feel like Rust structs might a good way to document the configuration format. They come with built-in type-safety, a documentation generation tool, and are generally easy to serialize / deserialize into formats like json, toml that can interop with other tools. And it would make the toolchain issue go away.

The properties could either be defined in the json/toml, or defined in Rust and output json/toml (that would allow us to keep the loops), wouldn't really matter. In my prototyping of this I made a new stylo_codegen crate with the idea being that it would run in build.rs of the style crate and could also be run standalone.

---

In general I'd like to limit the generated rust code to the minimum long term, since anything we do is going to interact really poorly with LSPs and most other stuff.

I've actually found that the generated code is fine with LSP. It can just right into it (in fact, this is by far the easiest way to find/read the generated code). It's annoying in other circumstances though, and some of the generated types are weird (there are a lot of types just called T (though many also have aliases). And LSPs don't like mako files very much (them being a weird mix of syntaxes). I hope we might be able to improve on some of that if we can demystify this code.

---

just use some manual rust code (or using the quote crate)

The quote crate is a good shout. I hadn't considered it, but it makes a lot of sense for this use case.