Generate root-relative URLs from Sphinx

[smashwilson]

Sphinx assumes that all URLs end with a trailing slash and generates relative URLs accordingly:

<a class="reference internal" href="cloud-interfaces/api/cloudimages-api/#cloudimages-api"><span>Cloud Images and SDKs and APIs</span></a>

These work when you visit the page with a trailing slash, but chop the final path element when you visit it without a trailing slash. Normalize URLs to be less sensitive to this.

Remaining Work

• Sphinx preparer: Generate content ID references instead of relative URLs in the Sphinx preparer. ([waiting] Generate content ID references instead of relative URLs. preparer-sphinx#27)
• presenter: Search and replace content ID references in envelope.body.
• presenter: Handle content IDs in envelope.next and envelope.previous elements.
• Jekyll preparer: Generate content ID references from all helpers that generate URLs.
• Jekyll preparer: Generate content IDs instead of relative URLs in next and previous elements.
• Test URL generation within the userguides to be sure that I didn't mess up anything else.
• Update deconst documentation to include the updated envelope schema with a contentID element.

[smashwilson]

This is harder than it looks, and related to #38.

The basic issue is that the preparers can't know what prefix the content they're preparing is rendered on at prepare-time. In fact, a control repo change after the content is built could change their prefix long after the content is prepared and submitted! In order to do this properly, we'll basically need to manipulate generated URLs at presenter-time, instead.

I can see three options:

1. Preparers submit content as plain HTML. Presenter uses something like Cheerio to discover all links and normalizes them intelligently based on the content's current rendered prefix. Pro: Will work uniformly for all preparers and markup, even if someone drops links in a .. raw:: html. Con: More expensive post-processing in the preparer would hurt performance.
2. As preparers build content, generated relative link hrefs are prefixed with a known token ([[PREFIX]] or something). Presenter does a global search-and-replace in body text and injects the content's mount prefix at presentation time. Pro: Efficient: it only adds a raw search and replace call to the presentation pipeline. Con: Will require deeper hooks within the preparers to manipulate page rendering. Hacky and ugly (although nobody will see it). Anyone who happens to try to include that exact token in the page somewhere by hand will have a bad time without some special handling.
3. Similar to "preparer" - Render .rst to json #2, but tie into the Nunjucks template render process instead: make the preparers emit Nunjucks directives that emit something like {{ prefix }} at the beginning of each link href. Pro: Opens up more powerful presenter integration from content in the future. Ties in to the existing templating infrastructure in a natural way. Con: Turns metadata.body into a dynamic template that requires rendering, which makes template execution more costly. Requires some weird-feeling "directives that emit directives" in the preparers, and requires deeper and more error-prone preparer hooks.

Personally, I'm leaning toward option 3 as the least hacky of the three.

[ktbartholomew]

@smashwilson would it be possible to have the preparer replace links with the Content ID of the item being linked to?

<a href="{contentId: https://github.com/rackerlabs/docs-quickstart/...}">Link Text</a>

The presenter could then parse the content and, using the control repo's mapping tables, replace content IDs with root-relative URLs based on where on the site that content should live.

[smashwilson]

Hmm! I like that, especially because it leads into the ability to create dynamic cross-references across content repositories.

Do you think it should be a nunjucks helper call, like:

<a href="{{ to('https://github.com/rackerlabs/docs-quickstart/...') }}">

Or something that we parse and replace manually ourselves? (Basically options 2 or 3 from above.)

[ktbartholomew]

I think we would have to parse/replace before passing it on to Nunjucks. Nunjucks just sees the content as a big string, and won't (and probably shouldn't) do nested parsing.

[smashwilson]

... and won't (and probably shouldn't) do nested parsing.

Yeah, this is what I keep going back and forth on.

On one hand, it would be expensive and feel odd to parse metadata.body as a template. On the other hand, it feels kludgey to do a bunch of magic regexp search and replace stuff.

How about this: I use a nunjucks-ish syntax for it, but implement it with a regexp? That leaves us open to expansion to full template parsing in the Distant Future ™️, keeps it light and quick for now, and gives us a reasonable-looking syntax for the intermediate documents.