Document: final encoding

[kubukoz]

Document has served us well, but we the maintainers think we can offer users a more efficient encoding of it: instead of storing the nodes as an ADT, in some situations (e.g. converting a datatype to be encoded to JSON later) it'd be more efficient to keep the datatype & its schema so that it can be written directly onto the wire / byte stream when it's time to actually encode. This would effectively mean all Document.Encoders based on schemas would have a O(1) runtime, at virtually no additional cost during the actual serialization.

In order to facilitate making such improvements transparently & without coupling Document's public API to serialization formats, we'll make Document an open datatype (unsealed trait) and rely on the visitor pattern for extracting values out of it (and the existing factory methods for creating documents from plain values).

I would suggest that we start by adding a visitor API and deprecating the public AST nodes, so that users can migrate existing code to the 0.19 version (which will be binary and source breaking) while we figure out the details.

[kubukoz]

The visitor should also somehow handle more numeric types - #1579 - from day 1, ideally.

[kubukoz]

For the record, I made a POC of fused A -> Document -> JSON encoding: kubukoz/demos@b47a70f...smithy4s-document-encoding (see commit-by-commit history)

Discussed this with @Baccata in DMs and we decided to put this part of the idea on hold, whereas we'll probably keep the Document interface sealed, but keep its internal nodes private. Instead of pattern matching, users will have projections and a visitor, with a default case so that we can extend the internal ADT in the future. Or revisit fused encoding as well (maybe with opt-in).

[kubukoz]

A new idea from the DMs: DocumentTag[A]. I will not elaborate further for now.