cram-codecs: rANS 4x8 order-1 frequency table example does not follow description

[zaeleus]

This is in regard to CRAM codec specification (version 3.1) (2023-03-15).

The example in § 2.1 "Frequency table: Order-1 encoding" splits the following input

abracadabraabracadabraabracadabraabracadabr

into

abracadabra abracadabra abracadabra abracadabr

It's not obvious why the example is split this way. The note directs you to § 2.2.2 "rANS entropy encoding: Interleaving", which says

We therefore split the input data into 4 approximately equal sized fragments⁵ starting at 0, ⌊len/4⌋, ⌊len/4⌋ × 2 and ⌊len/4⌋ × 3.

I.e.,

abracadabr aabracadab raabracada braabracad abr

This method invalidates the observed frequencies for the example and footnote 5.

⁵This was why the \0 → a context in the example above had a frequency of 4 instead of 1.