feat(mbtiles)!: split the cache schema into flat and normalized layouts

Mirror the flat vs normalized split of the regular schemas:

- `CacheSchema::Flat`: a single tile_cache table with the blob inline -
  simple and fast when few tiles share content.
- `CacheSchema::Normalized` (previously the only layout): tile_cache
  pointing into the xxh3-64-keyed, de-duplicated cache_data blob table -
  the recommended default for web caches.

Both keep the tile_cache name for the coordinate/metadata table
(detection tells them apart by the sixth column: tile_data vs tile_id)
and both expose the spec-compatible tiles view.

- `MbtType::Cache { schema: CacheSchema }`, mirroring Normalized.
- CLI values `cache-flat` and `cache-normalized`; `cache` stays as an
  alias for cache-normalized.
- Runtime API takes the schema like insert_tiles takes mbt_type;
  `MbtilesCache` detects and stores the layout (`open` creates
  normalized; `open_with_schema` chooses).
- Copies preserve expires/etag in any cache-to-cache direction,
  including across layouts, via a shared canonical select; only
  normalized needs the blob-insert + collision-check path.
- Flat purges skip the orphan-blob GC; per-tile validation of flat
  caches has nothing to hash-check, like flat.
- Cache detection and reads use runtime SQL (the two layouts' tile_cache
  columns cannot coexist in the sqlx-prepare database), dropping two
  .sqlx entries.
- Tests parametrized over both layouts (unit, pool, copy round-trips,
  2x2 cache-to-cache matrix, diff inputs); CLI golden tests cover both;
  normalized golden outputs are byte-identical to before.

BREAKING CHANGE: `MbtType::Cache` gains a payload, `MbtTypeCli::Cache`
is replaced by `CacheFlat`/`CacheNormalized`, and the cache read/write
APIs take a `CacheSchema` parameter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: nyurik

docs/content/files/init-cache-flat.sql

@@ -0,0 +1 @@
+../../../mbtiles/sql/init-cache-flat.sql

docs/content/files/init-cache-normalized.sql

@@ -0,0 +1 @@
+../../../mbtiles/sql/init-cache-normalized.sql

docs/content/files/init-cache.sql

@@ -68,25 +68,38 @@ In our next semver major, we plan to switch this default and produce `tiles_shal
 
 ## cache
 
-The `cache` schema is similar to `normalized`, but stores extra cache metadata (`expires` and `etag`) alongside each tile.
+The `cache` schemas store extra cache metadata (`expires` and `etag`) alongside each tile, so a file can serve as a persistent web-tile cache.
+Two layouts exist, mirroring the `flat` vs `normalized` split of the regular schemas.
+Both center on a `tile_cache` table holding the tile Z,X,Y coordinates and the cache metadata, and both create a spec-compatible `tiles` view so the file can still be read by any standard MBTiles reader (the `expires`/`etag` columns are simply invisible to it).
+
+### cache-flat
+
+The tile blob is stored inline in the `tile_cache` table.
+Simple and fast, best when few tiles share the same content.
+
+```sql
+--8<-- "files/init-cache-flat.sql"
+```
+
+### cache-normalized
+
 Tile blobs are de-duplicated in the `cache_data` table, keyed by an integer `tile_id` that is the [xxh3-64](https://github.com/Cyan4973/xxHash) hash of `tile_data` (stored as an `INTEGER PRIMARY KEY`, i.e. an alias for the rowid, so identical blobs collapse to a single row).
-The `tile_cache` table maps tile Z,X,Y coordinates (plus `expires`/`etag`) to a `tile_id`.
-A spec-compatible `tiles` view is also created, so the file can still be read by any standard MBTiles reader (the `expires`/`etag` columns are simply invisible to it).
+This is the recommended default for web-tile caches, where identical (e.g. empty or ocean) tiles are common; the `cache` CLI value is an alias for it.
 
 ```sql
---8<-- "files/init-cache.sql"
+--8<-- "files/init-cache-normalized.sql"
 ```
 
 ### Supported operations
 
-The `mbtiles` tool treats `cache` as a first-class schema, with a few deliberate restrictions:
+The `mbtiles` tool treats both cache layouts as first-class schemas, with a few deliberate restrictions:
 
 * `summary`, `validate`, `meta-*`, and serving the file with `martin` all work.
 * `copy` **from** a cache file to any schema works (reading via the `tiles` view); the per-tile `expires`/`etag` values are dropped, since standard schemas cannot store them.
-* `copy` **into** a cache file works from any schema (including `martin-cp --mbtiles-type cache`); the copied entries get `NULL` `expires`/`etag` (never expire). A cache-to-cache copy preserves `expires`/`etag` verbatim.
-* `diff`, `apply-patch`, and bin-diff **into or onto** a cache file are rejected: the `NOT NULL` blob table joined through the `tiles` view cannot represent the `NULL` "deleted tile" markers a diff needs. A cache file *can* be the compared-against or patch-source side (it is read through the view).
+* `copy` **into** a cache file works from any schema (including `martin-cp --mbtiles-type cache-flat|cache-normalized`); the copied entries get `NULL` `expires`/`etag` (never expire). Copies between cache files - including across the two layouts - preserve `expires`/`etag`.
+* `diff`, `apply-patch`, and bin-diff **into or onto** a cache file are rejected: the `NOT NULL` blob storage joined through the `tiles` view cannot represent the `NULL` "deleted tile" markers a diff needs. A cache file *can* be the compared-against or patch-source side (it is read through the view).
 * `cache-purge <file> [--max-size <MB>]` removes expired entries (and optionally evicts soonest-expiring entries until the file fits the size budget), then reclaims free pages via `PRAGMA incremental_vacuum`.
 
-Per-tile validation checks foreign-key integrity (every `tile_cache.tile_id` must exist in `cache_data`) and that each blob is stored under its xxh3-64 content key, allowing for the small linear-probing window the runtime API uses on hash collisions. Unreferenced `cache_data` rows are legal - they appear when an entry is overwritten and disappear on the next purge.
+For `cache-normalized`, per-tile validation checks foreign-key integrity (every `tile_cache.tile_id` must exist in `cache_data`) and that each blob is stored under its xxh3-64 content key, allowing for the small linear-probing window the runtime API uses on hash collisions. Unreferenced `cache_data` rows are legal - they appear when an entry is overwritten and disappear on the next purge. Like `flat`, the `cache-flat` layout has no hashes to check.
 
-Note that bulk SQL copies into a cache file cannot resolve xxh3-64 collisions the way the runtime `set_cached` API does (by linear probing); the copier instead verifies afterwards that no two different blobs mapped to the same key and fails the copy in that astronomically-unlikely case.
+Note that bulk SQL copies into a `cache-normalized` file cannot resolve xxh3-64 collisions the way the runtime `set_cached` API does (by linear probing); the copier instead verifies afterwards that no two different blobs mapped to the same key and fails the copy in that astronomically-unlikely case.

docs/content/mbtiles-schema.md

@@ -532,7 +532,9 @@ prepare-sqlite: fetch install-sqlx
     trap 'rm -f "$db"' EXIT
     # add every possible schema to a dummy temp file so that most queries would compile.
     # `init-flat` is listed before the view-defining layouts so `tiles` is a table.
-    for f in init-metadata init-flat init-flat-with-hash init-normalized init-normalized-dedup-id init-cache; do
+    # `init-cache-flat` is omitted: its `tile_cache` table conflicts with the normalized
+    # flavor's, and no compile-time query!() macros reference the cache tables.
+    for f in init-metadata init-flat init-flat-with-hash init-normalized init-normalized-dedup-id init-cache-normalized; do
         # it is safer to handle NULLs than to expect it to never be there
         sed -E -e 's/[[:space:]]+NOT[[:space:]]+NULL//g' \
                -e 's/CREATE (TABLE|VIEW) /CREATE \1 IF NOT EXISTS /' \

justfile

@@ -650,7 +650,12 @@ async fn init_schema(
                     hash_view: true,
                     schema: mbtiles::NormalizedSchema::Hash,
                 },
-                MbtTypeCli::Cache => MbtType::Cache,
+                MbtTypeCli::CacheFlat => MbtType::Cache {
+                    schema: mbtiles::CacheSchema::Flat,
+                },
+                MbtTypeCli::CacheNormalized => MbtType::Cache {
+                    schema: mbtiles::CacheSchema::Normalized,
+                },
             };
             init_mbtiles_schema(&mut *conn, mbt_type, false)
                 .await

martin/src/bin/martin-cp.rs

@@ -0,0 +1,17 @@
+CREATE TABLE tile_cache (
+    zoom_level INTEGER NOT NULL,
+    tile_column INTEGER NOT NULL,
+    tile_row INTEGER NOT NULL,
+    expires INTEGER,
+    etag TEXT,
+    tile_data BLOB NOT NULL,
+    PRIMARY KEY (zoom_level, tile_column, tile_row)
+);
+
+CREATE VIEW tiles AS
+SELECT
+    zoom_level,
+    tile_column,
+    tile_row,
+    tile_data
+FROM tile_cache;