tinshift: support TIN GeoPackage files

[rouault]

Such files are equivalent to already supported TIN JSON files, but scale better for arbitrarily large triangulations, in particular for network-based access.

The format is defined in source/specifications/tin_gpkg.rst and the https://github.com/rouault/PROJ-data/blob/tin_json_to_tin_gpkg/grid_tools/tin_json_to_tin_gpkg.py Python script may be used o convert an existing TIN JSON into a TIN GeoPackage.

$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_ykj_etrs35fin.gpkg
431943.0905 6992816.7826 0.0000 inf
$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_ykj_etrs35fin.json
431943.0905 6992816.7826 0.0000 inf
$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_n60_n2000.gpkg
3432087.0000 6995748.0000 0.3161 inf
$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_n60_n2000.json
3432087.0000 6995748.0000 0.3161 inf

Fixes #3732

[jjimenezshaw]

I didn't review the complete algorithm yet. But one usual optimization is to start searching on the last triangle successfully returned. If searched points an near, they will probably be in the same triangle. Is it something to include in the API somehow? Or just an implementation detail? (I want to think it is an implementation detail, but I am not sure. In that case we should store the last used triangle threadsafe somewhere)

[rouault]

But one usual optimization is to start searching on the last triangle successfully returned.

good idea. Implemented now

[jjimenezshaw]

Is this using curl or sqlite3 directly? Is it getting the whole file, or is it "cloud optimized"?

[rouault]

We use our NetworkFile implementation that uses curl underneath and range request, and plug it into sqlite3 I/O layer. That way we can have piece wise access to the parts of the file we need, and benefit from the local caching in our ~/.local/share/proj/cache.db of the chunks we have already read.

From my test on a synthetic grid of 500x500 nodes, with each square divided into 2 triangles, that generates a GeoPackage file of ~ 64 MB. The first access to extract a triangle involves ~ 20 network requests of 16 KB each. Accesses to other parts (not close to each other) of the file afterwards take between 6 and 14 requests.

It seems that FlatGeoBuf spatial index (packed Hilbert RTree) would involve less requests, but FlatGeoBuf supports only one layer per file, and we need 2 layers. The FlatGeoBuf files would also be larger than the GeoPackage one (presumably because both the triangle geometry and the RTree need to be stored, whereas with GeoPackage we only store the RTree of the bbox of the triangles, but not the geometry of the triangles themselves, just the indices to the vertices), although file size is probably less of a concern. We could possibly have a .tar of 2 flatgeobuf files? I'm not sure if that's worth the complication.