Skip to content

Commit e2228f8

Browse files
paulfouquetpaulfouquet
authored
feat(tileindex-validate)!: automatically select scale of output tiles TDE-1674 (#1287)
### Motivation Input TIFFs should be automatically tiled using their GSD and imagery type to determe the appropriate output scale. It avoids data managers to know what the supplied tile size is, what should be the output tile size, and whether or not a "retiling" (output tile made of several input TIFFs) is needed. Validation of perfect alignment and duplicates does not make sense at this stage as the tiling will always occurs when `--scale=auto`. This validation feature has been removed and will be part of a visual QA feature next. ### Modifications - ⚠️ **BREAKING CHANGE** `--retile` and `--validate` flags has been removed - Add an `auto` mode to the `--scale` flag - Always validate inputs have consistent `GSD` <!-- TODO: Say what changes you made. --> <!-- TODO: Attach screenshots if you changed the UI. --> ### Verification Automated and manual tests <!-- TODO: Say how you tested your changes. -->
1 parent 73361b5 commit e2228f8

File tree

5 files changed

+151
-424
lines changed

5 files changed

+151
-424
lines changed

‎COMMANDS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
| [stac-setup](./src/commands/stac-setup/README.md) | Collection-related STAC metadata setup. Outputs collection-id and linz-slug files within /tmp/stac-setup/ |
1616
| [stac-sync](./src/commands/stac-sync/README.md) | Sync STAC files |
1717
| [stac-validate](./src/commands/stac-validate/README.md) | Validate STAC files |
18-
| [tileindex-validate](./src/commands/tileindex-validate/README.md) | List input files and validate there are no duplicates. |
18+
| [tileindex-validate](./src/commands/tileindex-validate/README.md) | Map input TIFF files to output tiles based on their location. Validate their alignment to the tile grid and output retiling information. |
1919
| [pretty-print](./src/commands/pretty-print/README.md) | Pretty print JSON files |
2020
| [generate-path](./src/commands/generate-path/README.md) | Generate target path from collection metadata |
2121
| [verify-restore](./src/commands/verify-restore/README.md) | Verify that all the files requested to be restored with S3 Batch Operations Restore are restored yet. Output a boolean value to indicate if all files are restored. |

‎README.md‎

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -258,27 +258,21 @@ stac validate --checksum-assets --checksum-links --recursive s3://linz-imagery-s
258258

259259
### `tileindex-validate`
260260

261-
Validate or create retiling information for a list of tiffs.
261+
Map input TIFF files to output tiles based on their location. Validate their alignment to the tile grid and output retiling information.
262262

263-
Outputs files for visualisation of the tiles and as an list for [topo-imagery](https://github.com/linz/topo-imagery/pkgs/container/topo-imagery) to use for retiling with GDAL.
263+
Outputs files for visualisation of the tiles and a list of output tiles with their input TIFF files for [topo-imagery](https://github.com/linz/topo-imagery/pkgs/container/topo-imagery) to use for creating the tiles with GDAL.
264264

265265
- `input.geojson` GeoJSON file containing the bounding boxes of the source files. Example: [input.geojson](docs/input.geojson)
266266
- `output.geojson` GeoJSON file containing the bounding boxes of the requested target files. Example: [output.geojson](docs/output.geojson)
267267
- `file-list.json` a list of source and target files to be used as an input for `topo-imagery`. Example: [file-list.json](docs/file-list.json)
268268

269-
`--validate`
270-
Validate list of tiffs match a LINZ map sheet tile index and assert that there will be no duplicates. Example:
271-
272-
```bash
273-
tileindex-validate --validate --scale 5000 s3://linz-imagery/auckland/auckland_2010-2012_0.5m/rgb/2193/
274-
```
269+
#### Example
275270

276-
`--retile`
277-
Output a list of tiles to be retiled to the scale specified, and which tilename they should receive when merged. Example:
271+
Output a list of tiles to be automatically tiled to an appropriate scale determined by the system, and which tile name they should receive when merged. Example:
278272

279-
```bash
280-
tileindex-validate --retile --scale 10000 s3://linz-imagery/auckland/auckland_2010-2012_0.5m/rgb/2193/
281-
```
273+
```bash
274+
tileindex-validate --scale=auto s3://linz-imagery/auckland/auckland_2010-2012_0.5m/rgb/2193/
275+
```
282276

283277
### `bm-create-pr`
284278

‎src/commands/tileindex-validate/README.md‎

Lines changed: 17 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,34 @@
11
# tileindex-validate
22

3-
List input files and validate there are no duplicates.
3+
Map input TIFF files to output tiles based on their location. Validate their alignment to the tile grid and output retiling information.
44

55
## Usage
66

77
tileindex-validate <options> [...location]
88

99
### Arguments
1010

11-
| Usage | Description | Options |
12-
| ------------- | ------------------------------------------------------------ | ------- |
13-
| [...location] | Location of the source files. Accepts multiple source paths. | |
11+
| Usage | Description | Options |
12+
| ------------- | ----------------------------------------------------------- | ------- |
13+
| [...location] | Location of the source files. Accepts multiple source paths | |
1414

1515
### Options
1616

17-
| Usage | Description | Options |
18-
| ---------------------- | ---------------------------------------------------- | ------------- |
19-
| --config <str> | Location of role configuration file | optional |
20-
| --include <str> | Include files eg ".\*.tiff?$" | optional |
21-
| --scale <value> | Tile grid scale to align output tile to | |
22-
| --source-epsg <number> | Force epsg code for input tiffs | optional |
23-
| --preset <str> | Validate the input tiffs with a configuration preset | default: none |
24-
| --concurrency <number> | Number of TIFF files to read concurrently | optional |
17+
| Usage | Description | Options |
18+
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
19+
| --config <str> | Location of role configuration file | optional |
20+
| --include <str> | Include files eg ".\*.tiff?$" | optional |
21+
| --scale <value> | Tile grid scale to align output tile to. If set to "auto", the system will determine the appropriate scale based on the imagery type (`--preset`) and its GSD | optional |
22+
| --source-epsg <number> | Force epsg code for input tiffs | optional |
23+
| --preset <str> | Validate the input tiffs with a configuration preset | default: none |
24+
| --concurrency <number> | Number of TIFF files to read concurrently | optional |
2525

2626
### Flags
2727

28-
| Usage | Description | Options |
29-
| ---------------- | ---------------------------------------------------------- | -------------- |
30-
| --verbose | Verbose logging | |
31-
| --retile | Output tile configuration for retiling | default: false |
32-
| --validate | Validate that all input tiffs perfectly align to tile grid | default: true |
33-
| --force-output | force output additional files | default: false |
34-
| --includeDerived | Include input tiles as STAC `derived_from` links | default: false |
28+
| Usage | Description | Options |
29+
| ---------------- | ------------------------------------------------ | -------------- |
30+
| --verbose | Verbose logging | |
31+
| --force-output | force output additional files | default: false |
32+
| --includeDerived | Include input tiles as STAC `derived_from` links | default: false |
3533

3634
<!-- This file has been autogenerated by src/readme/readme.generate.ts -->

0 commit comments

Comments
 (0)