Skip to content

Conversation

@stephenkilbourn
Copy link
Contributor

@stephenkilbourn stephenkilbourn commented Sep 30, 2025

this is one of the tasks for issue #252 "Documentation to support external collection indexing"

Description

This adds documentation for indexing external data using titler-cmr. It provides the GPM_3IMERGDF example json config as an example

It also adds the instructions for ArcGIS

@github-actions
Copy link

github-actions bot commented Sep 30, 2025

PR Preview Action v1.6.2

🚀 View preview at
https://NASA-IMPACT.github.io/veda-docs/pr-preview/pr-255/

Built to branch gh-pages at 2025-10-10 16:13 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@anayeaye
Copy link
Collaborator

anayeaye commented Oct 7, 2025

@stephenkilbourn sorry for the delay here--the content is looking good but I am having trouble navigating to it in the preview (this may be a result of the project structure more than the PR--there seems to be a lot of material that requires digging to find).

@stephenkilbourn
Copy link
Contributor Author

@stephenkilbourn sorry for the delay here--the content is looking good but I am having trouble navigating to it in the preview (this may be a result of the project structure more than the PR--there seems to be a lot of material that requires digging to find).

@anayeaye - this should be working in the preview now. Everything is starting from the "External Collection Indexing" page

https://docs.openveda.cloud/pr-preview/pr-255/user-guide/content-curation/external-collection-indexing/

@stephenkilbourn stephenkilbourn changed the title add Titiler-CMR to external collection indexing docs add Titiler-CMR and ArcGIS to external collection indexing docs Oct 10, 2025
@sandrahoang686
Copy link
Contributor

Do we need to include the publishing process as well in the end with either the ingest UI/keycloak or opening a pull request to veda-data? cc @anayeaye @stephenkilbourn

@anayeaye
Copy link
Collaborator

@sandrahoang686

Do we need to include the publishing process as well in the end with either the ingest UI/keycloak or opening a pull request to veda-data?

I think we probably should just focus on the how to create metadata here and then we will direct all cases back to a consolidated publish step. I'm sure there are cases that won't perfectly fit that but in general I would think we want to maintain a single authoritative ingest-ui document. 🤔
cc: @stephenkilbourn @smohiudd

"stac_version": "1.1.0",
"description": "The Evaporative Stress Index (ESI) reveals regions of drought where vegetation is stressed due to lack of water. The ESI can capture early signals of flash drought and is produced weekly at 5-kilometer resolution for the entire globe.",
"stac_extensions": [
"https://stac-extensions.github.io/render/v2.0.0/schema.json"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"https://stac-extensions.github.io/render/v2.0.0/schema.json"
"https://stac-extensions.github.io/render/v2.0.0/schema.json",
"https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"

It looks like this meets the spec so I think we can also add the extension
https://github.com/stac-extensions/web-map-links?tab=readme-ov-file#ogc-wms

Copy link
Collaborator

@anayeaye anayeaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! I think I am going to resolve my questions about renders in the ArcGIS Server sections because it probably agrees with what we are actually doing even if it doesn't match the renders spec. Maybe we need to extend the render stac extension for these cases

"description": "Soil moisture percentile data from Land Information System (LIS) for disaster monitoring applications. This dataset provides volumetric soil moisture percentiles for the top 10cm of soil.",
"stac_extensions": [
"https://stac-extensions.github.io/render/v2.0.0/schema.json",
"https://stac-extensions.github.io/datacube/v2.2.0/schema.json"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"https://stac-extensions.github.io/datacube/v2.2.0/schema.json"
"https://stac-extensions.github.io/datacube/v2.2.0/schema.json",
"https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"

curl "https://staging.openveda.cloud/api/titiler-cmr/WebMercatorQuad/tilejson.json?concept_id=YOUR_CONCEPT_ID&datetime=2024-01-15&backend=xarray&variable=your_variable"

# Test info endpoint
curl "https://staging.openveda.cloud/api/titiler-cmr/info?concept_id=YOUR_CONCEPT_ID&datetime=2024-01-15&backend=xarray"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

B --> C[STAC Collection]
C --> D[VEDA Catalog]

subgraph "ArcGIS Services"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason the subgraphs in the architecture diagrams for both integration diagrams are hard to read--might be my browser settings?

Image

A[ArcGIS Server] --> B[pyarc2stac Library]
B --> C[STAC Collection]
C --> D[VEDA Catalog]

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
style A fill:#e1f5fe
style B fill:#f3e5f5
style C fill:#fff3e0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants