Add natural language search

[vgeorge]

This adds a natural language search UI to the STAC Browser, if an API endpoint is provided.

Based on the work from @sunu available at sunu#1.

Changes included:

• Added a Natural Language search form and logic
• Added a README section on how to enable the feature

To run this locally:

npm start -- --open --catalogUrl="https://earth-search.aws.element84.com/v1/" --semanticSearchApiUrl="https://api.stac-semantic-search.k8s.labs.ds.io"

Suggested next steps:

• Improve error handling (for API errors)
• Clean up console.log debug statements

@j08lue @sunu @AliceR what else do you think we need to change or implement for the demo?

ps: I set the target of this PR to a branch on this repo, but we can change it if we decide to push upstream.

[j08lue]

Nice! I'll spin this up to review the UI/UX. 🙏

Once we think it's ok, it would be great to deploy this somewhere (ds.io or Netlify or so), so we can show it to people (e.g. at LPS 23-27 June.

[sunu]

@vgeorge This is exciting! I was wondering if we could add one more thing -- would it be possible to use the return_search_params_only flag on our API so we can get just the STAC search parameters and then query the STAC API directly for items? Do you think that might be easy to implement?

Our current endpoint doesn’t handle pagination when returning items, so it would be fantastic if we could leverage the STAC API’s pagination features instead.

[j08lue]

Agree with @sunu - it would be great to let the client (STAC Browser) handle the actual STAC query and processing of results.

UI/UX-wise, the map works nicely for validating the geo-coded area of interest - it would be great to show the AOI on the map as well, on top of the scenes and their footprints.

As a user, I would like some more info about what happened and what query parameters the chatbot came up with.

We should surface the explanation that the LLM app returns. We should also provide an option for the user to show the CQL2 query that the query was translated into.

Cloud-free Sentinel-2 scenes over Berlin, Germany in June 2024

        "explanation": "Considering the following collections:\n- sentinel-2-l2a: This collection specifically contains Sentinel-2 imagery, which is required for the user's query.\n- hls2-s30: This collection includes Sentinel-2 data for analysis and has a high temporal frequency fitting the user's needs.\n- io-lulc: Derived from Sentinel-2 imagery, this collection could provide relevant land cover data for Berlin.\n- io-lulc-annual-v02: This collection offers annual land cover data based on Sentinel-2 imagery, likely containing relevant information for Berlin."

[vgeorge]

I added changes to address your feedback:

• Merged natural query input into search form (will display if feature is enabled)
• Display query explanation returned by the API under the input
• Display query intersect geojson on the map
• Populate remaining fields with query results

Populating the spatial extend field is a bit more complex. Currently it only accepts bounding boxes. In order to use the geojson from the natural search I think we need to change its behaviour to a toggle with three options:

• No spatial filter
• Bounding box filter: existing workflow, but displays the bounding box on the map after querying
• Natural search intersect: auto-selected after a natural search is finished, displays the natural search geojson on the map

This is ready for another review.

Preview:

[j08lue]

The UI looks great now, @vgeorge, thanks for addressing all my requests to make it easier for users to understand the submitted query and what happened.

I see you are not using return_search_params_only yet, but relying on the semantic search app to return the items. It probably takes a larger rework to change this - but we need to, please.

A few issues, probably due to the search API response:

1. The list of collections under the search bar has duplicates
   
2. The query against https://stac-semantic-search.labs.sunu.in returns very weird results for me, geocoding-wise. Marrakesh, Morocco, even London, UK shows up somewhere in the ocean.
3. I appreciate the warning that submitting a query will reset the form, @vgeorge . Could we clear the form right away when the user hits the Search button? That way, it is clear that this data is obsolete and that it gets replaced once the (new) query returns
   

[vgeorge]

@j08lue Thanks for the feedback. I’ve just updated the code to use return_search_params_only and only populate the form.

Regarding the item list:

1. This is coming from the API, I saw you opened an issue for it: Remove duplicate collection names stac-semantic-search#2
2. In the version you reviewed, those results were coming from the API. But now that we’re only retrieving parameters, I’ll update the form to use the AOI generated by the semantic search
3. I’m not sure I understood this one, could you please elaborate? You might also want to check the updated version

[vgeorge]

With the recent changes the form is now using the natural search area to query items:

I've changed the "Spatial Extent" input to use a radio group as there are three options now. The Natual Search Area option will be displayed only if present.

This is ready for a review.

[github-actions]

🚀 Preview deployed!

Your changes are now available at: https://developmentseed.github.io/developmentseed/stac-browser/preview/natural-language-search-demo-devseed/

This preview will be automatically updated on each push to this branch.

[github-actions]

🚀 Preview deployed!

Your changes are now available at: https://developmentseed.github.io/developmentseed/stac-browser/preview/natural-language-search-demo-devseed/

This preview will be automatically updated on each push to this branch.

[github-actions]

🚀 Preview deployed!

Your changes are now available at: https://developmentseed.github.io/stac-browser/preview/natural-language-search-demo-devseed/

This preview will be automatically updated on each push to this branch.

[j08lue]

Awesome this now works with client-side queries, @vgeorge.

Pagination in the client also works nicely now.

https://developmentseed.org/stac-browser/preview/natural-language-search-demo-devseed/#/search/external/earth-search.aws.element84.com/v1

Two small change requests

One-click query

It would be great if the search was already triggered when the user submitted their natural language query, instead of that being a two-step process. Would that be possible?

Make form inactive why LLM request is processing

When the user submits an LLM query, the form should be locked / inactive / cleared until the query returns, so they get a clear indication that the data in the form is invalid and a change is underway.

[AliceR]

That looks awesome @vgeorge ! 🚀

I have some questions around the UX... is seems confusing that the headings Natural Language Search and Temporal Extend are in the same form, like they complement each other. While actually the former populates the latter. Maybe a separator line, like before Sort, can help distinguish? And then, what is the radio button Natural Search Area doing? I understood the NLS request comes back with a bounding box, which we can use to populate the Spatial Extend checkbox and map?

Jonas' request for a One-click query could be solved with a two-button system, Populate and Search. I would suggest to make one button the primary, and style the other one in the secondary style (usually outline only).

[j08lue]

As we discussed, this version works very nicely and all essential functionality is in place. We can track additional enhancements in separate tickets / PRs.