docs/examples/time_series_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d30ab27f-db17-4533-826f-1e514bfca9b5",
   "metadata": {},
   "source": [
    "# time series API\n",
    "The `timeseries` extension provides endpoints for requesting results for all points or intervals\n",
    "along a time series. The `/timeseries` family of endpoints works by converting\n",
    "the provided time series parameters (`datetime`, `step`, and `temporal_mode`) into a set of\n",
    "`datetime` query parameters for the corresponding lower-level endpoint, running asynchronous\n",
    "requests to the lower-level endpoint, then collecting the results and formatting them in a coherent\n",
    "format for the user.\n",
    "\n",
    "The time series structure is defined by the `datetime`, `step`, and `temporal_mode` parameters.\n",
    "\n",
    "The `temporal_mode` mode parameter controls whether or not CMR is queried for a particular \n",
    "point-in-time (`temporal_mode=point`) or over an entire interval (`temporal_mode=interval`).\n",
    "In general, it is best to use `temporal_mode=point` for datasets where granules overlap completely\n",
    "in space (e.g. daily sea surface temperature predictions) because the /timeseries endpoints will\n",
    "create a mosaic of all assets returned by the query and the first asset to cover a pixel will\n",
    "be used. For datasets where it requires granules from multiple timestamps to fully cover an AOI,\n",
    "`temporal_mode=interval` is appropriate. For example, you can get weekly composites of satellite\n",
    "imagery for visualization purposes with `step=P1W & temporal_mode=interval`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e13048e-765e-4e8c-838a-d81ada066893",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import IFrame\n",
    "\n",
    "# if running titiler-cmr in the docker network\n",
    "# titiler_endpoint = \"http://localhost:8081\"\n",
    "\n",
    "# titiler-cmr-staging deployment\n",
    "titiler_endpoint = \"https://dev-titiler-cmr.delta-backend.com\"\n",
    "\n",
    "IFrame(f\"{titiler_endpoint}/api.html#Timeseries\", 900, 500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2442bc5-ba52-4be9-a609-e93052cd9d99",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from datetime import datetime\n",
    "\n",
    "import httpx\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from folium import LayerControl, Map, TileLayer\n",
    "from geojson_pydantic import Feature, Polygon\n",
    "from IPython.display import Image, display"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "927c00d4-fccd-4622-988c-6ea2366ece3a",
   "metadata": {},
   "source": [
    "## time series parameters\n",
    "\n",
    "The time series API makes it possible to return results for many points along a timeseries with a single request.\n",
    "The available parameters are:\n",
    "- **`datetime`** (str): Either a date-time, an interval, or a comma-separated list of date-times or intervals. Date and time expressions adhere to rfc3339 ('2020-06-01T09:00:00Z') format.\n",
    "- **`step`** (str): width of individual time steps expressed as a [IS8601 duration](https://en.wikipedia.org/wiki/ISO_8601#Durations)\n",
    "- **`temporal_mode`** (str): if `\"point\"`, queries will be made for the individual timestamps along the timeseries. If `\"interval\"`, queries will be made for the periods *between* each timestamp along the timeseries.\n",
    "\n",
    "There are many ways to combine the parameters to produce a time series.\n",
    "\n",
    "1. Exact points in time from a start to and end datetime:\n",
    "  - provide `datetime={start_datetime}/{end_datetime}`, `step={step_width}`, and `temporal_mode=point` where `step_width` is something like `P1D` for daily or `P2W` for bi-weekly.\n",
    "  - provide `datetime={start_datetime}/{end_datetime}`, and `temporal_mode=point` without `step` to get a point for every unique timestamp in the granules between `start_datetime` and `end_datetime`.\n",
    "2. Fixed-width intervals between a start and end datetime:\n",
    "  - provide `datetime={start_datetime}/{end_datetime}`, `step`, and `temporal_mode=interval`\n",
    "3. Specific datetimes\n",
    "  - provide `datetime=2024-10-01T00:00:01Z,2024-10-02T00:00:01Z`\n",
    "4. Specific datetime intervals\n",
    "  - provide `datetime=2024-10-01T00:00:01Z/2024-10-01T23:59:59Z,2024-10-05T00:00:01Z/2024-10-05T23:59:59Z`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2e193da-6a56-4025-b020-e7e7478eddec",
   "metadata": {},
   "source": [
    "## How to use the timeseries API with `titiler.cmr`\n",
    "\n",
    "The `/timeseries` endpoints work by interpreting the time series parameters (e.g. `datetime` and `step`) and parameterizing a set of lower-level requests to the related endpoint. For example, a request to `/timeseries/statistics` for a set of four points in time each one week apart will fire off four requests to the `/statistics` endpoint with a particular value in the `datetime` parameter. The results are collected and returned in a coherent format that can be consumed in a table or a chart. \n",
    "\n",
    "Every `/timeseries` request in `titiler.cmr` will require both a `concept_id` and a set of time series parameters. The [GHRSST Level 4 GAMSSA_28km Global Foundation Sea Surface Temperature Analysis v1.0 dataset (GDS2)](https://cmr.earthdata.nasa.gov/search/concepts/C2036881735-POCLOUD.html) is a useful dataset for demo purposes because the granule assets are small (~1MB each).\n",
    "\n",
    "See [Time series API limits](../../deployment/time_series_api_limits/) for details on limits to the spatial and temporal extent for requests to a `titiler-cmr` deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "100849a8-283f-4dd8-ae26-75aab82faa1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "concept_id = \"C2036881735-POCLOUD\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e79197d-6a77-4c6e-850f-f1bb5c6e7c01",
   "metadata": {},
   "source": [
    "The `/timeseries` `GET` endpoint is useful for demonstrating how the timeseries family of endpoints constructs sub-requests. It returns the list of `titiler.cmr` query parameters (`datetime` and `concept_id`) that will be used to generate the timeseries results."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f84fdf97-c8ce-4ddd-a31f-46dd0b9658a1",
   "metadata": {},
   "source": [
    "### Time series for all granules between a start/end datetime\n",
    "\n",
    "For some datasets that have granules that are regularly spaced in time (e.g. daily), it is useful to be able to quickly specify a summary of all points in time between a start and end datetime. You can do that by simply providing the `start_datetime` and `end_datetime` parameters. The application will query CMR and produce a list of unique `datetime` values from the results of the granule search. If a granule represents a datetime range, it will return the midpoint between the start and end for a single granule."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28b1a023-f755-4c9e-b9c7-45a429a4071d",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = httpx.get(\n",
    "    f\"{titiler_endpoint}/timeseries\",\n",
    "    params={\n",
    "        \"concept_id\": concept_id,\n",
    "        \"datetime\": \"2024-10-01T00:00:01Z/2024-10-05T00:00:01Z\",\n",
    "    },\n",
    "    timeout=None,\n",
    ").json()\n",
    "\n",
    "print(json.dumps(response, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "656d7070-cc15-4bd5-9f81-a0f69f06e9c7",
   "metadata": {},
   "source": [
    "### Weekly timeseries\n",
    "\n",
    "Sometimes you might be interested in a report with lower temporal resolution than the maximum availble for a dataset. By setting `step=\"P1W\"` and `temporal_mode=\"point\"`, you can get a weekly series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee52a82f-1e17-4f57-93fe-93096e0abfc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = httpx.get(\n",
    "    f\"{titiler_endpoint}/timeseries\",\n",
    "    params={\n",
    "        \"concept_id\": concept_id,\n",
    "        \"datetime\": \"2024-10-01T00:00:01Z/2024-10-30T00:00:01Z\",\n",
    "        \"step\": \"P1W\",\n",
    "        \"temporal_mode\": \"point\",\n",
    "    },\n",
    ").json()\n",
    "\n",
    "print(json.dumps(response, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09d36cf4-d15a-4a52-bf1c-cec1352d5c1f",
   "metadata": {},
   "source": [
    "### Periodic timeseries\n",
    "\n",
    "Some datasets (like satellite imagery) may consist of granules that do not fully cover an arbitrary area of interest. In this case it is useful to construct a time series from a set of datetime ranges so that granules can be mosaiced to ensure each step has full coverage.\n",
    "\n",
    "To create a set of non-overlapping week-long datetime ranges, you can modify the query to use `temporal_mode=\"interval\"` which will create ranges that start on the weekly values returned in the previous query and extend up to the second before the next value in the series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1858b4a3-56c7-4cbe-a6f3-203bc2fdc20f",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = httpx.get(\n",
    "    f\"{titiler_endpoint}/timeseries\",\n",
    "    params={\n",
    "        \"concept_id\": concept_id,\n",
    "        \"datetime\": \"2024-10-01T00:00:01Z/2024-10-30T00:00:01Z\",\n",
    "        \"step\": \"P1W\",\n",
    "        \"temporal_mode\": \"interval\",\n",
    "    },\n",
    ").json()\n",
    "\n",
    "print(json.dumps(response, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7aa1370e-2b4a-4424-95c5-5f7c4db203f0",
   "metadata": {},
   "source": [
    "### Custom time series\n",
    "If you want to specify the exact datetime values for a time series and you either cannot do not want to use the time series parameters, you can supply a set of comma-separated datetimes and/or datetime ranges to the `datetime` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37b7e718-e921-45f3-bdd8-c95f59c62044",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = httpx.get(\n",
    "    f\"{titiler_endpoint}/timeseries\",\n",
    "    params={\n",
    "        \"concept_id\": concept_id,\n",
    "        \"datetime\": \",\".join(\n",
    "            [\"2024-10-01T00:00:01Z\", \"2024-10-07T00:00:01Z/2024-10-09T23:59:59Z\"]\n",
    "        ),\n",
    "    },\n",
    ").json()\n",
    "\n",
    "print(json.dumps(response, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7b93515-70aa-4650-9040-df45f0dcb35e",
   "metadata": {},
   "source": [
    "## Example: sea surface temperature GIF\n",
    "\n",
    "The `/timeseries/bbox` endpoint can be used to produce a GIF that shows a visualization of granules over time.  \n",
    "\n",
    "The example below shows biweekly sea surface temperature estimates from the GAMSSA dataset for the period from November 2023 through October 2024."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f5512063-ea52-486a-8dc1-83052352f58e",
   "metadata": {},
   "outputs": [],
   "source": [
    "minx, miny, maxx, maxy = -180, -90, 180, 90\n",
    "request = httpx.get(\n",
    "    f\"{titiler_endpoint}/timeseries/bbox/{minx},{miny},{maxx},{maxy}.gif\",\n",
    "    params={\n",
    "        \"concept_id\": concept_id,\n",
    "        \"datetime\": \"2023-11-01T00:00:01Z/2024-10-30T23:59:59Z\",\n",
    "        \"step\": \"P2W\",\n",
    "        \"temporal_mode\": \"point\",\n",
    "        \"variable\": \"analysed_sst\",\n",
    "        \"backend\": \"xarray\",\n",
    "        \"colormap_name\": \"thermal\",\n",
    "        \"rescale\": [[273, 315]],\n",
    "    },\n",
    "    timeout=None,\n",
    ")\n",
    "display(Image(request.content))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da58f5bc-e29e-4710-af9e-9f8a93f92572",
   "metadata": {},
   "source": [
    "## Example: HLSL30 GIF\n",
    "The example below shows a weekly mosaic of imagery from the Harmonized Landsat Sentinel L30 (HLSL30) collection for the period from July through October 2021 in the area of the [Greenwood Fire](https://en.wikipedia.org/wiki/Greenwood_Fire)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dbb3ae5-eced-4cc4-85eb-654dcc30a2a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "minx, miny, maxx, maxy = -91.816, 47.491, -91.359, 47.716\n",
    "request = httpx.get(\n",
    "    f\"{titiler_endpoint}/timeseries/bbox/{minx},{miny},{maxx},{maxy}/512x512.gif\",\n",
    "    params={\n",
    "        \"concept_id\": \"C2021957657-LPCLOUD\",\n",
    "        \"datetime\": \"2021-07-01T00:00:00Z/2021-10-30T00:00:00Z\",\n",
    "        \"step\": \"P5D\",\n",
    "        \"temporal_mode\": \"interval\",\n",
    "        \"backend\": \"rasterio\",\n",
    "        \"bands_regex\": \"B[0-9][0-9]\",\n",
    "        \"bands\": [\"B04\", \"B03\", \"B02\"],\n",
    "        \"color_formula\": \"Gamma RGB 3.5 Saturation 1.7 Sigmoidal RGB 15 0.35\",\n",
    "        \"fps\": 2,\n",
    "    },\n",
    "    timeout=None,\n",
    ")\n",
    "\n",
    "display(Image(request.content))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b60879b-20e9-4c4e-a2e1-4293bb78374f",
   "metadata": {},
   "source": [
    "## Example: sea surface temperature statistics\n",
    "The `/timeseries/statistics` endpoint will produce summary statistics for an AOI for all points along a timeseries.\n",
    "\n",
    "The example below shows daily sea surface temperature summary statistics for the Gulf of Mexico from the GAMSSA dataset for the period from March 2022 through October 2024."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a9fffbe-2b49-48dd-abb3-ae558e2bcd52",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "minx, miny, maxx, maxy = -98.676, 18.857, -81.623, 31.097\n",
    "geojson = Feature(\n",
    "    type=\"Feature\",\n",
    "    geometry=Polygon.from_bounds(minx, miny, maxx, maxy),\n",
    "    properties={},\n",
    ")\n",
    "request = httpx.post(\n",
    "    f\"{titiler_endpoint}/timeseries/statistics\",\n",
    "    params={\n",
    "        \"concept_id\": concept_id,\n",
    "        \"datetime\": \"2022-03-01T00:00:01Z/2024-10-30T23:59:59Z\",\n",
    "        \"step\": \"P1D\",\n",
    "        \"temporal_mode\": \"point\",\n",
    "        \"variable\": \"analysed_sst\",\n",
    "        \"backend\": \"xarray\",\n",
    "    },\n",
    "    json=geojson.model_dump(exclude_none=True),\n",
    "    timeout=None,\n",
    ")\n",
    "\n",
    "request.raise_for_status()\n",
    "response = request.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d1f6fdd-de66-4feb-844e-80dd40ab40a0",
   "metadata": {},
   "source": [
    "The `/timeseries/statistics` endpoint returns the GeoJSON with statistics for each step in the time series embedded in the properties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30d5a2e2-ed87-472e-ac6a-68a061f33a82",
   "metadata": {},
   "outputs": [],
   "source": [
    "stats = response[\"properties\"][\"statistics\"]\n",
    "print(len(stats))\n",
    "\n",
    "stats_preview = {\n",
    "    timestamp: sst_stats\n",
    "    for i, (timestamp, sst_stats) in enumerate(stats.items())\n",
    "    if i < 2\n",
    "}\n",
    "print(json.dumps(stats_preview, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2298b432-0c12-4dca-8d14-078339d2ae4f",
   "metadata": {},
   "source": [
    "The statistics output can be used to generate plots like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "348cb80e-dd1a-4a68-93fc-b9a320d223dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = response[\"properties\"][\"statistics\"]\n",
    "\n",
    "dates = []\n",
    "means = []\n",
    "stds = []\n",
    "\n",
    "for date_str, values in data.items():\n",
    "    dates.append(datetime.fromisoformat(date_str))\n",
    "    means.append(values[\"analysed_sst\"][\"mean\"])\n",
    "    stds.append(values[\"analysed_sst\"][\"std\"])\n",
    "\n",
    "plt.figure(figsize=(10, 6))\n",
    "\n",
    "plt.plot(dates, means, \"b-\", label=\"Mean\")\n",
    "\n",
    "plt.fill_between(\n",
    "    dates,\n",
    "    np.array(means) - np.array(stds),\n",
    "    np.array(means) + np.array(stds),\n",
    "    alpha=0.2,\n",
    "    color=\"b\",\n",
    "    label=\"Standard Deviation\",\n",
    ")\n",
    "\n",
    "plt.xlabel(\"Date\")\n",
    "plt.ylabel(\"Temperature (K)\")\n",
    "plt.title(\"Mean sea surface temperature in the Gulf of Mexico\")\n",
    "plt.legend()\n",
    "\n",
    "plt.xticks(rotation=45)\n",
    "\n",
    "plt.tight_layout()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b6faa38-3563-439c-af53-3f107bb1f09c",
   "metadata": {},
   "source": [
    "## Example: Time series raster tiles\n",
    "\n",
    "It could be useful to allow users to select a timestep in an interactive map. You can use the `/timeseries/tilejson` endpoint for that purpose. The following example shows how you could use it to provide time series capability to an interactive map of sea ice cover."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fcd07c0-3e14-4c1b-908a-dc7c98da8516",
   "metadata": {},
   "outputs": [],
   "source": [
    "minx, miny, maxx, maxy = -180, -90, 180, 90\n",
    "request = httpx.get(\n",
    "    f\"{titiler_endpoint}/timeseries/WebMercatorQuad/tilejson.json\",\n",
    "    params={\n",
    "        \"concept_id\": concept_id,\n",
    "        \"datetime\": \"2023-11-01T00:00:01Z/2024-10-30T23:59:59Z\",\n",
    "        \"step\": \"P1M\",\n",
    "        \"temporal_mode\": \"point\",\n",
    "        \"variable\": \"sea_ice_fraction\",\n",
    "        \"backend\": \"xarray\",\n",
    "        \"colormap_name\": \"blues_r\",\n",
    "        \"rescale\": [[0, 1]],\n",
    "    },\n",
    "    timeout=None,\n",
    ")\n",
    "tilejsons = request.json()\n",
    "tilejson_preview = {\n",
    "    timestamp: tilejson\n",
    "    for i, (timestamp, tilejson) in enumerate(tilejsons.items())\n",
    "    if i < 2\n",
    "}\n",
    "print(json.dumps(tilejson_preview, indent=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d443a41d-6b87-4fbf-9390-e210c4f3cf89",
   "metadata": {},
   "outputs": [],
   "source": [
    "m = Map(location=[0, 0], zoom_start=3, min_zoom=3)\n",
    "for datetime_, tilejson in tilejsons.items():\n",
    "    label = datetime.fromisoformat(datetime_).strftime(\"%Y-%m\")\n",
    "    TileLayer(\n",
    "        tiles=tilejson[\"tiles\"][0],\n",
    "        attr=\"GAMSSA SST\",\n",
    "        overlay=True,\n",
    "        name=label,\n",
    "        show=False,\n",
    "    ).add_to(m)\n",
    "LayerControl(collapsed=False).add_to(m)\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca3e8eb4-6d11-41d1-accf-b1e13e3366e1",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}