You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With each data release, Overture generates a [GERS](../) changelog to capture changes in the data tied to the unique ID for each feature. This information can be used to guide decisions about data matching, better understand data stability, and help detect data errors.
9
+
With each data release, Overture generates a [GERS](../) changelog to capture changes in the data tied to the unique ID for each feature. This information can be used to guide decisions about data matching, better understand data stability, and help detect data errors.
10
10
11
11
The GERS changelog is available as Parquet files — partitioned by theme, type, and change type — at the following locations:
12
12
@@ -15,7 +15,7 @@ The GERS changelog is available as Parquet files — partitioned by theme, t
@@ -29,24 +29,18 @@ The types of changes in the changelog include:
29
29
30
30
## Querying the GERS changelog
31
31
32
-
We can query the changelog with DuckDB to get a quick look at changes in data from the last release to the current release. In this example, we're grabbing ID, feature type, and change type for buildings in several towns across the [Amhara Region](https://en.wikipedia.org/wiki/Amhara_Region) in Ethiopia.
32
+
We can query the changelog with DuckDB to get a quick look at changes in data from the last release to the current release. In this example, we're grabbing ID, feature type, and change type for buildings in several towns across the [Amhara Region](https://en.wikipedia.org/wiki/Amhara_Region) in Ethiopia.
Taking this one step further, this time using Athena to run our query, we can join the changelog and data via Overture ID to connect the change type and feature geometries and properties.
40
+
Taking this one step further, this time using Athena to run our query, we can join the changelog and data via Overture ID to connect the change type and feature geometries and properties.
Finally we can use the results of our query to visualize building features by change type and inspect the properties for each feature. The example below, created using [kepler.gl](https://kepler.gl/), shows buildings in [Finote Selam](https://en.wikipedia.org/wiki/Finote_Selam), a city in the Amhara Region of Ethiopia. The data is from Overture's `2024-06-13-beta.0` release, to which we added 100 million new buildings, many of them derived from satellite imagery by Microsoft.
45
-
44
+
Finally we can use the results of our query to visualize building features by change type and inspect the properties for each feature. The example below, created using [kepler.gl](https://kepler.gl/), shows buildings in [Finote Selam](https://en.wikipedia.org/wiki/Finote_Selam), a city in the Amhara Region of Ethiopia. The data is from Overture's `2024-06-13-beta.0` release, to which we added 100 million new buildings, many of them derived from satellite imagery by Microsoft.
46
45
47
46
Copy file name to clipboardExpand all lines: docs/getting-data/data-mirrors/bigquery.mdx
+25-72
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,8 @@
2
2
title: BigQuery (Google)
3
3
---
4
4
5
+
6
+
5
7
Overture data is accessible in Google BigQuery as part of the [Google Cloud Public Dataset Program](https://cloud.google.com/bigquery/public-data), with the data being listed and maintained by [CARTO](https://www.carto.com).
6
8
7
9
@@ -13,104 +15,55 @@ Below is a step-by-step guide on how to access and use Overture data in BigQuery
13
15
To get started using Overture data in BigQuery, you must create or select a Google Cloud project with billing [enabled](https://cloud.google.com/billing/docs/how-to/verify-billing-enabled#confirm_billing_is_enabled_on_a_project). BigQuery is automatically enabled in new projects, but in order to activate it in a preexisting project, you should enable the [BigQuery API](https://console.cloud.google.com/apis/enableflow?apiid=bigquery&inv=1&invt=Abns8w).
14
16
15
17
## Getting Overture data in Google BigQuery
18
+
16
19
1. Log in to your [Google Cloud Console](https://console.cloud.google.com/).
17
20
2. Navigate to the **BigQuery** section by selecting `BigQuery` from the side menu or searching for it in the search bar.
18
21
19
-
20
-
21
-
1. In the BigQuery console, click **Add Data**. Then, click on **Public Datasets**.
22
-
23
-
22
+
24
23
25
-
1. The Google Cloud data marketplace will open. In the search box look for **Overture Maps**.
24
+
3. In the BigQuery console, click **Add Data**. Then, click on **Public Datasets**.
26
25
26
+
27
27
28
-
<img
29
-
src="/img/getting-data/bigquery-5.png"
30
-
alt="Overture data in Google Cloud"
31
-
width="900"
32
-
height="auto"
33
-
/>
28
+
4. The Google Cloud data marketplace will open. In the search box look for **Overture Maps**.
34
29
30
+
35
31
36
32
5. Select the **Overture Maps Data** listing. Then, click on **View dataset**.
37
33
38
-
39
-
<img
40
-
src="/img/getting-data/bigquery-6.png"
41
-
alt="Overture data in Google Cloud"
42
-
width="900"
43
-
height="auto"
44
-
/>
45
-
34
+
46
35
47
36
6. Back again in the BigQuery console you will see the `bigquery-public-data` project and the `overture_maps`dataset selected.
48
37
49
-
50
-
<img
51
-
src="/img/getting-data/bigquery-7.png"
52
-
alt="Overture data in Google Cloud"
53
-
width="900"
54
-
height="auto"
55
-
/>
56
-
38
+
57
39
58
40
7. Our recommendation is that you add the `overture_maps` dataset or entirely the `bigquery-public-data` project in your starred selection in order to have access to the Overture data always at hand.
59
41
60
-
61
-
<img
62
-
src="/img/getting-data/bigquery-8.png"
63
-
alt="Overture data in Google Cloud"
64
-
width="900"
65
-
height="auto"
66
-
/>
67
-
42
+
68
43
69
44
## Working with Overture data in BigQuery
70
45
71
-
8. Now that Overture Maps data is available in your BigQuery console, you can start using it. Note that in the `overture_maps` dataset you have access to all tables from all the different Overture Maps themes: Addresses, Base, Buildings, Divisions, Places and Transporatation.
72
-
73
-
74
-
<img
75
-
src="/img/getting-data/bigquery-9.png"
76
-
alt="Overture data in Google Cloud"
77
-
width="900"
78
-
height="auto"
79
-
/>
80
-
46
+
1. Now that Overture Maps data is available in your BigQuery console, you can start using it. Note that in the `overture_maps` dataset you have access to all tables from all the different Overture Maps themes: Addresses, Base, Buildings, Divisions, Places and Transportation.
81
47
82
-
9. The release version is detailed as a `label`in each of the tables. You can check it out in the **Details** section of each table.
48
+
83
49
50
+
2. The release version is detailed as a `label` in each of the tables. You can check it out in the **Details** section of each table.
84
51
85
-
<img
86
-
src="/img/getting-data/bigquery-10.png"
87
-
alt="Overture data in Google Cloud"
88
-
width="900"
89
-
height="auto"
90
-
/>
52
+
91
53
54
+
3. You can now query any of the tables directly from your SQL Editor in BigQuery. Here's one example for the Places data in Overture Maps.
92
55
93
-
10. You can now query any of the tables directly from your SQL Editor in BigQuery. Here's one example for the Places data in Overture Maps.
56
+
94
57
95
-
96
-
<img
97
-
src="/img/getting-data/bigquery-11.png"
98
-
alt="Overture data in Google Cloud"
99
-
width="900"
100
-
height="auto"
101
-
/>
102
-
103
-
104
-
```sql
105
-
// Identify places within the "Restaurant" category
106
-
SELECT id, phones.list[0].element AS phone, names.primaryAS name
107
-
FROM`bigquery-public-data.overture_maps.place`
108
-
WHEREcategories.primary="restaurant"
109
-
LIMIT100;
110
-
```
58
+
```sql
59
+
-- Identify places within the "Restaurant" category
60
+
SELECT id, phones.list[0].element AS phone, names.primaryAS name
61
+
FROM`bigquery-public-data.overture_maps.place`
62
+
WHEREcategories.primary="restaurant"
63
+
LIMIT100;
64
+
```
111
65
112
66
## Additional notes
113
67
114
68
-**Updates**: CARTO regularly updates the Overture datasets in BigQuery and keeps the dataset synced with the last release. You can check the release version of the data as a metadata label in each table.
115
-
-**Support**: If you encounter issues accessing the data, contact CARTO support via [email protected].
Copy file name to clipboardExpand all lines: docs/getting-data/data-mirrors/databricks.mdx
+14-76
Original file line number
Diff line number
Diff line change
@@ -5,6 +5,7 @@ title: Databricks
5
5
Overture data is accessible through the [Databricks Marketplace](https://marketplace.databricks.com/?searchKey=CARTO&sortBy=date) via public listings published and maintained by [CARTO](https://www.carto.com). Below is a step-by-step guide on how to access and use the data:
6
6
7
7
## Before you begin
8
+
8
9
Before accessing the Overture Maps data in Databricks, ensure you have:
9
10
10
11
-**Databricks Account**: An active Databricks account with access to the Databricks Marketplace.
@@ -16,108 +17,45 @@ Before accessing the Overture Maps data in Databricks, ensure you have:
16
17
1. Log in to your Databricks workspace.
17
18
2. From the Databricks workspace home page, navigate to the **Databricks Marketplace** by selecting the **Marketplace** tab from the sidebar.
4. The listings have been organized by Overture theme; therefore, you will find the following listings available: Addresses, Base, Buildings, Divisions, Places and Transportation. Select one to access the listing details.
7. Before you get access you can specify where are you planning to use the data and also provide a custom name to the Databricks Catalog in which the data will be made available. After accepting the terms, click on **Get instant access**.
10. Once you have the Overture Maps data that you need in your Catalogs you can start working with it. Following our example, in the Place table you can click on **Create** and then **Query**.
98
-
99
-
100
-
<img
101
-
src="/img/getting-data/databricks-8.png"
102
-
alt="Databricks Marketplace"
103
-
width="900"
104
-
height="auto"
105
-
/>
106
-
107
-
108
-
11. That will open the SQL Editor ready to query the `place` table.
49
+
1. Once you have the Overture Maps data that you need in your Catalogs you can start working with it. Following our example, in the Place table you can click on **Create** and then **Query**.
-**Updates**: CARTO regularly updates the Overture datasets in Databricks and keeps the listings synced with the last release. You can check the release version of the data from the share description in each listing.
122
-
-**Support**: If you encounter issues accessing the data, contact CARTO support via [email protected].
-**Databricks**: For more information on Databricks Marketplace, refer to the official [Databricks documentation](https://docs.databricks.com/en/marketplace/index.html).
0 commit comments