Skip to content

Commit f9a296c

Browse files
xavipereztrxavipereztr
authored
Adding BigQuery, Snowflake and Databricks guides for "Getting Data" (#151)
* Create snowflake.mdx * Couple of images to test * Update snowflake.mdx * Snowflake Marketplace images * Create snowflake_places.sql * Update snowflake.mdx Complete Snowflake guide. * bigquery image * Create bigquery.mdx * Adding some image for bigquery guide * More images for bigquery guide * Add image * Update bigquery.mdx * Update bigquery.mdx * Images for databricks guide * Create databricks.mdx
1 parent f3bbb8b commit f9a296c

30 files changed

+360
-0
lines changed

‎docs/getting-data/bigquery.mdx

+137
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
---
2+
title: Google BigQuery
3+
---
4+
5+
Overture data is accessible in Google BigQuery as part of the [Google Cloud Public Dataset Program](https://cloud.google.com/bigquery/public-data), with the data being listed and maintained by [CARTO](https.//www.carto.com).
6+
7+
8+
<img
9+
src="../../static/img/getting-data/BigQuery-1.png"
10+
alt="Overture data in Google Cloud"
11+
width="900"
12+
height="auto"
13+
/>
14+
15+
16+
Below is a step-by-step guide on how to access and use Overture data in BigQuery:
17+
18+
## Before you begin
19+
20+
To get started using Overture data in BigQuery, you must create or select a Google Cloud project with billing [enabled](https://cloud.google.com/billing/docs/how-to/verify-billing-enabled#confirm_billing_is_enabled_on_a_project). BigQuery is automatically enabled in new projects, but in order to activate it in a preexisting project, you should enable the [BigQuery API](https://console.cloud.google.com/apis/enableflow?apiid=bigquery&inv=1&invt=Abns8w).
21+
22+
## Getting Overture Data in Google BigQuery
23+
1. Log in to your [Google Cloud Console](https://console.cloud.google.com/).
24+
2. Navigate to the **BigQuery** section by selecting `BigQuery` from the side menu or searching for it in the search bar.
25+
26+
27+
<img
28+
src="../../static/img/getting-data/bigquery-2.png"
29+
alt="Overture data in Google Cloud"
30+
width="900"
31+
height="auto"
32+
/>
33+
34+
35+
3. In the BigQuery console, click **Add Data**. Then, click on **Public Datasets**.
36+
37+
38+
<img
39+
src="../../static/img/getting-data/bigquery-4.png"
40+
alt="Overture data in Google Cloud"
41+
width="900"
42+
height="auto"
43+
/>
44+
45+
46+
4. The Google Cloud data marketplace will open. In the search box look for **Overture Maps**.
47+
48+
49+
<img
50+
src="../../static/img/getting-data/bigquery-5.png"
51+
alt="Overture data in Google Cloud"
52+
width="900"
53+
height="auto"
54+
/>
55+
56+
57+
5. Select the **Overture Maps Data** listing. Then, click on **View dataset**.
58+
59+
60+
<img
61+
src="../../static/img/getting-data/bigquery-6.png"
62+
alt="Overture data in Google Cloud"
63+
width="900"
64+
height="auto"
65+
/>
66+
67+
68+
6. Back again in the BigQuery console you will see the `bigquery-public-data` project and the `overture_maps`dataset selected.
69+
70+
71+
<img
72+
src="../../static/img/getting-data/bigquery-7.png"
73+
alt="Overture data in Google Cloud"
74+
width="900"
75+
height="auto"
76+
/>
77+
78+
79+
7. Our recommendation is that you add the `overture_maps` dataset or entirely the `bigquery-public-data` project in your starred selection in order to have access to the Overture data always at hand.
80+
81+
82+
<img
83+
src="../../static/img/getting-data/bigquery-8.png"
84+
alt="Overture data in Google Cloud"
85+
width="900"
86+
height="auto"
87+
/>
88+
89+
90+
## Working with Overture Data in BigQuery
91+
92+
8. Now that Overture Maps data is available in your BigQuery console, you can start using it. Note that in the `overture_maps` dataset you have access to all tables from all the different Overture Maps themes: Addresses, Base, Buildings, Divisions, Places and Transporatation.
93+
94+
95+
<img
96+
src="../../static/img/getting-data/bigquery-9.png"
97+
alt="Overture data in Google Cloud"
98+
width="900"
99+
height="auto"
100+
/>
101+
102+
103+
9. The release version is detailed as a `label` in each of the tables. You can check it out in the **Details** section of each table.
104+
105+
106+
<img
107+
src="../../static/img/getting-data/bigquery-10.png"
108+
alt="Overture data in Google Cloud"
109+
width="900"
110+
height="auto"
111+
/>
112+
113+
114+
10. You can now query any of the tables directly from your SQL Editor in BigQuery. Here's one example for the Places data in Overture Maps.
115+
116+
117+
<img
118+
src="../../static/img/getting-data/bigquery-11.png"
119+
alt="Overture data in Google Cloud"
120+
width="900"
121+
height="auto"
122+
/>
123+
124+
125+
```sql
126+
// Identify places within the "Restaurant" category
127+
SELECT id, phones.list[0].element AS phone, names.primary AS name
128+
FROM `bigquery-public-data.overture_maps.place`
129+
WHERE categories.primary = "restaurant"
130+
LIMIT 100;
131+
```
132+
133+
## Additional Notes
134+
135+
- **Updates**: CARTO regularly updates the Overture datasets in BigQuery and keeps the dataset synced with the last release. You can check the release version of the data as a metadata label in each table.
136+
- **Support**: If you encounter issues accessing the data, contact CARTO support via [email protected].
137+
- **CARTO**: Once you have the Overture data you need in BigQuery, you can use CARTO's cloud-native spatial analytics platform running on your BigQuery account to build interactive maps, data procesing and analytical workflows or geospatial applications. Not yet on CARTO? Open a 14-day free trial [here](https://www.carto.com/signup).

‎docs/getting-data/databricks.mdx

+124
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
---
2+
title: Databricks
3+
---
4+
5+
Overture data is accessible through the [Databricks Marketplace](https://marketplace.databricks.com/?searchKey=CARTO&sortBy=date) via public listings published and maintained by [CARTO](https.//www.carto.com). Below is a step-by-step guide on how to access and use the data:
6+
7+
## Before you begin
8+
Before accessing the Overture Maps data in Databricks, ensure you have:
9+
10+
- **Databricks Account**: An active Databricks account with access to the Databricks Marketplace.
11+
- **Workspace Access**: A Databricks workspace where you can import and analyze the data.
12+
- **Marketplace Permissions**: The appropriate permissions to browse and subscribe to datasets in the Databricks Marketplace.
13+
14+
## Getting Overture Data from Databricks Marketplace
15+
16+
1. Log in to your Databricks workspace.
17+
2. From the Databricks workspace home page, navigate to the **Databricks Marketplace** by selecting the **Marketplace** tab from the sidebar.
18+
19+
20+
<img
21+
src="../../static/img/getting-data/databricks-1.png"
22+
alt="Databricks Marketplace"
23+
width="900"
24+
height="auto"
25+
/>
26+
27+
28+
3. Use the search bar in the Databricks Marketplace to search for `CARTO` or `Overture Maps`.
29+
30+
31+
<img
32+
src="../../static/img/getting-data/databricks-2.png"
33+
alt="Databricks Marketplace"
34+
width="900"
35+
height="auto"
36+
/>
37+
38+
39+
4. The listings have been organized by Overture theme; therefore, you will find the following listings available: Addresses, Base, Buildings, Divisions, Places and Transportation. Select one to access the listing details.
40+
41+
42+
<img
43+
src="../../static/img/getting-data/databricks-3.png"
44+
alt="Databricks Marketplace"
45+
width="900"
46+
height="auto"
47+
/>
48+
49+
50+
5. Review the dataset description, terms of use, and any documentation provided by CARTO.
51+
6. Click **Get instant access** to request access to the dataset.
52+
53+
54+
<img
55+
src="../../static/img/getting-data/databricks-4.png"
56+
alt="Databricks Marketplace"
57+
width="900"
58+
height="auto"
59+
/>
60+
61+
62+
7. Before you get access you can specify where are you planning to use the data and also provide a custom name to the Databricks Catalog in which the data will be made available. After accepting the terms, click on **Get instant access**.
63+
64+
65+
<img
66+
src="../../static/img/getting-data/databricks-5.png"
67+
alt="Databricks Marketplace"
68+
width="900"
69+
height="auto"
70+
/>
71+
72+
73+
8. Now, in the same listing page you have the option to click on **Open**.
74+
75+
76+
<img
77+
src="../../static/img/getting-data/databricks-6.png"
78+
alt="Databricks Marketplace"
79+
width="900"
80+
height="auto"
81+
/>
82+
83+
84+
9. After clicking on **Open** you will be taken to your Catalogs with the newly created Catalog `carto_overtue_maps_places`open.
85+
86+
87+
<img
88+
src="../../static/img/getting-data/databricks-7.png"
89+
alt="Databricks Marketplace"
90+
width="900"
91+
height="auto"
92+
/>
93+
94+
95+
## Working with Overture Data in Databricks
96+
97+
10. Once you have the Overture Maps data that you need in your Catalogs you can start working with it. Following our example, in the Place table you can click on **Create** and then **Query**.
98+
99+
100+
<img
101+
src="../../static/img/getting-data/databricks-8.png"
102+
alt="Databricks Marketplace"
103+
width="900"
104+
height="auto"
105+
/>
106+
107+
108+
11. That will open the SQL Editor ready to query the `place` table.
109+
110+
111+
<img
112+
src="../../static/img/getting-data/databricks-9.png"
113+
alt="Databricks Marketplace"
114+
width="900"
115+
height="auto"
116+
/>
117+
118+
119+
## Additional Notes
120+
121+
- **Updates**: CARTO regularly updates the Overture datasets in Databricks and keeps the listings synced with the last release. You can check the release version of the data from the share description in each listing.
122+
- **Support**: If you encounter issues accessing the data, contact CARTO support via [email protected].
123+
- **CARTO**: Once you have the Overture data you need in Databricks, you can use CARTO's cloud-native spatial analytics platform running on your Databricks account to build interactive maps, data procesing and analytical workflows or geospatial applications. Not yet on CARTO? Open a 14-day free trial [here](https://www.carto.com/signup).
124+
- **Databricks**: For more information on Databricks Marketplace, refer to the official [Databricks documentation](https://docs.databricks.com/en/marketplace/index.html).

‎docs/getting-data/snowflake.mdx

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
title: Snowflake
3+
---
4+
5+
Overture data is accessible through the [Snowflake Data Marketplace](https://app.snowflake.com/marketplace/providers/GZT0ZKUCHE3/CARTO) via public listings published and maintained by [CARTO](https.//www.carto.com). Below is a step-by-step guide on how to access and use the data:
6+
7+
## Getting Overture Data from Snowflake Marketplace
8+
9+
1. Log in to your Snowflake account. Note that you must have an active Snowflake account with the necessary permissions to access the Data Marketplace.
10+
2. In the Snowflake interface, navigate to the **Marketplace** tab in the sidebar menu within section **Data Products**.
11+
12+
13+
<img
14+
src="../../static/img/getting-data/snowflake-marketplace-1.png"
15+
alt="Snowflake Data Marketplace"
16+
width="900"
17+
height="auto"
18+
/>
19+
20+
21+
3. Use the search bar at the top of the Data Marketplace and enter `CARTO`.
22+
4. The listings have been organized by Overture theme; therefore, you will find the following listings available: Addresses, Base, Buildings, Divisions, Places and Transportation. Select one to access the listing details.
23+
24+
25+
<img
26+
src="../../static/img/getting-data/snowflake-marketplace-2.png"
27+
alt="Overture Listings"
28+
width="900"
29+
height="auto"
30+
/>
31+
32+
33+
5. Review the data description, example queries, terms of use, etc.
34+
6. Click on **Get**.
35+
36+
37+
<img
38+
src="../../static/img/getting-data/snowflake-marketplace-3.png"
39+
alt="Overture Places"
40+
width="900"
41+
height="auto"
42+
/>
43+
44+
45+
7. After clicking on **Get**, you are allowed to specify the name of the database in which to include the data and the user roles that will be able to work with it.
46+
47+
<img
48+
src="../../static/img/getting-data/snowflake-marketplace-4.png"
49+
alt="Getting data"
50+
width="900"
51+
height="auto"
52+
/>
53+
54+
55+
## Working with Overture Data in Snowflake
56+
8. After access is granted, navigate to the **Databases** section in Snowflake (under **Data**).
57+
9. Locate the Overture dataset in the database that you have just created.
58+
59+
60+
<img
61+
src="../../static/img/getting-data/snowflake-marketplace-5.png"
62+
alt="Overture data in database"
63+
width="900"
64+
height="auto"
65+
/>
66+
67+
13. Once the database is created, you can then navigate to your worksheet in Snowflake and start working with the data.
68+
14. Use SQL to query the Overture data. Below is an example query to retrieve some attributes from the Places data in Overture.
69+
70+
71+
```sql
72+
// Identify places within the "Restaurant" category
73+
SELECT id, phones['list'][0]['element'] AS phone, names['primary'] AS name
74+
FROM CARTO.PLACE
75+
WHERE categories['primary'] = 'restaurant'
76+
LIMIT 100;
77+
```
78+
79+
<img
80+
src="../../static/img/getting-data/snowflake-marketplace-6.png"
81+
alt="Overture data in database"
82+
width="900"
83+
height="auto"
84+
/>
85+
86+
## Additional Notes
87+
88+
- **Updates**: CARTO regularly updates the Overture datasets in Snowflake and keeps the listings synced with the last release. You can check the release version of the data from the share description in each listing.
89+
- **Support**: If you encounter issues accessing the data, contact CARTO support via [email protected].
90+
- **CARTO**: Once you have the Overture data you need in Snowflake, you can use CARTO's cloud-native spatial analytics platform running on your Snowflake account to build interactive maps, data procesing and analytical workflows or geospatial applications. Not yet on CARTO? Open a 14-day free trial [here](https://www.carto.com/signup).
91+
- **Snowflake**: For more information on Snowflake Data Marketplace, refer to the official [Snowflake documentation](https://docs.snowflake.com/en/user-guide/data-marketplace.html).

‎src/queries/Snowflake/snowflake_places.sql

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
// Identify places within the "Restaurant" category
2+
/*
3+
Retrieves the geographic IDs (geoid), phone numbers (phones), and primary names (names) of places categorized as "restaurant" from the specified table. Limited to 100 results.
4+
*/
5+
SELECT id, phones['list'][0]['element'] AS phone, names['primary'] AS name
6+
FROM CARTO.PLACE
7+
WHERE categories['primary'] = 'restaurant'
8+
LIMIT 100;

‎static/img/getting-data/BigQuery-1.png

486 KB
Loading

‎static/img/getting-data/bigquery-10.png

483 KB
Loading

‎static/img/getting-data/bigquery-11.png

497 KB
Loading

‎static/img/getting-data/bigquery-2.png

481 KB
Loading

‎static/img/getting-data/bigquery-3.png

503 KB
Loading

‎static/img/getting-data/bigquery-4.png

584 KB
Loading

‎static/img/getting-data/bigquery-5.png

410 KB
Loading

‎static/img/getting-data/bigquery-6.png

562 KB
Loading

‎static/img/getting-data/bigquery-7.png

490 KB
Loading

‎static/img/getting-data/bigquery-8.png

492 KB
Loading

‎static/img/getting-data/bigquery-9.png

519 KB
Loading

‎static/img/getting-data/databricks-1.png

549 KB
Loading

‎static/img/getting-data/databricks-2.png

451 KB
Loading

‎static/img/getting-data/databricks-3.png

855 KB
Loading

‎static/img/getting-data/databricks-4.png

889 KB
Loading

‎static/img/getting-data/databricks-5.png

955 KB
Loading

‎static/img/getting-data/databricks-6.png

787 KB
Loading

‎static/img/getting-data/databricks-7.png

430 KB
Loading

‎static/img/getting-data/databricks-8.png

456 KB
Loading

‎static/img/getting-data/databricks-9.png

369 KB
Loading

‎static/img/getting-data/snowflake-marketplace-1.png

745 KB
Loading

‎static/img/getting-data/snowflake-marketplace-2.png

481 KB
Loading

‎static/img/getting-data/snowflake-marketplace-3.png

480 KB
Loading

‎static/img/getting-data/snowflake-marketplace-4.png

624 KB
Loading

‎static/img/getting-data/snowflake-marketplace-5.png

715 KB
Loading

‎static/img/getting-data/snowflake-marketplace-6.png

570 KB
Loading

0 commit comments

Comments
 (0)