From b821a55f6eaf7f0722dc14f0cb02919df23f1a88 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Mon, 12 May 2025 21:46:57 -0500 Subject: [PATCH 01/17] Update setup.md 1. Create folder for workshop - store data and venv folder 2. Call virtual environment folder "venv", following conventions 3. Require Python 3.11.9. Newer versions of Python are not yet compatible with Tensorflow --- learners/setup.md | 111 ++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 93 insertions(+), 18 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index e90c45d0..7c47037c 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -1,37 +1,86 @@ --- title: Setup --- -## Software Setup +## Setup +The setup steps below will help you prepare a folder for the workshop, install Python and the needed libraries in a virtual environment, and download the data we'll be exploring. + +## 1. Setup workshop folder + +Create a folder on your desktop called `dl_workshop` for storing the workshop data and required packages. + +```shell +cd ~/Desktop +mkdir dl_workshop +cd dl_workshop +pwd +``` ::::::::::::::::::::::::::::::::::::::: discussion -### Installing Python +## 2. Installing Python + +[Python][python] is a popular language for scientific computing and a frequent choice for machine learning. -[Python][python] is a popular language for scientific computing, and a frequent choice -for machine learning as well. -To install Python, follow the [Beginner's Guide](https://wiki.python.org/moin/BeginnersGuide/Download) or head straight to the [download page](https://www.python.org/downloads/). +Python version requirement: This workshop requires Python 3.11.9. Newer versions like 3.12 or 3.13 are not yet fully compatible with TensorFlow and may cause issues. Even Python 3.11.9 may have some edge cases, but it works well enough to be the default in Google Colab and is stable for the purposes of this workshop. -Please set up your python environment at least a day in advance of the workshop. -If you encounter problems with the installation procedure, ask your workshop organizers via e-mail for assistance so -you are ready to go as soon as the workshop begins. +To install Python 3.11.9, go to the [official 3.11.9 downloads page](https://www.python.org/downloads/release/python-3119//). Choose the installer that matches your operating system (Windows, macOS, or Linux). + +Please set up your Python environment at least a day in advance of the workshop. If you run into issues with installation, contact the workshop organizers by email so you're ready to begin on time. ::::::::::::::::::::::::::::::::::::::::::::::::::: -## Installing the required packages{#packages} + +## 3. Determine which `python` command to use for downstream setup steps + +Different systems and Python installations (e.g., Anaconda, Git Bash, system Python, Windows Store, etc.) may register different command names. This quick check helps identify which one points to Python 3.11.9 on your machine. + +Run the following in your terminal (Git Bash, Anaconda Prompt, or macOS/Linux shell): + +```shell +python --version +py --version +python3 --version +``` + +Use whichever one returns Python 3.11.9 for the rest of the setup steps. + +Example output: + +```output +$ python --version +Python 3.11.9 + +$ py --version +Python 3.13.2 + +$ python3 --version +Python was not found... +``` +In this case, use python throughout the remainder of the instructions. + +If none of the commands return Python 3.11.9: + +- Download and install Python 3.11.9 +- On Windows, be sure to check "Add Python to PATH" during installation +- Then re-run the checks above in a new terminal window + +If you're still stuck, ask the workshop organizers for help before proceeding. + +## 4. Installing the required packages{#packages} [Pip](https://pip.pypa.io/en/stable/) is the package management system built into Python. Pip should be available in your system once you installed Python successfully. Open a terminal (Mac/Linux) or Command Prompt (Windows) and run the following commands. -1. Create a [virtual environment](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments) called `dl_workshop`: +1. Create a [virtual environment](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments) called `venv` using the "venv" command: ::: spoiler ### On Linux/macOs ```shell -python3 -m venv dl_workshop +python3 -m venv venv # Use python or py instead if one of them points to 3.11.9. ``` ::: @@ -41,11 +90,21 @@ python3 -m venv dl_workshop ### On Windows ```shell -py -m venv dl_workshop +py -m venv venv # Use python3 or python instead if one of them points to 3.11.9. ``` ::: +If you run the `ls` command from `~/Desktop/dl_workshop`, you should see a new `venv` folder inside it + +```shell +ls +``` + +```output +venv/ +``` + 2. Activate the newly created virtual environment: ::: spoiler @@ -53,7 +112,7 @@ py -m venv dl_workshop ### On Linux/macOs ```shell -source dl_workshop/bin/activate +source venv/bin/activate ``` ::: @@ -63,21 +122,36 @@ source dl_workshop/bin/activate ### On Windows ```shell -dl_workshop\Scripts\activate +venv\Scripts\activate ``` +If you're using Git Bash on Windows, you need to add the source command first. + +```shell +source venv/Scripts/activate +``` ::: -Remember that you need to activate your environment every time you restart your terminal! +**Note**: Remember that you need to activate your environment every time you restart your terminal, and before you launch Jupyter Lab! + +3. Upgrade pip before installing other packages. This is a good practice to follow when you first initialize your virtual environment. + +```shell +python -m pip install --upgrade pip # remember: use python3 or py instead if it points to 3.11.9 +``` -3. Install the required packages: +4. Install the required packages: ::: spoiler ### On Linux/macOs + + + + ```shell -python3 -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot +python3 -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot # Use python or py instead if one of them points to 3.11.9. ``` Note for MacOS users: there is a package `tensorflow-metal` which accelerates the training of machine learning models with TensorFlow on a recent Mac with a Silicon chip (M1/M2/M3). @@ -89,8 +163,9 @@ However, the installation is currently broken in the most recent version (as of ### On Windows + ```shell -py -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot +py -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot # Use python or python3 instead if one of them points to 3.11.9. ``` ::: From 812de1016c43959fe3d79253b6b6deddaadf98c6 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Tue, 13 May 2025 12:28:17 -0500 Subject: [PATCH 02/17] Update setup.md --- learners/setup.md | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index 7c47037c..4f426e1b 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -2,7 +2,13 @@ title: Setup --- ## Setup -The setup steps below will help you prepare a folder for the workshop, install Python and the needed libraries in a virtual environment, and download the data we'll be exploring. +Please complete the setup at least a day in advance of the workshop. If you run into issues, contact the workshop organizers by email so you're ready to begin on time. + +The workshop setup steps below include: +1. Setup workshop folder +2. Install Python 3.11.9 +3. Setup virtual environment with required packages +4. Download the data ## 1. Setup workshop folder @@ -15,7 +21,9 @@ cd dl_workshop pwd ``` -::::::::::::::::::::::::::::::::::::::: discussion +```output +~/Desktop/dl_workshop +``` ## 2. Installing Python @@ -27,10 +35,8 @@ To install Python 3.11.9, go to the [official 3.11.9 downloads page](https://www Please set up your Python environment at least a day in advance of the workshop. If you run into issues with installation, contact the workshop organizers by email so you're ready to begin on time. -::::::::::::::::::::::::::::::::::::::::::::::::::: - -## 3. Determine which `python` command to use for downstream setup steps +### Determine which `python` command to use for downstream setup steps Different systems and Python installations (e.g., Anaconda, Git Bash, system Python, Windows Store, etc.) may register different command names. This quick check helps identify which one points to Python 3.11.9 on your machine. @@ -66,10 +72,7 @@ If none of the commands return Python 3.11.9: If you're still stuck, ask the workshop organizers for help before proceeding. -## 4. Installing the required packages{#packages} - -[Pip](https://pip.pypa.io/en/stable/) is the package management system built into Python. -Pip should be available in your system once you installed Python successfully. +## 3. Configure virtual environment Open a terminal (Mac/Linux) or Command Prompt (Windows) and run the following commands. @@ -134,7 +137,7 @@ source venv/Scripts/activate **Note**: Remember that you need to activate your environment every time you restart your terminal, and before you launch Jupyter Lab! -3. Upgrade pip before installing other packages. This is a good practice to follow when you first initialize your virtual environment. +3. Upgrade pip before installing other packages. This is a good practice to follow when you first initialize your virtual environment. [Pip](https://pip.pypa.io/en/stable/) is the package management system built into Python.Pip should be available in your system once you installed Python successfully. ```shell python -m pip install --upgrade pip # remember: use python3 or py instead if it points to 3.11.9 @@ -218,7 +221,7 @@ If a local installation does not work for you, it is also possible to run this l Alternatively you can use [Google colab](https://colab.research.google.com/). If you open a jupyter notebook here, the required packages are already pre-installed. Note that google colab uses jupyter notebook instead of Jupyter Lab. -## Downloading the required datasets +## 4. Downloading the required datasets Download the [weather dataset prediction csv][weatherdata] and [Dollar street dataset (4 files in total)][dollar-street] From 28c1d7ee23733df9928f64a3610b81f7dc380cbf Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Tue, 13 May 2025 13:12:12 -0500 Subject: [PATCH 03/17] Update setup.md --- learners/setup.md | 27 +++------------------------ 1 file changed, 3 insertions(+), 24 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index 4f426e1b..77c0c0f9 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -78,26 +78,10 @@ Open a terminal (Mac/Linux) or Command Prompt (Windows) and run the following co 1. Create a [virtual environment](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments) called `venv` using the "venv" command: -::: spoiler - -### On Linux/macOs - -```shell -python3 -m venv venv # Use python or py instead if one of them points to 3.11.9. -``` - -::: - -::: spoiler - -### On Windows - ```shell -py -m venv venv # Use python3 or python instead if one of them points to 3.11.9. +python -m venv venv # Use python3 or py instead if one of them points to 3.11.9. ``` -::: - If you run the `ls` command from `~/Desktop/dl_workshop`, you should see a new `venv` folder inside it ```shell @@ -149,10 +133,6 @@ python -m pip install --upgrade pip # remember: use python3 or py instead if it ### On Linux/macOs - - - - ```shell python3 -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot # Use python or py instead if one of them points to 3.11.9. ``` @@ -166,9 +146,8 @@ However, the installation is currently broken in the most recent version (as of ### On Windows - ```shell -py -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot # Use python or python3 instead if one of them points to 3.11.9. +python -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot # Use py or python3 instead if one of them points to 3.11.9. ``` ::: @@ -196,7 +175,7 @@ jupyter lab ## Check your setup To check whether all packages installed correctly, start a jupyter notebook in jupyter lab as -explained above. Run the following lines of code: +explained above (**with virtual environment activated**). Run the following lines of code: ```python import sklearn print('sklearn version: ', sklearn.__version__) From e2b394b780e9f88741a07183069daef11a09fd16 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Tue, 13 May 2025 13:20:10 -0500 Subject: [PATCH 04/17] Update setup.md --- learners/setup.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index 77c0c0f9..0a1e46bd 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -2,9 +2,8 @@ title: Setup --- ## Setup -Please complete the setup at least a day in advance of the workshop. If you run into issues, contact the workshop organizers by email so you're ready to begin on time. +Please complete the setup at least a day in advance of the workshop. If you run into issues, contact the workshop organizers by email so you're ready to begin on time. The setup steps include: -The workshop setup steps below include: 1. Setup workshop folder 2. Install Python 3.11.9 3. Setup virtual environment with required packages @@ -79,7 +78,8 @@ Open a terminal (Mac/Linux) or Command Prompt (Windows) and run the following co 1. Create a [virtual environment](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments) called `venv` using the "venv" command: ```shell -python -m venv venv # Use python3 or py instead if one of them points to 3.11.9. +# Use python3 or py instead if one of them points to 3.11.9. +python -m venv venv # 1st "venv" is commmand, 2nd venv is name of the virtual environment / folder ``` If you run the `ls` command from `~/Desktop/dl_workshop`, you should see a new `venv` folder inside it @@ -121,20 +121,24 @@ source venv/Scripts/activate **Note**: Remember that you need to activate your environment every time you restart your terminal, and before you launch Jupyter Lab! -3. Upgrade pip before installing other packages. This is a good practice to follow when you first initialize your virtual environment. [Pip](https://pip.pypa.io/en/stable/) is the package management system built into Python.Pip should be available in your system once you installed Python successfully. +3. After activating the enviornment, upgrade pip. This is a good practice to follow when you first initialize your virtual environment (beforing installing additional packages). [Pip](https://pip.pypa.io/en/stable/) is the package management system built into Python.Pip should be available in your system once you installed Python successfully. ```shell -python -m pip install --upgrade pip # remember: use python3 or py instead if it points to 3.11.9 + # remember: use python3 or py instead if it points to 3.11.9 +python -m pip install --upgrade pip ``` 4. Install the required packages: +Follow the OS-specific instructions below. NOte that It may take 10-20 minutes to install everything. + ::: spoiler ### On Linux/macOs ```shell -python3 -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot # Use python or py instead if one of them points to 3.11.9. + # Use python or py instead if one of them points to 3.11.9. +python3 -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot ``` Note for MacOS users: there is a package `tensorflow-metal` which accelerates the training of machine learning models with TensorFlow on a recent Mac with a Silicon chip (M1/M2/M3). @@ -147,7 +151,8 @@ However, the installation is currently broken in the most recent version (as of ### On Windows ```shell -python -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot # Use py or python3 instead if one of them points to 3.11.9. +# Use py or python3 instead if one of them points to 3.11.9. +python -m pip install jupyter seaborn scikit-learn pandas tensorflow pydot ``` ::: From 2ea33ddc2a28ab2d6afb98d882d446fe6711199e Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Tue, 13 May 2025 13:23:14 -0500 Subject: [PATCH 05/17] Update setup.md --- learners/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 0a1e46bd..5372aade 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -39,7 +39,7 @@ Please set up your Python environment at least a day in advance of the workshop. Different systems and Python installations (e.g., Anaconda, Git Bash, system Python, Windows Store, etc.) may register different command names. This quick check helps identify which one points to Python 3.11.9 on your machine. -Run the following in your terminal (Git Bash, Anaconda Prompt, or macOS/Linux shell): +Run the following in your terminal ([Git Bash](https://git-scm.com/downloads), Anaconda Prompt, or macOS/Linux shell): ```shell python --version From 5f45564b3d6a4fc4177de39aca80d2412e3c8d24 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Tue, 13 May 2025 13:30:13 -0500 Subject: [PATCH 06/17] Update setup.md --- learners/setup.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index 5372aade..e4da9d35 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -39,7 +39,7 @@ Please set up your Python environment at least a day in advance of the workshop. Different systems and Python installations (e.g., Anaconda, Git Bash, system Python, Windows Store, etc.) may register different command names. This quick check helps identify which one points to Python 3.11.9 on your machine. -Run the following in your terminal ([Git Bash](https://git-scm.com/downloads), Anaconda Prompt, or macOS/Linux shell): +Run the following in your terminal ([Git Bash recommended for Windows users](https://git-scm.com/downloads), Anaconda Prompt, or macOS/Linux shell): ```shell python --version @@ -65,8 +65,8 @@ In this case, use python throughout the remainder of the instructions. If none of the commands return Python 3.11.9: -- Download and install Python 3.11.9 -- On Windows, be sure to check "Add Python to PATH" during installation +- Download and install Python 3.11.9. On Windows, be sure to check "Add Python to PATH" during installation +- If you're on Windows using Anaconda Prompt, try using [Git Bash](https://git-scm.com/downloads) instead. - Then re-run the checks above in a new terminal window If you're still stuck, ask the workshop organizers for help before proceeding. From 4b355ffaa436d702fff47bec53a1daaf14a40b37 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Tue, 13 May 2025 13:41:05 -0500 Subject: [PATCH 07/17] Update setup.md --- learners/setup.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index e4da9d35..88f39d12 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -180,7 +180,17 @@ jupyter lab ## Check your setup To check whether all packages installed correctly, start a jupyter notebook in jupyter lab as -explained above (**with virtual environment activated**). Run the following lines of code: +explained above (**with virtual environment activated**). Run the following check to verify you have the right version of Python configured. +```python +!python --version +``` + +If you don't see 3.11.9, make sure your virtual environment was activated prior to launching Jupyter Lab. +```output +Python 3.11.9 +``` + +Then, run the following lines of code: ```python import sklearn print('sklearn version: ', sklearn.__version__) @@ -197,6 +207,7 @@ print('Tensorflow version: ', tensorflow.__version__) This should output the versions of all required packages without giving errors. Most versions will work fine with this lesson, but: + - For Keras and Tensorflow, the minimum version is 2.12.0 - For sklearn, the minimum version is 1.2.2 From 5b848b5ea923c5cf67867a064c17a3255b767e4b Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Tue, 13 May 2025 17:09:36 -0500 Subject: [PATCH 08/17] specify to move data to workshop folder --- learners/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 88f39d12..6f4fb425 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -218,7 +218,7 @@ Alternatively you can use [Google colab](https://colab.research.google.com/). If ## 4. Downloading the required datasets -Download the [weather dataset prediction csv][weatherdata] and [Dollar street dataset (4 files in total)][dollar-street] +Download the [weather dataset prediction csv][weatherdata] and [Dollar street dataset (4 files in total)][dollar-street]. Move these files to your workshop folder, `~/Desktop/dl_workshop/`. [dollar-street]: https://zenodo.org/api/records/10970014/files-archive [jupyter]: http://jupyter.org/ From 02097ba1bfb4d3beb7861d80275fc7c89f16b35a Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Wed, 14 May 2025 08:47:45 -0500 Subject: [PATCH 09/17] explicitly tell learners where to put all 5 data files (in data subfolder) --- learners/setup.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 6f4fb425..b0b546c4 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -218,7 +218,28 @@ Alternatively you can use [Google colab](https://colab.research.google.com/). If ## 4. Downloading the required datasets -Download the [weather dataset prediction csv][weatherdata] and [Dollar street dataset (4 files in total)][dollar-street]. Move these files to your workshop folder, `~/Desktop/dl_workshop/`. +Download the [weather dataset prediction csv][weatherdata] and [Dollar street dataset (4 files in total)][dollar-street]. Create a subfolder in your workshop folder called data, `~/Desktop/dl_workshop/data`, and move all 5 files to the data subfolder: + +- `dl_workshop/data/weather_prediction_dataset_light.csv` +- `dl_workshop/data/`train_labels.npy` +- `dl_workshop/data/`test_labels.npy` +- `dl_workshop/data/`train_images.npy` +- `dl_workshop/data/`test_images.npy` + +**Note**: If you end up using Google Colab for the workshop, you'll want to upload your `dl_workshop` folder to your Google drive. You can access files from Google drive using Colab and the following code: + +```python +from google.colab import drive +drive.mount('/content/drive') +``` + +A prompt will appear asking you to authorize access to your Google Drive. After authorization, your Drive will be accessible under `/content/drive/My Drive/`. You can use standard Python I/O or libraries like pandas, os, glob, etc. to interact with files. Example below: + +```python +import pandas as pd +# Load a CSV file from Drive +df = pd.read_csv('/content/drive/My Drive/dl_workshop/data/weather_prediction_dataset_light.csv') +``` [dollar-street]: https://zenodo.org/api/records/10970014/files-archive [jupyter]: http://jupyter.org/ From 7268d6954428391f45651e3935a2e1bc6c2a1539 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Wed, 14 May 2025 08:54:40 -0500 Subject: [PATCH 10/17] match setup (data stored in data subfolder) --- episodes/3-monitor-the-model.md | 2 +- episodes/4-advanced-layer-types.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/episodes/3-monitor-the-model.md b/episodes/3-monitor-the-model.md index 76232724..2040d284 100644 --- a/episodes/3-monitor-the-model.md +++ b/episodes/3-monitor-the-model.md @@ -72,7 +72,7 @@ into a local folder and load the data using the code below. ```python import pandas as pd -filename_data = "weather_prediction_dataset_light.csv" +filename_data = "data/weather_prediction_dataset_light.csv" data = pd.read_csv(filename_data) data.head() ``` diff --git a/episodes/4-advanced-layer-types.md b/episodes/4-advanced-layer-types.md index a09ba1c6..066919fa 100644 --- a/episodes/4-advanced-layer-types.md +++ b/episodes/4-advanced-layer-types.md @@ -35,7 +35,7 @@ The [MLCommons Dollar Street Dataset](https://www.kaggle.com/datasets/mlcommons/ import pathlib import numpy as np -DATA_FOLDER = pathlib.Path('data/dataset_dollarstreet/') # change to location where you stored the data +DATA_FOLDER = pathlib.Path('data/') # change to location where you stored the data train_images = np.load(DATA_FOLDER / 'train_images.npy') val_images = np.load(DATA_FOLDER / 'test_images.npy') train_labels = np.load(DATA_FOLDER / 'train_labels.npy') From 81128adcf0df3c8f727aff3ac5242d5f1e962a55 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Wed, 14 May 2025 08:58:12 -0500 Subject: [PATCH 11/17] fix typo from last commit --- learners/setup.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index b0b546c4..c6c6fa73 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -221,10 +221,10 @@ Alternatively you can use [Google colab](https://colab.research.google.com/). If Download the [weather dataset prediction csv][weatherdata] and [Dollar street dataset (4 files in total)][dollar-street]. Create a subfolder in your workshop folder called data, `~/Desktop/dl_workshop/data`, and move all 5 files to the data subfolder: - `dl_workshop/data/weather_prediction_dataset_light.csv` -- `dl_workshop/data/`train_labels.npy` -- `dl_workshop/data/`test_labels.npy` -- `dl_workshop/data/`train_images.npy` -- `dl_workshop/data/`test_images.npy` +- `dl_workshop/data/train_labels.npy` +- `dl_workshop/data/test_labels.npy` +- `dl_workshop/data/train_images.npy` +- `dl_workshop/data/test_images.npy` **Note**: If you end up using Google Colab for the workshop, you'll want to upload your `dl_workshop` folder to your Google drive. You can access files from Google drive using Colab and the following code: From a337f557fc1f96c8c4539c997379be192732f98b Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Wed, 14 May 2025 09:16:14 -0500 Subject: [PATCH 12/17] update data path to match setup --- episodes/5-transfer-learning.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/episodes/5-transfer-learning.md b/episodes/5-transfer-learning.md index ceba2709..0981c462 100644 --- a/episodes/5-transfer-learning.md +++ b/episodes/5-transfer-learning.md @@ -41,7 +41,7 @@ We load the data in the same way as the previous episode: import pathlib import numpy as np -DATA_FOLDER = pathlib.Path('data/dataset_dollarstreet/') # change to location where you stored the data +DATA_FOLDER = pathlib.Path('data/') # change to location where you stored the data train_images = np.load(DATA_FOLDER / 'train_images.npy') val_images = np.load(DATA_FOLDER / 'test_images.npy') train_labels = np.load(DATA_FOLDER / 'train_labels.npy') From c5ef5d7e1b1387707608fd44876e6fa88bb63729 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Wed, 14 May 2025 15:59:10 -0500 Subject: [PATCH 13/17] adjust headers (only core steps as 2nd level header) --- learners/setup.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index c6c6fa73..5e227d53 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -163,7 +163,7 @@ An [optional challenge in episode 2](episodes/2-keras.md) requires installation and instructions for doing that can be found [by following this link](https://graphviz.org/download/). -## Starting Jupyter Lab +### Starting Jupyter Lab We will teach using Python in [Jupyter Lab][jupyter], a programming environment that runs in a web browser. Jupyter Lab is compatible with Firefox, Chrome, Safari and Chromium-based browsers. @@ -178,7 +178,7 @@ and type the command: jupyter lab ``` -## Check your setup +### Check your virtual software setup To check whether all packages installed correctly, start a jupyter notebook in jupyter lab as explained above (**with virtual environment activated**). Run the following check to verify you have the right version of Python configured. ```python @@ -211,7 +211,7 @@ Most versions will work fine with this lesson, but: - For Keras and Tensorflow, the minimum version is 2.12.0 - For sklearn, the minimum version is 1.2.2 -## Fallback option: cloud environment +### Fallback option: cloud environment If a local installation does not work for you, it is also possible to run this lesson in [Binder Hub](https://mybinder.org/v2/gh/carpentries-incubator/deep-learning-intro/scaffolds). This should give you an environment with all the required software and data to run this lesson, nothing which is saved will be stored, please copy any files you want to keep. Note that if you are the first person to launch this in the last few days it can take several minutes to startup. The second person who loads it should find it loads in under a minute. Instructors who intend to use this option should start it themselves shortly before the workshop begins. Alternatively you can use [Google colab](https://colab.research.google.com/). If you open a jupyter notebook here, the required packages are already pre-installed. Note that google colab uses jupyter notebook instead of Jupyter Lab. From e6faa3c545513e33a04d4384cf2c8a1fadb98010 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Sun, 18 May 2025 14:40:14 -0500 Subject: [PATCH 14/17] avoid uploading venv note --- learners/setup.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 5e227d53..a5f5f192 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -226,7 +226,9 @@ Download the [weather dataset prediction csv][weatherdata] and [Dollar street da - `dl_workshop/data/train_images.npy` - `dl_workshop/data/test_images.npy` -**Note**: If you end up using Google Colab for the workshop, you'll want to upload your `dl_workshop` folder to your Google drive. You can access files from Google drive using Colab and the following code: +**Note**: If you end up using Google Colab for the workshop, you'll want to create a folder called `dl_workshop` within your Google drive, and then create the same `data` subfolder as above with all 5 files uploaded. **Avoid uploading the `venv` folder to your google drive**; you'll be using Colab's pre-built environment instead, and the `venv` folder contains MANY files from the libraries you installed. + +You can access files from Google drive using Colab and the following code: ```python from google.colab import drive @@ -241,6 +243,8 @@ import pandas as pd df = pd.read_csv('/content/drive/My Drive/dl_workshop/data/weather_prediction_dataset_light.csv') ``` +You can use the file navigator (folder icon) within Colab to help navigate your Google drive files. + [dollar-street]: https://zenodo.org/api/records/10970014/files-archive [jupyter]: http://jupyter.org/ [jupyter-install]: http://jupyter.readthedocs.io/en/latest/install.html#optional-for-experienced-python-developers-installing-jupyter-with-pip From 285e2d95b219479b2fbfd156dc22bedb58cd1ef5 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Sun, 18 May 2025 14:58:24 -0500 Subject: [PATCH 15/17] windows should use terminal as well. never use windows command prompt --- learners/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index a5f5f192..6b5f09ad 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -73,7 +73,7 @@ If you're still stuck, ask the workshop organizers for help before proceeding. ## 3. Configure virtual environment -Open a terminal (Mac/Linux) or Command Prompt (Windows) and run the following commands. +Open a terminal and run the following commands. 1. Create a [virtual environment](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments) called `venv` using the "venv" command: From 4ebb21dd4f4d3eda40da7130ee6bf29ce5770992 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Sun, 18 May 2025 15:09:05 -0500 Subject: [PATCH 16/17] bold activate env. --- learners/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 6b5f09ad..39098c9d 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -171,7 +171,7 @@ Note that Internet Explorer and Edge are *not* supported. See the [Jupyter Lab documentation](https://jupyterlab.readthedocs.io/en/latest/getting_started/accessibility.html#compatibility-with-browsers-and-assistive-technology) for an up-to-date list of supported browsers. To start Jupyter Lab, open a terminal (Mac/Linux) or Command Prompt (Windows), -make sure that you activated the virtual environment you created for this course, +**make sure that you activated the virtual environment you created for this course**, and type the command: ```shell From 0bb44abfcf1c3c429138463fd53e204fd1736501 Mon Sep 17 00:00:00 2001 From: Chris Endemann Date: Sun, 18 May 2025 15:16:12 -0500 Subject: [PATCH 17/17] add min version numbers as comments --- learners/setup.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index 39098c9d..f3d8febc 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -192,17 +192,17 @@ Python 3.11.9 Then, run the following lines of code: ```python +import tensorflow +print('Tensorflow version: ', tensorflow.__version__) # >= 2.12.0 + import sklearn -print('sklearn version: ', sklearn.__version__) +print('sklearn version: ', sklearn.__version__) # >= 1.2.2 import seaborn -print('seaborn version: ', seaborn.__version__) - +print('seaborn version: ', seaborn.__version__) # any version + import pandas -print('pandas version: ', pandas.__version__) - -import tensorflow -print('Tensorflow version: ', tensorflow.__version__) +print('pandas version: ', pandas.__version__) # any version ``` This should output the versions of all required packages without giving errors.