Remove deprecated rcp-caas-test cluster (#11)

* Remove deprecated rcp-caas-test cluster

* Change references to rcp-prod to simply rcp

Author: JulienVig

Diff for: README.md

@@ -43,7 +43,7 @@ The step-by-step instructions for first time users to quickly get a job running.
 
 > [!TIP] 
 > After completing the setup, the **TL;DR** of the interaction with the cluster (using the scripts in this repo) is:
-> * Choose a cluster and just run the command to set it up: `ic-cluster`, `rcp-cluster`, or `rcp-cluster-prod`
+> * Choose a cluster and just run the command to set it up: `ic-cluster` or `rcp-cluster`
 > 
 > * Get a running job with one GPU that is reserved for you: `python csub.py -n sandbox`
 > 
@@ -96,18 +96,12 @@ curl -o  ~/.kube/config https://raw.githubusercontent.com/epfml/getting-started/
 # Sketch for macOS with Apple Silicon
 # Download the CLI from the link shown in the help section.
 # for Linux: replace `darwin` with `linux`
-wget --content-disposition https://rcp-caas-test.rcp.epfl.ch/cli/darwin
+wget --content-disposition https://rcp-caas-prod.rcp.epfl.ch/cli/darwin
 # Give it the right permissions and move it.
 chmod +x ./runai
 sudo mv ./runai /usr/local/bin/runai-rcp
 sudo chown root: /usr/local/bin/runai-rcp
 
-# Repeat for RCP Prod Cluster  
-wget --content-disposition https://rcp-caas-prod.rcp.epfl.ch/cli/darwin
-chmod +x ./runai
-sudo mv ./runai /usr/local/bin/runai-rcp-prod
-sudo chown root: /usr/local/bin/runai-rcp-prod
-
 # Repeat for IC Cluster
 # for Linux: replace `macos` with `linux`
 wget --content-disposition https://go.epfl.ch/iccluster-runai-macos
@@ -128,7 +122,7 @@ runai-ic list projects
 # Put default project
 runai-ic config project mlo-$GASPAR_USERNAME
 # Repeat for the RCP cluster
-runai-rcp config cluster rcp-caas-test
+runai-rcp config cluster rcp-caas
 runai-rcp login
 runai-rcp list projects
 runai-rcp config project mlo-$GASPAR_USERNAME
@@ -151,11 +145,10 @@ source ~/.zshrc
 # Let's use the normal RCP cluster
 rcp-cluster
 # Try to submit a job that mounts our shared storage and see its content.
-# (side note: on the new rcp-prod, the pvc is called mlo-scratch, so the arg below has to be changed)
 runai submit \
   --name setup-test-storage \
   --image ubuntu \
-  --pvc runai-mlo-$GASPAR_USERNAME-scratch:/mloscratch \
+  --pvc mlo-scratch:/mloscratch \
   -- ls -la /mloscratch/homes
 # Check the status of the job
 runai describe job setup-test-storage
@@ -235,7 +228,7 @@ For remote development (changing code, debugging, etc.), we recommend using VSCo
 >
 > To have a job that can run in the background, do `python csub.py -n sandbox --train --command "cd /mloscratch/homes/<your username>/<your code>; python main.py "`
 >
->  There are differences between the clusters of IC and RCP, which require different tool versions (`runai-ic`, `runai-rcp`, ...). Since this is a bit of a hassle, we made it easy to switch between the clusters via the commands `ic-cluster`, `rcp-cluster` and `rcp-cluster-prod`. To make sure you're aware of the cluster you're using, the `csub` script asks you to set the cluster to use before submitting a job: `python csub.py -n sandbox --cluster ic-caas` (choosing between `["rcp-caas-test", "ic-caas", "rcp-caas-prod"]`). It only works when the cluster argument matches your currently chosen cluster. 
+>  There are differences between the clusters of IC and RCP, which require different tool versions (`runai-ic`, `runai-rcp`, ...). Since this is a bit of a hassle, we made it easy to switch between the clusters via the commands `ic-cluster` and `rcp-cluster`. To make sure you're aware of the cluster you're using, the `csub` script asks you to set the cluster to use before submitting a job: `python csub.py -n sandbox --cluster ic-caas` (choosing between `["ic-caas", "rcp-caas"]`). It only works when the cluster argument matches your currently chosen cluster. 
 
 You're good to go now! :) It's up to you to customize your environment and install the packages you need. Read up on the rest of this README to learn more about the cluster and the scripts.
 
@@ -338,7 +331,7 @@ The python script `csub.py` is a wrapper around the run:ai CLI that makes it eas
 General usage:
 
 ```bash
-python csub.py --n <job_name> -g <number of GPUs> -t <time> --cluster rcp-caas-test -i ic-registry.epfl.ch/mlo/mlo:v1 --command <cmd> [--train]
+python csub.py --n <job_name> -g <number of GPUs> -t <time> --cluster rcp-caas -i ic-registry.epfl.ch/mlo/mlo:v1 --command <cmd> [--train]
 ```
 Check the arguments for the script to see what they do.
 
@@ -382,7 +375,7 @@ kubectl port-forward <pod_name> 8888:8888
 ```
 
 ## Distributed training
-Newer versions of runai support distributed training, meaning the ability to use run accross multiple compute nodes, even beyond the several GPUs available on one node. This is currently set up on the new RCP Prod cluster (rcp-caas-prod).
+Newer versions of runai support distributed training, meaning the ability to use run accross multiple compute nodes, even beyond the several GPUs available on one node. This is currently set up on the new RCP Prod cluster (rcp-caas).
 A nice [documentation to get started with distributed jobs is available here](docs/multinode.md).
 
 # File overview of this repository

Diff for: csub.py

@@ -21,8 +21,8 @@
     "-cl",
     "--cluster",
     type=str,
-    default="rcp-caas-test",
-    choices=["rcp-caas-test", "ic-caas", "rcp-caas-prod"],
+    default="rcp-caas",
+    choices=["ic-caas", "rcp-caas"],
 )
 parser.add_argument(
     "-c",
@@ -147,11 +147,10 @@
         text=True,
     ).stdout.strip()
 
-    if current_cluster == "rcp-caas-prod":
+    if current_cluster == "rcp-caas":
+        # the latest version can be found on https://wiki.rcp.epfl.ch/home/CaaS/FAQ/how-to-prepare-environment
         runai_cli_version = "2.16.70"
         scratch_name = "mlo-scratch"
-    elif current_cluster == "rcp-caas-test":
-        runai_cli_version = "2.9.25"
     elif current_cluster == "ic-caas":
         runai_cli_version = "2.16.52"
     assert (

Diff for: kubeconfig.yaml

@@ -8,11 +8,6 @@ clusters:
     cluster:
       server: https://caas-prod.rcp.epfl.ch:443
       certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJRHdwSElpTmQrVUV3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TkRBMk1UUXdPVFU1TWpkYUZ3MHpOREEyTVRJeE1EQTBNamRhTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUURvOHJDRjNjeXdRRTlxTVpEOHNGTXo2K0FzSEpnWi81WVNwMGNhWHNKd0JWUERneGdwRGZKY0hnYXYKS2tOdVhTNGpBN1VrZkg1amZXQitvdytpamN3OUR4cjV6STB2TUNReWtzYk9kMVFFMis0Q0J1U0JXU01Gc1pYZQp2T01SanltN056SytxWkVldHpxR0M0bU5LdU9qbC92cGd4ZDNuM2Y2L3loRHhockp2bkVWKzZlUE5icWpDZURZCld1VWFZdUYxRmM4QnZHN0hma3FYRlRWWVdlNkpNa3JSbDQxOVo5a2diNnIvUFNZVzZqdDhhNThTSGNHSVhnTFcKOTBta3BFb1JCMENOSG0wQllEQjdjNFJxMmdyaWtZTUlldGM0eXk2L3NSdFp6NzFiTUQrM2ZDNk92NDdvOXUzWgpld0VWeEJ4dG11ZkVvVGduVEVyNXFYMlhxWFZMQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJSazdCMm84a3cxcyt0Ny9ZaGxmV1h1MnR6TkdEQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQXFOdnQrR01lTwp6QnZZZEQ2SExCakFVeWc1czd0TDgzOVltd0RhRXBseG45ZlBRdUV6UW14cnEwUEoxcnVZNnRvRks1SEN4RFVzCmJDN3R3WlMzaVdNNXQ5NEJveHJGVC92c3QrQmtzbWdvTGM2T0N1MitYcngyMUg3UnFLTnNVR01LN2tFdGN6cHgKeXUrYTB6T0tISEUxNWFSVENPbklzQ1pXaTRhVFhIZ00zQ2U4VEhBMXRxaW9pREFHMVFUQXNhNXhTeVM3RWlUSQpDYi9xbktPRlVvM3V3bkRocWljRTU3dE1LTjliRE8rV3hNMzVxT2lBZXVXOUVnc2JlOFA5aDY2NG1tK1QzbjY0ClJNL1l1NHhmcDZwMHMvdGZyZTVjaUFvT0dGekYyRmVKek5PYm1vRkVseUtKc0RwbEorcWFTVXlaL2NtNWRIYUUKQVUxOVMrUWpFc1cvCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
-  # Cluster RCP Test
-  - name: caas-test.rcp.epfl.ch
-    cluster:
-      server: https://caas-test.rcp.epfl.ch:443
-      certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJek1EUXlOakE0TVRRME5sb1hEVE16TURReU16QTRNVFEwTmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTFAxCmtTZ2E4NWRWU0p0VUxGQ1g5VWo1K1lTT2dCbG9MZGVxZVgrM1ByVGtQZkptWFBxeXlsVVBLN0tJUWlvSUplNm8KRTBaS2JZbU03SnEvL0lPaHF4R0VraUNrTHJCamJrYXF5M3NibkNhWGFMa1pQYkhNWjgwdmlMMGNFZHNJTWN4WgozdHpMTzFNTldwZW9mZlJ6L1NvbXpqSTVDQldJbUptTmhvZXpJQUVNOGJuaDJKeFBFNzRwWThTS1BTRk5YVzN0CjgxNmM5cXRvc1lJQjVrTnh1UjRGWVh5bGloZHZ3UmVqVW9wajA2ME1rSkl3QmpXM01YTFUrdkVyandKeFc5Q1cKZ2plUndzOG5kdW5VVHREcy9CVjhGbW5JZy81VVNhZTBzUE5FQWxvZC9TbGhrMnNuWTJvUXZlTHpFNkhrMnluRgpHNXd1VGVXRDZGY2Erd1pNMjM4Q0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZNVVhkVWVnK2xMdTlHWElMQ2VlOVJzOENmUXpNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBR051a2ZUR3E0RTlrckkreVZQbApaem1reSszaUNTMnYvTU9OU3h0S01idWZ2V0ROZFM3QzZaK1RDQTJSd0c1Y2gzZUh5UW9oTSs0K2wrSTJxMTFwCjNJVGRxYVI4RDhpQkFCbXV6Yzl2a3BKanZTTzZ4VVpnTFJZMHRDTUxXZ3g2b2tBcWhxZDV3YTZIYmN6Z1QrSUcKQlVGbERtR0R4K0MxTnFIYVFKUVN1bENqL1ZyS1RROVFlY1NoZGZqVDgvS1NVUjQ4VTlEdlA3dnU0YkRnWW5DKwpoOXEwUlFpUGR4TEtlL2Q5aGd0UnM5TjFQdGRYZXAxdHB3NCs3Y3N4TE1DSXNmYTBwaW8yb3lEems0bTNjSWRNCi9iNElHUEZaM2hYZktOVGtybnUrWmdCUms5Yjk3emNKZVdhendxTXUyd1dkV2JiQjdpaU5ZK2xtWkl1S0dUeFQKWWpRPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
   # Cluster IC
   - name: ic-caas
     cluster:
@@ -44,14 +39,10 @@ users:
         name: oidc
 contexts:
   # Contexts (a context a cluster associated with a user)
-  - name: rcp-caas-prod
+  - name: rcp-caas
     context:
       cluster: caas-prod.rcp.epfl.ch
       user: runai-rcp-authenticated-user
-  - name: rcp-caas-test
-    context:
-      cluster: caas-test.rcp.epfl.ch
-      user: runai-rcp-authenticated-user
   - name: ic-caas
     context:
       cluster: ic-caas

Diff for: template/cluster_switch.sh

@@ -4,16 +4,10 @@ function ic-cluster {
     runai config cluster ic-caas
     runai config project mlo-$GASPAR_USERNAME
 }
+
 function rcp-cluster {
     alias runai=runai-rcp
     # This is actually changing the context not the cluster ...
-    runai config cluster rcp-caas-test
-    runai config project mlo-$GASPAR_USERNAME
-}
-
-function rcp-prod-cluster {
-    alias runai=runai-rcp-prod
-    # This is actually changing the context not the cluster ...
-    runai config cluster rcp-caas-prod
+    runai config cluster rcp-caas
     runai config project mlo-$GASPAR_USERNAME
 }