Use binder -- can we run the big notebook?

[brian-rose]

Does the new Pythia binder resolve #3 ?

[github-actions]

👋 Thanks for opening this PR! The Cookbook will be automatically built with GitHub Actions. To see the status of your deployment, click below.
🔍 Git commit SHA: 32a5e1d
✅ Deployment Preview URL: https://ProjectPythia.github.io/cesm-lens-aws-cookbook/_preview/16

[brian-rose]

The build is erroring out with

asyncio.exceptions.TimeoutError

same as I found over at ProjectPythia/na-cordex-viz-cookbook#4

I guess there is a timeout limit specfied somewhere, I will investigate.

[ktyle]

It looks like some additional infrastructre is necessary in order to provide Dask capabilities. https://www.zonca.dev/posts/2022-04-04-dask-gateway-jupyterhub.html

[brian-rose]

Yes, the binderbot CLI has an input option --nb-timeout that defaults to 600 seconds.

We don't currently have an option to modify this default in our build workflow, but I will add this.

[brian-rose]

It looks like some additional infrastructre is necessary in order to provide Dask capabilities. https://www.zonca.dev/posts/2022-04-04-dask-gateway-jupyterhub.html

Both this notebook and the one in https://github.com/ProjectPythia/na-cordex-viz-cookbook run interactively on the Pythia binder, they just take a while.

Both notebooks use Dask but only with distributed.LocalCluster.

Using Dask Gateway to dynamically spin up big worker clusters would be a big asset, but it seems like there are some technical barriers to implementing this capability on JetStream2. We budgeted to develop this capability in our under-review NSF grant.

[brian-rose]

I'm going to try implementing a longer timeout on binderbot for this repo. I think it needs upward of 1 hour of wall time.

Assuming that works, I might change the "nightly build" to a "weekly build" -- this could be a good model for resource-intensive Cookbooks.

[brian-rose]

Fixed one obsolete bit of code that raised exceptions during interactive run.

Set a new flag to allow notebook execution up to one hour on the binder.

Changed the cron schedule for the nightly-build.yaml to weekly rather than nightly.

Let's see if this builds successfully!

[brian-rose]

If this works, then we can close #3

[brian-rose]

Yay, it works!

[brian-rose]

@ktyle @r-ford any comments before merging this?

The cookbook is now fully reproducing.