-
-
Notifications
You must be signed in to change notification settings - Fork 90
Open
Labels
chart/daskRelated to the dask chartRelated to the dask chart
Description
I'm using Dask to coordinate some ML jobs, which have internal parallelism. I'd like to restrict my cluster so that only one job can run on a node at a time, even though nodes have multiple CPUs. However, this is currently impossible, since the Kubernetes worker resource limit is used to customize --nthreads
:
https://github.com/dask/helm-chart/blob/master/dask/templates/dask-worker-deployment.yaml#L37-L38
This also prevents giving nodes e.g. 0.5 or 1.5 cpus, because even though Kubernetes allows it Dask will get confused and crash (i.e. --nthreads
fails to parse as an integer.)
This is moderately esoteric so if I should just fork the chart and use that I don't mind.
Metadata
Metadata
Assignees
Labels
chart/daskRelated to the dask chartRelated to the dask chart