Dont set scale to zero as default when creating an Endpoint

[tomaarsen]

Hello!

Pull Request overview

• Dont set scale to zero as default when creating an Endpoint

Details

As discussed internally, this PR sets the scale_to_zero_timeout to None by default. It's not entirely clear to me if the min_replica/max_replica is also intended to be set to None from the discussion though - we can make those changes still.

Also, when someone specifies the scale_to_zero_timeout, we can help them out by overriding the min_replica to 0, but we can also do nothing as users will get an error anyways that the minReplica should not be 1.

I've also added an example of how to create an endpoint that scales to zero, and I quickly tested that running that example with the new scale_to_zero_timeout & min_replica defaults indeed creates an endpoint that doesn't scale to zero.

cc @Wauplin @Vaibhavs10 @ErikKaum

• Tom Aarsen

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.