Lightweight installation: use safetensors without torch

[hedgeho]

What does this PR do?

This PR removes unnecessary imports of torch from model load/save code to enable lightweight installation (without torch) as described here in docs.

It achieves this mainly by replacing calls to safetensors.torch.[save|load]_file() with safetensors.numpy.[save|load]_file(), and removing other calls to torch methods.

Fixes #2304

Before submitting

• This PR fixes a typo or improves the docs (if yes, ignore all other checks!).
• Did you read the contributor guideline?
• Was this discussed/approved via a Github issue? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes (if applicable)?
• Did you write any new necessary tests?

[hedgeho]

Should work now. Tested saving&loading locally with environment without torch.

Tested also loading of a model that was saved with stable version, the model behaved exactly the same.

[MaartenGr]

Thanks for the PR, it is greatly appreciated. Just did some tests and it is working nicely for me. The code is looking great, so no comments there.