Enabling different linking options for MKL downstream. #2841
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using
candle/mkl
right now, it forces the use of static linking.However, this is not optimal in some circumstances, for instance we need to hotpatch the library to enable faster runtime on non-Intel CPU: https://github.com/huggingface/text-embeddings-inference/blob/main/Dockerfile#L39-L40
In order for the thing to be practical it's easier to use dynamic linking and use clever ordering to force patch the library at runtime.
I made the change non breaking.
mkl
becomes_mkl
and doesn't include the actualintel-mkl-src/mkl-static-lp64-iomp
feature, so it's agnostic the linking procedure.mkl
feature which solely addsmkl-static-lp64-iomp
to keep backward compatibility in examples and downstream cratescandle-nn
candle-transformers
(user can simply opt-out ofmkl
altogether since no code actually uses it)So now for power-users than want to custom link
mkl
all they need to do is enable the_mkl
feature, and manually enable theintel-mkl-src/mkl-dynamic-lp64-iomp
in their own crate/binary.I haven't modified the documentation since it's a power user feature anyway, but I can do it if deemed necessary.