Python: Add Intel Gaudi HPU Support to Semantic Kernel #11064
Conversation
markwallace-microsoft
added
python
github-actions
bot
changed the title
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
We're starting to move towards maintaining less files, so please put all this stuff into a single _gaudi.py file inside the hugging_face services folder.
|
|
||
|
|
||
| @dataclass | ||
| class Config: |
There was a problem hiding this comment.
I think it would be better to split this into, a settings setup, similar to what we use for a bunch of stuff, like OpenAISettings, this would be for thing like model_names and auth tokens, those can be passed in or read from the ENV. And other stuff seems like execution settings, but I guess more of this is needed in the constructor, correct? We also use pydantic for a lot of things, and considering what happens in the init, I would use that, so you can just call model_validate with kwargs in the setup.
| # Update config with model_kwargs if provided | ||
| if model_kwargs: | ||
| # Extract Gaudi-specific parameters from model_kwargs | ||
| for key, value in model_kwargs.items(): |
There was a problem hiding this comment.
leveraging pydantic features here would make this much simpler.
| from .config import Config | ||
|
|
||
| SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__)) | ||
| sys.path.append(os.path.dirname(SCRIPT_DIR)) |
| cd semantic-kernel | ||
|
|
||
| # Run the Gaudi installation script | ||
| ./install_for_gaudi.sh |
|
|
||
| ## Files | ||
|
|
||
| - `config.py`: Contains the `Config` dataclass that defines all configuration options |
There was a problem hiding this comment.
this seems ai generated and we do not use .py files for config anywhere!
|
|
||
| For optimal performance on Gaudi HPUs: | ||
|
|
||
| 1. Use the `optimum-habana` package (automatically installed with the `gaudi` extras) |
There was a problem hiding this comment.
add the install command
| **pipeline_kwargs or {}, | ||
| ) | ||
| resolved_device = f"cuda:{device}" if device >= 0 and torch.cuda.is_available() else "cpu" | ||
| if use_hpu: |
There was a problem hiding this comment.
considering the number of parameters, I think it would be better to allow people to pass in the pipeline, and then they can construct their own GaudiPipeline with the parameters of that, instead of this very hard to discover set of configs
There was a problem hiding this comment.
that has been something I wanted to do anyway, since that also gives extra capabilities for just HF settings.
Summary
This PR introduces support for Intel Gaudi HPUs (Habana Processing Units) to Semantic Kernel, enabling optimized AI workloads on specialized hardware.
Changes
install-gauditarget to handle installation on Gaudi HPU instances, excluding PyTorch.optimum-habanadependency for Gaudi optimization.config.py), a Gaudi-specific pipeline (pipeline.py), and helper functions (utils.py) to streamline integration and performance optimization.How to Test
Notes
Contribution Checklist