Bitblas supports FP8 Inference as well

[LeiWang1999]

Hello @mgoin, it's a pleasant surprise to discover this project. Thank you for your contributions to BitBLAS. We have recently added support for FP8 Matmul, hoping it will help this project.

[mgoin]

@LeiWang1999 Would you be able to share a minimal example for using FP8 W8A8? I would love to test this!

[LeiWang1999]

checkout this dude: https://github.com/microsoft/BitBLAS/blob/main/testing/python/operators/test_general_matmul_fp8.py

[LeiWang1999]

I'm recently working on quantizing an FP8 W/A language model, this project has been a great help to me, thanks :)