Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 38.9k 5.8k

2. bytedance/lightseq bytedance/lightseq Public

   LightSeq: A High Performance Library for Sequence Processing and Generation

   C++ 3.3k 332

3. deepspeedai/Megatron-DeepSpeed deepspeedai/Megatron-DeepSpeed Public

   Forked from NVIDIA/Megatron-LM

   Ongoing research training transformer language models at scale, including: BERT & GPT-2

   Python 2k 349

4. AniZpZ/AutoSmoothQuant AniZpZ/AutoSmoothQuant Public

   An easy-to-use package for implementing SmoothQuant for LLMs

   Python 92 7

5. QQQ QQQ Public

   QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.

   Python 102 9

6. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python