model parallelism

[HUA9803]

I would like to ask when the model can support parallelism inference?

[zhuohan123]

Thanks for the question. All our models already supports tensor parallel execution. For example, if you have 2 GPUs, you can pass in argument --tesnor-parallel-size 2 or -tp 2. We will add documents on distributed execution (#206).