Skip to content

swift infer 推理结果 和 merge后模型的推理结果输出差距过大,swift infer有思考,merge后模型无法思考 #5196

@qy0720

Description

@qy0720

1、第一种方式可以正常推理
CUDA_VISIBLE_DEVICES=0 swift infer --adapters ./output/v2-20250731-152527/checkpoint-300 --stream true --merge_lora true --infer_backend vllm --vllm_max_model_len 8192 --temperature 0.7 --max_new_tokens 2048

2、swift export --adapters ./output/v2-20250731-152527/checkpoint-300 --merge_lora true

merge后的模型只能输出结果,无法思考

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions