gpu环境并发请求：RuntimeError: could not execute a primitive

[June-Li]

Description of the bug | 错误描述

2025-03-10 19:35:58 - ERROR - OCRApiDeploy2.py:upload_ocr_result:217 - Traceback (most recent call last):
File "/workspace/JuneLi/bbtv/cube-ocr/WORKFLOW/OTHER/OCR/v0/OCRApiDeploy2.py", line 161, in upload_ocr_result
result = op(
File "/workspace/JuneLi/bbtv/cube-ocr/WORKFLOW/OTHER/OCR/v0/OCRModelDeploy.py", line 356, in call
sd_hypertxt = self.get_result(file_path, force_cv=force_cv, lang=lang)
File "/workspace/JuneLi/bbtv/cube-ocr/WORKFLOW/OTHER/OCR/v0/OCRModelDeploy.py", line 321, in get_result
infer_result = ds.apply(doc_analyze, ocr=True, lang=ds._lang)
File "/root/.cache/pypoetry/virtualenvs/cube-ocr-CKM964-4-py3.10/lib/python3.10/site-packages/magic_pdf/data/dataset.py", line 212, in apply
return proc(self, *args, **kwargs)
File "/root/.cache/pypoetry/virtualenvs/cube-ocr-CKM964-4-py3.10/lib/python3.10/site-packages/magic_pdf/model/doc_analyze_by_custom_model.py", line 197, in doc_analyze
analyze_result = batch_model(images)
File "/root/.cache/pypoetry/virtualenvs/cube-ocr-CKM964-4-py3.10/lib/python3.10/site-packages/magic_pdf/model/batch_analyze.py", line 134, in call
ocr_res = self.model.ocr_model.ocr(
File "/root/.cache/pypoetry/virtualenvs/cube-ocr-CKM964-4-py3.10/lib/python3.10/site-packages/magic_pdf/model/sub_modules/ocr/paddleocr/ppocr_273_mod.py", line 85, in ocr
dt_boxes, rec_res, _ = self.call(img, cls, mfd_res=mfd_res)
File "/root/.cache/pypoetry/virtualenvs/cube-ocr-CKM964-4-py3.10/lib/python3.10/site-packages/magic_pdf/model/sub_modules/ocr/paddleocr/ppocr_273_mod.py", line 148, in call
dt_boxes, elapse = self.text_detector(img)
File "/root/.cache/pypoetry/virtualenvs/cube-ocr-CKM964-4-py3.10/lib/python3.10/site-packages/paddleocr/tools/infer/predict_det.py", line 396, in call
dt_boxes, elapse = self.predict(img)
File "/root/.cache/pypoetry/virtualenvs/cube-ocr-CKM964-4-py3.10/lib/python3.10/site-packages/paddleocr/tools/infer/predict_det.py", line 255, in predict
self.predictor.run()
RuntimeError: could not execute a primitive

How to reproduce the bug | 如何复现

flaskapi或者fastapi部署服务后，同时发两个请求就会出现以上报错

Operating system | 操作系统

Linux

Python version | Python 版本

3.10

Software version | 软件版本 (magic-pdf --version)

1.0.x

Device mode | 设备模式

cuda

[ubbiiy]

你是不是指定了flask用多进程来跑？需要在 if name == "main"那里加这一句，不然子进程cuda初始化会失败
multiprocessing . set_start_method ( 'spawn' )

[June-Li]

你是不是指定了flask用多进程来跑？需要在 if name == "main"那里加这一句，不然子进程cuda初始化会失败 multiprocessing . set_start_method ( 'spawn' )

没有用多进程；
我用的fastapi（flaskapi也试过，问题一样）；
这是fastapi中的代码片段，如果保留async，他其实是单线程（不报错），因为你同时发两个请求，只有一个在处理并且会阻塞另一个请求，也就是另一个请求在排队，如果把async删掉的话，就可以多线程并发处理了，但是会报错，这个报错是从paddleocr传出来的。但是我们的服务是需要多线程的，不能阻塞在那；
@app.post(
"/pdf_parse",
tags=["projects"],
summary="Parse PDF files (supports local files and S3)",
)
async def pdf_parse(
pdf_file: UploadFile = None,
pdf_path: str = None,
parse_method: str = "auto",
is_json_md_dump: bool = False,
output_dir: str = "output",
return_layout: bool = False,
return_info: bool = False,
return_content_list: bool = False,
return_images: bool = False,
):

[myhloli]

paddle框架不支持多线程，不可并发，目前并发需求只能通过多进程来解决

[June-Li]

paddle框架不支持多线程，不可并发，目前并发需求只能通过多进程来解决

但是多进行每次都会重新加载模型，会很费时，而且多进程每个进程都会加载模型，也会很费显存；
多进程下耗时和耗内存的问题可以解决吗？

[myhloli]

要么空间换时间，要么时间换空间，要么换掉paddle

[June-Li]

要么空间换时间，要么时间换空间，要么换掉paddle

多谢

[mauryaland]

I have the same problem with fastapi, could I ask what is the way to fix it? I don't really get it with the translation from google.