Cannot run on aarch64 Linux

[SuperCowProducts]

System Info / 系統信息

Host OS: Asahi ALARM
Hardware: M2 Macbook Air (16GB)

(.venv) root@80a09d6d82c7:/# uv --version
uv 0.11.26 (aarch64-unknown-linux-gnu)
(.venv) root@80a09d6d82c7:/# python --version
Python 3.12.13
(.venv) root@80a09d6d82c7:/# uv pip list
Package                                  Version
---------------------------------------- -------------
accelerate                               1.14.0
aiohappyeyeballs                         2.7.1
aiohttp                                  3.14.1
aiosignal                                1.4.0
annotated-doc                            0.0.4
annotated-types                          0.7.0
anthropic                                0.116.0
anyio                                    4.14.1
apache-tvm-ffi                           0.1.12
astor                                    0.8.1
attrs                                    26.1.0
blake3                                   1.0.9
cachetools                               7.1.4
cbor2                                    6.1.3
certifi                                  2026.6.17
cffi                                     2.0.0
charset-normalizer                       3.4.7
click                                    8.4.2
cloudpickle                              3.1.2
compressed-tensors                       0.17.0
cryptography                             49.0.0
depyf                                    0.20.0
detect-installer                         0.1.0
dill                                     0.4.1
diskcache                                5.6.3
distro                                   1.9.0
dnspython                                2.8.0
docstring-parser                         0.18.0
einops                                   0.8.2
email-validator                          2.3.0
fastapi                                  0.136.3
fastapi-cli                              0.0.28
fastapi-cloud-cli                        0.22.1
fastar                                   0.11.0
filelock                                 3.29.5
frozenlist                               1.8.0
fsspec                                   2026.6.0
glmocr                                   0.1.5
googleapis-common-protos                 1.75.0
grpcio                                   1.81.1
h11                                      0.16.0
hf-xet                                   1.5.1
httpcore                                 1.0.9
httptools                                0.8.0
httpx                                    0.28.1
httpx-sse                                0.4.3
huggingface-hub                          1.22.0
idna                                     3.18
ijson                                    3.5.0
interegular                              0.3.3
jinja2                                   3.1.6
jiter                                    0.16.0
jmespath                                 1.1.0
jsonschema                               4.26.0
jsonschema-specifications                2025.9.1
lark                                     1.2.2
llguidance                               1.7.6
llvmlite                                 0.47.0
lm-format-enforcer                       0.11.3
loguru                                   0.7.3
markdown-it-py                           4.2.0
markupsafe                               3.0.3
mcp                                      1.28.1
mdurl                                    0.1.2
mistral-common                           1.11.5
model-hosting-container-standards        0.1.16
mpmath                                   1.3.0
msgspec                                  0.21.1
multidict                                6.7.1
networkx                                 3.6.1
ninja                                    1.13.0
numba                                    0.65.0
numpy                                    2.3.5
openai                                   2.44.0
openai-harmony                           0.0.8
opencv-python-headless                   5.0.0.93
opentelemetry-api                        1.43.0
opentelemetry-exporter-otlp              1.43.0
opentelemetry-exporter-otlp-proto-common 1.43.0
opentelemetry-exporter-otlp-proto-grpc   1.43.0
opentelemetry-exporter-otlp-proto-http   1.43.0
opentelemetry-proto                      1.43.0
opentelemetry-sdk                        1.43.0
opentelemetry-semantic-conventions       0.64b0
opentelemetry-semantic-conventions-ai    0.5.1
outlines-core                            0.2.14
packaging                                26.2
partial-json-parser                      0.2.1.1.post7
pillow                                   12.3.0
pip                                      26.1.2
portalocker                              3.2.0
prometheus-client                        0.25.0
prometheus-fastapi-instrumentator        8.0.2
propcache                                0.5.2
protobuf                                 7.35.1
psutil                                   7.2.2
py-cpuinfo                               9.0.0
pybase64                                 1.4.3
pycountry                                26.2.16
pycparser                                3.0
pydantic                                 2.13.4
pydantic-core                            2.46.4
pydantic-extra-types                     2.11.1
pydantic-settings                        2.14.2
pygments                                 2.20.0
pyjwt                                    2.13.0
pymupdf                                  1.28.0
pypdfium2                                5.11.0
python-dotenv                            1.2.2
python-json-logger                       4.1.0
python-multipart                         0.0.32
pyyaml                                   6.0.3
pyzmq                                    27.1.0
referencing                              0.37.0
regex                                    2026.6.28
requests                                 2.34.2
rich                                     15.0.0
rich-toolkit                             0.20.1
rignore                                  0.7.6
rpds-py                                  2026.6.3
safetensors                              0.8.0
sentencepiece                            0.2.1
sentry-sdk                               2.64.0
setproctitle                             1.3.7
setuptools                               77.0.3
shellingham                              1.5.4
six                                      1.17.0
sniffio                                  1.3.1
sse-starlette                            3.4.5
starlette                                1.3.1
supervisor                               4.3.0
sympy                                    1.14.0
tiktoken                                 0.13.0
tokenizers                               0.22.2
torch                                    2.11.0+cpu
torchaudio                               2.11.0+cpu
torchvision                              0.26.0+cpu
tqdm                                     4.68.3
transformers                             5.13.0
typer                                    0.26.8
typing-extensions                        4.16.0
typing-inspection                        0.4.2
urllib3                                  2.7.0
uvicorn                                  0.50.0
uvloop                                   0.22.1
vllm                                     0.24.0+cpu
watchfiles                               1.2.0
websockets                               16.0
xgrammar                                 0.2.3
yarl                                     1.24.2
(.venv) root@80a09d6d82c7:/# cat /etc/os-release 
PRETTY_NAME="Ubuntu 24.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.4 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
(.venv) root@80a09d6d82c7:/# 

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

sudo docker run -it --name vM ubuntu:24.04 bin/bash
export TZ="Europe/Rome"
apt update
apt upgrade -y
apt install -y --no-install-recommends libtcmalloc-minimal4
find / -iname *libtcmalloc_minimal.so.4
TC_PATH=/usr/lib/aarch64-linux-gnu/libtcmalloc_minimal.so.4
export LD_PRELOAD="$TC_PATH:$LD_PRELOAD"
apt install -y pipx
pipx install uv
pipx ensurepat
source ~/.bashrc
uv --version
uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate
apt install -y wget
wget https://github.com/vllm-project/vllm/releases/download/v0.24.0/vllm-0.24.0+cpu-cp38-abi3-manylinux_2_34_aarch64.whl 
uv pip install --torch-backend cpu vllm-0.24.0+cpu-cp38-abi3-manylinux_2_34_aarch64.whl 
uv pip install "glmocr[selfhosted]"
uv pip install "transformers>=5.3.0"
vllm serve zai-org/GLM-OCR  --port 8080 --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' --served-model-name glm-ocr

(.venv) root@80a09d6d82c7:/# vllm serve zai-org/GLM-OCR  --port 8080 --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' --served-model-name glm-ocr
INFO 07-05 14:07:05 [importing.py:81] Triton not installed or not compatible; certain GPU-related functions will not be available.
(APIServer pid=408) INFO 07-05 14:07:05 [api_utils.py:339] 
(APIServer pid=408) INFO 07-05 14:07:05 [api_utils.py:339]        █     █     █▄   ▄█
(APIServer pid=408) INFO 07-05 14:07:05 [api_utils.py:339]  ▄▄ ▄█ █     █     █ ▀▄▀ █  version 0.24.0
(APIServer pid=408) INFO 07-05 14:07:05 [api_utils.py:339]   █▄█▀ █     █     █     █  model   zai-org/GLM-OCR
(APIServer pid=408) INFO 07-05 14:07:05 [api_utils.py:339]    ▀▀  ▀▀▀▀▀ ▀▀▀▀▀ ▀     ▀
(APIServer pid=408) INFO 07-05 14:07:05 [api_utils.py:339] 
(APIServer pid=408) INFO 07-05 14:07:05 [api_utils.py:273] non-default args: {'model_tag': 'zai-org/GLM-OCR', 'port': 8080, 'model': 'zai-org/GLM-OCR', 'served_model_name': ['glm-ocr'], 'speculative_config': {'method': 'mtp', 'num_speculative_tokens': 3}}
(APIServer pid=408) Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
(APIServer pid=408) INFO 07-05 14:07:07 [model.py:598] Resolved architecture: GlmOcrForConditionalGeneration
(APIServer pid=408) INFO 07-05 14:07:07 [model.py:1725] Using max model len 131072
(APIServer pid=408) INFO 07-05 14:07:09 [model.py:598] Resolved architecture: GlmOcrMTPModel
(APIServer pid=408) INFO 07-05 14:07:09 [model.py:1725] Using max model len 131072
(APIServer pid=408) WARNING 07-05 14:07:09 [speculative.py:761] Enabling num_speculative_tokens > 1 will run multiple times of forward on same MTP layer,which may result in lower acceptance rate
(APIServer pid=408) INFO 07-05 14:07:09 [vllm.py:1006] Asynchronous scheduling is enabled.
(APIServer pid=408) INFO 07-05 14:07:09 [kernel.py:276] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=408) WARNING 07-05 14:07:09 [vllm.py:1614] max_num_scheduled_tokens is set to 2048 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens or max_num_seqs.
(APIServer pid=408) WARNING 07-05 14:07:11 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
(APIServer pid=408) WARNING 07-05 14:07:11 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
(APIServer pid=408) WARNING 07-05 14:07:11 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
(APIServer pid=408) WARNING 07-05 14:07:11 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
INFO 07-05 14:07:26 [importing.py:81] Triton not installed or not compatible; certain GPU-related functions will not be available.
(EngineCore pid=440) INFO 07-05 14:07:27 [core.py:114] Initializing a V1 LLM engine (v0.24.0) with config: model='zai-org/GLM-OCR', speculative_config=SpeculativeConfig(method='mtp', model='zai-org/GLM-OCR', num_spec_tokens=3), tokenizer='zai-org/GLM-OCR', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=131072, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=True, quantization=None, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cpu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_verbose=False), seed=0, served_model_name=glm-ocr, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.DYNAMO_TRACE_ONCE: 2>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none', '+gelu'], 'ir_enable_torch_wrap': False, 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': None, 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'dce': True, 'nan_asserts': False, 'epilogue_fusion': True, 'cpp.dynamic_threads': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, moe_backend='auto', linear_backend='auto')
(EngineCore pid=440) INFO 07-05 14:07:27 [multiproc_executor.py:140] DP group leader: node_rank=0, node_rank_within_dp=0, master_addr=127.0.0.1, mq_connect_ip=172.17.0.2 (local), world_size=1, local_world_size=1
(EngineCore pid=440) INFO 07-05 14:07:27 [ompmultiprocessing.py:185] OpenMP thread binding info: 
(EngineCore pid=440) INFO 07-05 14:07:27 [ompmultiprocessing.py:185] 	VLLM_CPU_OMP_THREADS_BIND='auto', auto_setup=True, skip_setup=False
(EngineCore pid=440) INFO 07-05 14:07:27 [ompmultiprocessing.py:185] 	local_world_size=1, reserve_cpu_num=1
(EngineCore pid=440) INFO 07-05 14:07:27 [ompmultiprocessing.py:185] 	local_rank=0, core ids=[0, 1, 2, 3, 4, 5, 6]
(EngineCore pid=440) INFO 07-05 14:07:27 [ompmultiprocessing.py:185] 	reserved_cpus=[7]
INFO 07-05 14:07:31 [importing.py:81] Triton not installed or not compatible; certain GPU-related functions will not be available.
WARNING 07-05 14:07:32 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
WARNING 07-05 14:07:32 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
WARNING 07-05 14:07:32 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
WARNING 07-05 14:07:32 [cpu.py:453] Failed to import from vllm._C: ImportError('libnuma.so.1: cannot open shared object file: No such file or directory')
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
ERROR 07-05 14:07:36 [multiproc_executor.py:898] WorkerProc failed to start.
ERROR 07-05 14:07:36 [multiproc_executor.py:898] Traceback (most recent call last):
ERROR 07-05 14:07:36 [multiproc_executor.py:898]   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 865, in worker_main
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     worker = WorkerProc(*args, **kwargs)
ERROR 07-05 14:07:36 [multiproc_executor.py:898]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-05 14:07:36 [multiproc_executor.py:898]   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     return func(*args, **kwargs)
ERROR 07-05 14:07:36 [multiproc_executor.py:898]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 07-05 14:07:36 [multiproc_executor.py:898]   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 618, in __init__
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     wrapper.init_worker(all_kwargs)
ERROR 07-05 14:07:36 [multiproc_executor.py:898]   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     return func(*args, **kwargs)
ERROR 07-05 14:07:36 [multiproc_executor.py:898]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 07-05 14:07:36 [multiproc_executor.py:898]   File "/.venv/lib/python3.12/site-packages/vllm/v1/worker/worker_base.py", line 319, in init_worker
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     self.worker = worker_class(**kwargs)
ERROR 07-05 14:07:36 [multiproc_executor.py:898]                   ^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-05 14:07:36 [multiproc_executor.py:898]   File "/.venv/lib/python3.12/site-packages/vllm/v1/worker/cpu_worker.py", line 58, in __init__
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     torch.ops._C.init_cpu_memory_env([cpu_core.numa_node])
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-05 14:07:36 [multiproc_executor.py:898]   File "/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1379, in __getattr__
ERROR 07-05 14:07:36 [multiproc_executor.py:898]     raise AttributeError(
ERROR 07-05 14:07:36 [multiproc_executor.py:898] AttributeError: '_OpNamespace' '_C' object has no attribute 'init_cpu_memory_env'
(EngineCore pid=440) INFO 07-05 14:07:36 [multiproc_executor.py:426] [shutdown] Executor: waiting for worker exit count=1
(EngineCore pid=440) INFO 07-05 14:07:37 [multiproc_executor.py:433] [shutdown] Executor: all workers exited gracefully
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231] EngineCore failed to start.
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231] Traceback (most recent call last):
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1200, in run_engine_core
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     return func(*args, **kwargs)
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 966, in __init__
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     super().__init__(
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 123, in __init__
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     self.model_executor = executor_class(vllm_config)
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 108, in __init__
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     super().__init__(vllm_config)
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     return func(*args, **kwargs)
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 109, in __init__
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     self._init_executor()
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 201, in _init_executor
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 762, in wait_for_ready
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231]     raise e from None
(EngineCore pid=440) ERROR 07-05 14:07:37 [core.py:1231] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(EngineCore pid=440) Process EngineCore:
(EngineCore pid=440) Traceback (most recent call last):
(EngineCore pid=440)   File "/root/.local/share/uv/python/cpython-3.12.13-linux-aarch64-gnu/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore pid=440)     self.run()
(EngineCore pid=440)   File "/root/.local/share/uv/python/cpython-3.12.13-linux-aarch64-gnu/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore pid=440)     self._target(*self._args, **self._kwargs)
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1235, in run_engine_core
(EngineCore pid=440)     raise e
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1200, in run_engine_core
(EngineCore pid=440)     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=440)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=440)     return func(*args, **kwargs)
(EngineCore pid=440)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 966, in __init__
(EngineCore pid=440)     super().__init__(
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 123, in __init__
(EngineCore pid=440)     self.model_executor = executor_class(vllm_config)
(EngineCore pid=440)                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 108, in __init__
(EngineCore pid=440)     super().__init__(vllm_config)
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=440)     return func(*args, **kwargs)
(EngineCore pid=440)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 109, in __init__
(EngineCore pid=440)     self._init_executor()
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 201, in _init_executor
(EngineCore pid=440)     self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=440)                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=440)   File "/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 762, in wait_for_ready
(EngineCore pid=440)     raise e from None
(EngineCore pid=440) Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(APIServer pid=408) Traceback (most recent call last):
(APIServer pid=408)   File "/.venv/bin/vllm", line 10, in <module>
(APIServer pid=408)     sys.exit(main())
(APIServer pid=408)              ^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 95, in main
(APIServer pid=408)     args.dispatch_function(args)
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 148, in cmd
(APIServer pid=408)     uvloop.run(run_server(args))
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=408)     return __asyncio.run(
(APIServer pid=408)            ^^^^^^^^^^^^^^
(APIServer pid=408)   File "/root/.local/share/uv/python/cpython-3.12.13-linux-aarch64-gnu/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=408)     return runner.run(main)
(APIServer pid=408)            ^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/root/.local/share/uv/python/cpython-3.12.13-linux-aarch64-gnu/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=408)     return self._loop.run_until_complete(task)
(APIServer pid=408)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=408)     return await main
(APIServer pid=408)            ^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 663, in run_server
(APIServer pid=408)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 677, in run_server_worker
(APIServer pid=408)     async with build_async_engine_client(
(APIServer pid=408)                ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/root/.local/share/uv/python/cpython-3.12.13-linux-aarch64-gnu/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=408)     return await anext(self.gen)
(APIServer pid=408)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 99, in build_async_engine_client
(APIServer pid=408)     async with build_async_engine_client_from_engine_args(
(APIServer pid=408)                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/root/.local/share/uv/python/cpython-3.12.13-linux-aarch64-gnu/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=408)     return await anext(self.gen)
(APIServer pid=408)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 135, in build_async_engine_client_from_engine_args
(APIServer pid=408)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=408)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 217, in from_vllm_config
(APIServer pid=408)     return cls(
(APIServer pid=408)            ^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 146, in __init__
(APIServer pid=408)     self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=408)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=408)     return func(*args, **kwargs)
(APIServer pid=408)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 132, in make_async_mp_client
(APIServer pid=408)     return AsyncMPClient(*client_args)
(APIServer pid=408)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=408)     return func(*args, **kwargs)
(APIServer pid=408)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 963, in __init__
(APIServer pid=408)     super().__init__(
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 573, in __init__
(APIServer pid=408)     with launch_core_engines(
(APIServer pid=408)          ^^^^^^^^^^^^^^^^^^^^
(APIServer pid=408)   File "/root/.local/share/uv/python/cpython-3.12.13-linux-aarch64-gnu/lib/python3.12/contextlib.py", line 144, in __exit__
(APIServer pid=408)     next(self.gen)
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 1213, in launch_core_engines
(APIServer pid=408)     wait_for_engine_startup(
(APIServer pid=408)   File "/.venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 1272, in wait_for_engine_startup
(APIServer pid=408)     raise RuntimeError(
(APIServer pid=408) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
(.venv) root@80a09d6d82c7:/# 

Expected behavior / 期待表现

Model loads successfully

[SuperCowProducts]

I also tested with vllm-vulkan for GPU support, but it still the same error, so clearly issue is more fundamental.