mksquashfs -comp lz4 CalledProcessError during Compressing the environment step #595
Description
Steps I was able to complete before the reported issue:
- 1.
pip install fal(version 1.29.1 on Python 3.12.5) - 2.
fal auth - 3.
fal secret set HF_TOKEN - 4. run the pyjokes quick start
then as a next step, I want to check if I can run one of the standard diffusers quick starts, with the example from https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev#using-with-diffusers-🧨
the source below is the quick start from the HF page above, with my edits of attempting to use download_file and FAL_PERSISTENT_DIR to exercise the fal.toolkit
import fal
from fal.toolkit import FAL_PERSISTENT_DIR, download_file
@fal.function(
"virtualenv",
python_version="3.12",
requirements=["git+https://github.com/huggingface/diffusers.git", "hf_transfer", "torch", "transformers"]
)
def main() -> str:
import datetime
import torch
from diffusers import FluxKontextPipeline
from diffusers.utils import load_image
now = datetime.datetime.now()
formatted_time = now.strftime("%Y%m%d%H%M")
pipe = FluxKontextPipeline.from_pretrained("black-forest-labs/FLUX.1-Kontext-dev", torch_dtype=torch.bfloat16)
pipe.to("cuda")
input_path = download_file("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png")
input_image = load_image(input_path)
image = pipe(
image=input_image,
prompt="Add a green hat to the cat",
guidance_scale=2.5
).images[0]
image.save(save_path := FAL_PERSISTENT_DIR / f"output_{formatted_time}.png")
return save_path
the pip install seems to go well
2025-08-15 23:11:03.955 [info ] Starting runner d940ea9b-fb2b-44c6-b41e-d40627b15c73
2025-08-15 23:11:06.122 [info ] Opening connection to runner
2025-08-15 23:11:08.596 [info ] Collecting git+https://github.com/huggingface/diffusers.git
2025-08-15 23:11:08.597 [info ] Cloning https://github.com/huggingface/diffusers.git to /tmp/pip-req-build-b0ug9qfv
2025-08-15 23:11:08.603 [info ] Running command git clone --filter=blob:none --quiet https://github.com/huggingface/diffusers.git /tmp/pip-req-build-b0ug9qfv
2025-08-15 23:11:12.507 [info ] Resolved https://github.com/huggingface/diffusers.git to commit a58a4f665b4aa86205fb8c1795e79c331d65bb18
2025-08-15 23:11:12.511 [info ] Installing build dependencies: started
2025-08-15 23:11:13.374 [info ] Installing build dependencies: finished with status 'done'
2025-08-15 23:11:13.375 [info ] Getting requirements to build wheel: started
2025-08-15 23:11:13.780 [info ] Getting requirements to build wheel: finished with status 'done'
2025-08-15 23:11:13.781 [info ] Preparing metadata (pyproject.toml): started
2025-08-15 23:11:14.023 [info ] Preparing metadata (pyproject.toml): finished with status 'done'
2025-08-15 23:11:14.139 [info ] Collecting isolate==0.18.0
2025-08-15 23:11:14.177 [info ] Downloading isolate-0.18.0-py3-none-any.whl.metadata (3.2 kB)
2025-08-15 23:11:14.213 [info ] Collecting dill==0.3.7
2025-08-15 23:11:14.214 [info ] Using cached dill-0.3.7-py3-none-any.whl.metadata (9.9 kB)
2025-08-15 23:11:14.244 [info ] Collecting cloudpickle==3.0.0
2025-08-15 23:11:14.246 [info ] Using cached cloudpickle-3.0.0-py3-none-any.whl.metadata (7.0 kB)
2025-08-15 23:11:14.292 [info ] Collecting hf_transfer
2025-08-15 23:11:14.301 [info ] Downloading hf_transfer-0.1.9-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:14.362 [info ] Collecting torch
2025-08-15 23:11:14.370 [info ] Downloading torch-2.8.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (30 kB)
2025-08-15 23:11:14.438 [info ] Collecting transformers
2025-08-15 23:11:14.446 [info ] Downloading transformers-4.55.2-py3-none-any.whl.metadata (41 kB)
2025-08-15 23:11:14.689 [info ] Collecting importlib_metadata (from diffusers==0.35.0.dev0)
2025-08-15 23:11:14.702 [info ] Downloading importlib_metadata-8.7.0-py3-none-any.whl.metadata (4.8 kB)
2025-08-15 23:11:14.739 [info ] Collecting filelock (from diffusers==0.35.0.dev0)
2025-08-15 23:11:14.748 [info ] Downloading filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB)
2025-08-15 23:11:14.803 [info ] Collecting huggingface-hub>=0.34.0 (from diffusers==0.35.0.dev0)
2025-08-15 23:11:14.814 [info ] Downloading huggingface_hub-0.34.4-py3-none-any.whl.metadata (14 kB)
2025-08-15 23:11:15.030 [info ] Collecting numpy (from diffusers==0.35.0.dev0)
2025-08-15 23:11:15.040 [info ] Downloading numpy-2.3.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (62 kB)
2025-08-15 23:11:15.408 [info ] Collecting regex!=2019.12.17 (from diffusers==0.35.0.dev0)
2025-08-15 23:11:15.421 [info ] Downloading regex-2025.7.34-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (40 kB)
2025-08-15 23:11:15.465 [info ] Collecting requests (from diffusers==0.35.0.dev0)
2025-08-15 23:11:15.473 [info ] Downloading requests-2.32.4-py3-none-any.whl.metadata (4.9 kB)
2025-08-15 23:11:15.636 [info ] Collecting safetensors>=0.3.1 (from diffusers==0.35.0.dev0)
2025-08-15 23:11:15.647 [info ] Downloading safetensors-0.6.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)
2025-08-15 23:11:15.835 [info ] Collecting Pillow (from diffusers==0.35.0.dev0)
2025-08-15 23:11:15.844 [info ] Downloading pillow-11.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (9.0 kB)
2025-08-15 23:11:16.434 [info ] Collecting grpcio<2,>=1.64.0 (from isolate==0.18.0)
2025-08-15 23:11:16.445 [info ] Downloading grpcio-1.74.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
2025-08-15 23:11:16.638 [info ] Collecting protobuf (from isolate==0.18.0)
2025-08-15 23:11:16.648 [info ] Downloading protobuf-6.32.0-cp39-abi3-manylinux2014_x86_64.whl.metadata (593 bytes)
2025-08-15 23:11:16.676 [info ] Collecting tblib>=1.7.0 (from isolate==0.18.0)
2025-08-15 23:11:16.678 [info ] Using cached tblib-3.1.0-py3-none-any.whl.metadata (25 kB)
2025-08-15 23:11:16.709 [info ] Collecting platformdirs (from isolate==0.18.0)
2025-08-15 23:11:16.711 [info ] Using cached platformdirs-4.3.8-py3-none-any.whl.metadata (12 kB)
2025-08-15 23:11:16.745 [info ] Collecting typing-extensions>=4.10.0 (from torch)
2025-08-15 23:11:16.753 [info ] Downloading typing_extensions-4.14.1-py3-none-any.whl.metadata (3.0 kB)
2025-08-15 23:11:16.899 [info ] Collecting setuptools (from torch)
2025-08-15 23:11:16.901 [info ] Using cached setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
2025-08-15 23:11:16.934 [info ] Collecting sympy>=1.13.3 (from torch)
2025-08-15 23:11:16.943 [info ] Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
2025-08-15 23:11:16.980 [info ] Collecting networkx (from torch)
2025-08-15 23:11:16.989 [info ] Downloading networkx-3.5-py3-none-any.whl.metadata (6.3 kB)
2025-08-15 23:11:17.020 [info ] Collecting jinja2 (from torch)
2025-08-15 23:11:17.028 [info ] Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
2025-08-15 23:11:17.063 [info ] Collecting fsspec (from torch)
2025-08-15 23:11:17.071 [info ] Downloading fsspec-2025.7.0-py3-none-any.whl.metadata (12 kB)
2025-08-15 23:11:17.133 [info ] Collecting nvidia-cuda-nvrtc-cu12==12.8.93 (from torch)
2025-08-15 23:11:17.141 [info ] Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.168 [info ] Collecting nvidia-cuda-runtime-cu12==12.8.90 (from torch)
2025-08-15 23:11:17.177 [info ] Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.205 [info ] Collecting nvidia-cuda-cupti-cu12==12.8.90 (from torch)
2025-08-15 23:11:17.214 [info ] Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.238 [info ] Collecting nvidia-cudnn-cu12==9.10.2.21 (from torch)
2025-08-15 23:11:17.247 [info ] Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
2025-08-15 23:11:17.270 [info ] Collecting nvidia-cublas-cu12==12.8.4.1 (from torch)
2025-08-15 23:11:17.279 [info ] Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.305 [info ] Collecting nvidia-cufft-cu12==11.3.3.83 (from torch)
2025-08-15 23:11:17.313 [info ] Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.338 [info ] Collecting nvidia-curand-cu12==10.3.9.90 (from torch)
2025-08-15 23:11:17.346 [info ] Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.377 [info ] Collecting nvidia-cusolver-cu12==11.7.3.90 (from torch)
2025-08-15 23:11:17.385 [info ] Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
2025-08-15 23:11:17.413 [info ] Collecting nvidia-cusparse-cu12==12.5.8.93 (from torch)
2025-08-15 23:11:17.428 [info ] Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
2025-08-15 23:11:17.457 [info ] Collecting nvidia-cusparselt-cu12==0.7.1 (from torch)
2025-08-15 23:11:17.467 [info ] Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (7.0 kB)
2025-08-15 23:11:17.489 [info ] Collecting nvidia-nccl-cu12==2.27.3 (from torch)
2025-08-15 23:11:17.498 [info ] Downloading nvidia_nccl_cu12-2.27.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB)
2025-08-15 23:11:17.524 [info ] Collecting nvidia-nvtx-cu12==12.8.90 (from torch)
2025-08-15 23:11:17.533 [info ] Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
2025-08-15 23:11:17.559 [info ] Collecting nvidia-nvjitlink-cu12==12.8.93 (from torch)
2025-08-15 23:11:17.567 [info ] Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.588 [info ] Collecting nvidia-cufile-cu12==1.13.1.3 (from torch)
2025-08-15 23:11:17.597 [info ] Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.644 [info ] Collecting triton==3.4.0 (from torch)
2025-08-15 23:11:17.652 [info ] Downloading triton-3.4.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.7 kB)
2025-08-15 23:11:17.775 [info ] Collecting packaging>=20.0 (from transformers)
2025-08-15 23:11:17.784 [info ] Downloading packaging-25.0-py3-none-any.whl.metadata (3.3 kB)
2025-08-15 23:11:17.837 [info ] Collecting pyyaml>=5.1 (from transformers)
2025-08-15 23:11:17.839 [info ] Using cached PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
2025-08-15 23:11:18.055 [info ] Collecting tokenizers<0.22,>=0.21 (from transformers)
2025-08-15 23:11:18.065 [info ] Downloading tokenizers-0.21.4-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
2025-08-15 23:11:18.122 [info ] Collecting tqdm>=4.27 (from transformers)
2025-08-15 23:11:18.130 [info ] Downloading tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
2025-08-15 23:11:18.217 [info ] Collecting hf-xet<2.0.0,>=1.1.3 (from huggingface-hub>=0.34.0->diffusers==0.35.0.dev0)
2025-08-15 23:11:18.226 [info ] Downloading hf_xet-1.1.7-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (703 bytes)
2025-08-15 23:11:18.281 [info ] Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
2025-08-15 23:11:18.290 [info ] Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
2025-08-15 23:11:18.329 [info ] Collecting zipp>=3.20 (from importlib_metadata->diffusers==0.35.0.dev0)
2025-08-15 23:11:18.338 [info ] Downloading zipp-3.23.0-py3-none-any.whl.metadata (3.6 kB)
2025-08-15 23:11:18.403 [info ] Collecting MarkupSafe>=2.0 (from jinja2->torch)
2025-08-15 23:11:18.412 [info ] Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
2025-08-15 23:11:18.512 [info ] Collecting charset_normalizer<4,>=2 (from requests->diffusers==0.35.0.dev0)
2025-08-15 23:11:18.521 [info ] Downloading charset_normalizer-3.4.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (36 kB)
2025-08-15 23:11:18.549 [info ] Collecting idna<4,>=2.5 (from requests->diffusers==0.35.0.dev0)
2025-08-15 23:11:18.558 [info ] Downloading idna-3.10-py3-none-any.whl.metadata (10 kB)
2025-08-15 23:11:18.600 [info ] Collecting urllib3<3,>=1.21.1 (from requests->diffusers==0.35.0.dev0)
2025-08-15 23:11:18.608 [info ] Downloading urllib3-2.5.0-py3-none-any.whl.metadata (6.5 kB)
2025-08-15 23:11:18.640 [info ] Collecting certifi>=2017.4.17 (from requests->diffusers==0.35.0.dev0)
2025-08-15 23:11:18.648 [info ] Downloading certifi-2025.8.3-py3-none-any.whl.metadata (2.4 kB)
2025-08-15 23:11:18.671 [info ] Downloading isolate-0.18.0-py3-none-any.whl (71 kB)
2025-08-15 23:11:18.677 [info ] Using cached dill-0.3.7-py3-none-any.whl (115 kB)
2025-08-15 23:11:18.679 [info ] Using cached cloudpickle-3.0.0-py3-none-any.whl (20 kB)
2025-08-15 23:11:18.693 [info ] Downloading grpcio-1.74.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.2 MB)
2025-08-15 23:11:18.751 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.2/6.2 MB 122.7 MB/s eta 0:00:00
2025-08-15 23:11:18.761 [info ] Downloading hf_transfer-0.1.9-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
2025-08-15 23:11:18.791 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 125.0 MB/s eta 0:00:00
2025-08-15 23:11:18.801 [info ] Downloading torch-2.8.0-cp312-cp312-manylinux_2_28_x86_64.whl (887.9 MB)
2025-08-15 23:11:25.418 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 887.9/887.9 MB 86.8 MB/s eta 0:00:00
2025-08-15 23:11:25.430 [info ] Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl (594.3 MB)
2025-08-15 23:11:29.736 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 594.3/594.3 MB 110.5 MB/s eta 0:00:00
2025-08-15 23:11:29.749 [info ] Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (10.2 MB)
2025-08-15 23:11:29.834 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.2/10.2 MB 124.1 MB/s eta 0:00:00
2025-08-15 23:11:29.843 [info ] Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (88.0 MB)
2025-08-15 23:11:30.457 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.0/88.0 MB 144.1 MB/s eta 0:00:00
2025-08-15 23:11:30.466 [info ] Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (954 kB)
2025-08-15 23:11:30.478 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 954.8/954.8 kB 94.1 MB/s eta 0:00:00
2025-08-15 23:11:30.486 [info ] Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl (706.8 MB)
2025-08-15 23:11:35.607 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 706.8/706.8 MB 88.7 MB/s eta 0:00:00
2025-08-15 23:11:35.618 [info ] Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (193.1 MB)
2025-08-15 23:11:37.024 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 193.1/193.1 MB 137.7 MB/s eta 0:00:00
2025-08-15 23:11:37.035 [info ] Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.2 MB)
2025-08-15 23:11:37.048 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 101.3 MB/s eta 0:00:00
2025-08-15 23:11:37.058 [info ] Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl (63.6 MB)
2025-08-15 23:11:37.521 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.6/63.6 MB 138.1 MB/s eta 0:00:00
2025-08-15 23:11:37.530 [info ] Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl (267.5 MB)
2025-08-15 23:11:39.650 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 267.5/267.5 MB 125.3 MB/s eta 0:00:00
2025-08-15 23:11:39.661 [info ] Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (288.2 MB)
2025-08-15 23:11:41.793 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 288.2/288.2 MB 130.2 MB/s eta 0:00:00
2025-08-15 23:11:41.803 [info ] Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl (287.2 MB)
2025-08-15 23:11:43.646 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 287.2/287.2 MB 152.0 MB/s eta 0:00:00
2025-08-15 23:11:43.656 [info ] Downloading nvidia_nccl_cu12-2.27.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (322.4 MB)
2025-08-15 23:11:45.876 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 322.4/322.4 MB 135.7 MB/s eta 0:00:00
2025-08-15 23:11:45.885 [info ] Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (39.3 MB)
2025-08-15 23:11:46.157 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.3/39.3 MB 146.1 MB/s eta 0:00:00
2025-08-15 23:11:46.165 [info ] Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89 kB)
2025-08-15 23:11:46.181 [info ] Downloading triton-3.4.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (155.6 MB)
2025-08-15 23:11:47.237 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.6/155.6 MB 147.6 MB/s eta 0:00:00
2025-08-15 23:11:47.248 [info ] Downloading transformers-4.55.2-py3-none-any.whl (11.3 MB)
2025-08-15 23:11:47.356 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 106.0 MB/s eta 0:00:00
2025-08-15 23:11:47.365 [info ] Downloading huggingface_hub-0.34.4-py3-none-any.whl (561 kB)
2025-08-15 23:11:47.376 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 561.5/561.5 kB 47.0 MB/s eta 0:00:00
2025-08-15 23:11:47.384 [info ] Downloading hf_xet-1.1.7-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
2025-08-15 23:11:47.412 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 116.8 MB/s eta 0:00:00
2025-08-15 23:11:47.422 [info ] Downloading tokenizers-0.21.4-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
2025-08-15 23:11:47.446 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 146.8 MB/s eta 0:00:00
2025-08-15 23:11:47.454 [info ] Downloading fsspec-2025.7.0-py3-none-any.whl (199 kB)
2025-08-15 23:11:47.472 [info ] Downloading numpy-2.3.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.6 MB)
2025-08-15 23:11:47.676 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.6/16.6 MB 81.5 MB/s eta 0:00:00
2025-08-15 23:11:47.686 [info ] Downloading packaging-25.0-py3-none-any.whl (66 kB)
2025-08-15 23:11:47.692 [info ] Using cached PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767 kB)
2025-08-15 23:11:47.712 [info ] Downloading regex-2025.7.34-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (801 kB)
2025-08-15 23:11:47.721 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 801.9/801.9 kB 77.2 MB/s eta 0:00:00
2025-08-15 23:11:47.733 [info ] Downloading safetensors-0.6.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (485 kB)
2025-08-15 23:11:47.742 [info ] Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB)
2025-08-15 23:11:47.752 [info ] Downloading sympy-1.14.0-py3-none-any.whl (6.3 MB)
2025-08-15 23:11:47.796 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 151.2 MB/s eta 0:00:00
2025-08-15 23:11:47.804 [info ] Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
2025-08-15 23:11:47.813 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 53.7 MB/s eta 0:00:00
2025-08-15 23:11:47.814 [info ] Using cached tblib-3.1.0-py3-none-any.whl (12 kB)
2025-08-15 23:11:47.824 [info ] Downloading tqdm-4.67.1-py3-none-any.whl (78 kB)
2025-08-15 23:11:47.837 [info ] Downloading typing_extensions-4.14.1-py3-none-any.whl (43 kB)
2025-08-15 23:11:47.854 [info ] Downloading filelock-3.19.1-py3-none-any.whl (15 kB)
2025-08-15 23:11:47.869 [info ] Downloading importlib_metadata-8.7.0-py3-none-any.whl (27 kB)
2025-08-15 23:11:47.882 [info ] Downloading zipp-3.23.0-py3-none-any.whl (10 kB)
2025-08-15 23:11:47.897 [info ] Downloading jinja2-3.1.6-py3-none-any.whl (134 kB)
2025-08-15 23:11:47.910 [info ] Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23 kB)
2025-08-15 23:11:47.926 [info ] Downloading networkx-3.5-py3-none-any.whl (2.0 MB)
2025-08-15 23:11:47.941 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 150.0 MB/s eta 0:00:00
2025-08-15 23:11:47.953 [info ] Downloading pillow-11.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB)
2025-08-15 23:11:47.995 [info ] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 163.3 MB/s eta 0:00:00
2025-08-15 23:11:47.997 [info ] Using cached platformdirs-4.3.8-py3-none-any.whl (18 kB)
2025-08-15 23:11:48.011 [info ] Downloading protobuf-6.32.0-cp39-abi3-manylinux2014_x86_64.whl (322 kB)
2025-08-15 23:11:48.026 [info ] Downloading requests-2.32.4-py3-none-any.whl (64 kB)
2025-08-15 23:11:48.038 [info ] Downloading charset_normalizer-3.4.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (151 kB)
2025-08-15 23:11:48.051 [info ] Downloading idna-3.10-py3-none-any.whl (70 kB)
2025-08-15 23:11:48.066 [info ] Downloading urllib3-2.5.0-py3-none-any.whl (129 kB)
2025-08-15 23:11:48.080 [info ] Downloading certifi-2025.8.3-py3-none-any.whl (161 kB)
2025-08-15 23:11:53.301 [info ] Building wheels for collected packages: diffusers
2025-08-15 23:11:53.304 [info ] Building wheel for diffusers (pyproject.toml): started
2025-08-15 23:11:54.600 [info ] Building wheel for diffusers (pyproject.toml): finished with status 'done'
2025-08-15 23:11:54.606 [info ] Created wheel for diffusers: filename=diffusers-0.35.0.dev0-py3-none-any.whl size=4109736 sha256=d5e2fe8407de828c07c2a29a4ed55c12a1b70fbc7ea25b544beb0d16e62fb422
2025-08-15 23:11:54.606 [info ] Stored in directory: /tmp/pip-ephem-wheel-cache-h5t_otru/wheels/23/0f/7d/f97813d265ed0e599a78d83afd4e1925740896ca79b46cccfd
2025-08-15 23:11:54.614 [info ] Successfully built diffusers
2025-08-15 23:11:54.834 [info ] Installing collected packages: nvidia-cusparselt-cu12, mpmath, zipp, urllib3, typing-extensions, tqdm, tblib, sympy, setuptools, safetensors, regex, pyyaml, protobuf, platformdirs, Pillow, packaging, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufile-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, MarkupSafe, idna, hf-xet, hf_transfer, grpcio, fsspec, filelock, dill, cloudpickle, charset_normalizer, certifi, triton, requests, nvidia-cusparse-cu12, nvidia-cufft-cu12, nvidia-cudnn-cu12, jinja2, isolate, importlib_metadata, nvidia-cusolver-cu12, huggingface-hub, torch, tokenizers, diffusers, transformers
2025-08-15 23:13:19.222 [info ] Successfully installed MarkupSafe-3.0.2 Pillow-11.3.0 certifi-2025.8.3 charset_normalizer-3.4.3 cloudpickle-3.0.0 diffusers-0.35.0.dev0 dill-0.3.7 filelock-3.19.1 fsspec-2025.7.0 grpcio-1.74.0 hf-xet-1.1.7 hf_transfer-0.1.9 huggingface-hub-0.34.4 idna-3.10 importlib_metadata-8.7.0 isolate-0.18.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.5 numpy-2.3.2 nvidia-cublas-cu12-12.8.4.1 nvidia-cuda-cupti-cu12-12.8.90 nvidia-cuda-nvrtc-cu12-12.8.93 nvidia-cuda-runtime-cu12-12.8.90 nvidia-cudnn-cu12-9.10.2.21 nvidia-cufft-cu12-11.3.3.83 nvidia-cufile-cu12-1.13.1.3 nvidia-curand-cu12-10.3.9.90 nvidia-cusolver-cu12-11.7.3.90 nvidia-cusparse-cu12-12.5.8.93 nvidia-cusparselt-cu12-0.7.1 nvidia-nccl-cu12-2.27.3 nvidia-nvjitlink-cu12-12.8.93 nvidia-nvtx-cu12-12.8.90 packaging-25.0 platformdirs-4.3.8 protobuf-6.32.0 pyyaml-6.0.2 regex-2025.7.34 requests-2.32.4 safetensors-0.6.2 setuptools-80.9.0 sympy-1.14.0 tblib-3.1.0 tokenizers-0.21.4 torch-2.8.0 tqdm-4.67.1 transformers-4.55.2 triton-3.4.0 typing-extensions-4.14.1 urllib3-2.5.0 zipp-3.23.0
the output here pauses for a while before continuing:
2025-08-15 23:13:19.303 [info ] [notice] A new release of pip is available: 25.1.1 -> 25.2
2025-08-15 23:13:19.303 [info ] [notice] To update, run: /root/.cache/isolate/virtualenv/12ff153f65b4d8134859e8575c67a14be498c7238407f2d2602664be147d0132/bin/python -m pip install --upgrade pip
after that, this Compressing the environment... step here also takes a while before showing Traceback:
2025-08-15 23:13:19.962 [info ] Compressing the environment...
Traceback (most recent call last):
File "/Users/anthonywu/workspace/mflux-ops/.venv/bin/mflux-fal-run", line 10, in <module>
sys.exit(main())
^^^^^^
File "/Users/anthonywu/workspace/mflux-ops/.venv/lib/python3.12/site-packages/fal/api.py", line 845, in __call__
return self.host.run(
^^^^^^^^^^^^^^
File "/Users/anthonywu/workspace/mflux-ops/.venv/lib/python3.12/site-packages/fal/api.py", line 235, in handler
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/anthonywu/workspace/mflux-ops/.venv/lib/python3.12/site-packages/fal/api.py", line 447, in run
for partial_result in self._connection.run(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/anthonywu/workspace/mflux-ops/.venv/lib/python3.12/site-packages/fal/sdk.py", line 521, in run
yield from_grpc(partial_result)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/anthonywu/.local/share/uv/python/cpython-3.12.5-macos-aarch64-none/lib/python3.12/functools.py", line 907, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/anthonywu/workspace/mflux-ops/.venv/lib/python3.12/site-packages/fal/sdk.py", line 314, in _from_grpc_hosted_run_result
return_value = from_grpc(message.return_value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/anthonywu/.local/share/uv/python/cpython-3.12.5-macos-aarch64-none/lib/python3.12/functools.py", line 907, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/anthonywu/workspace/mflux-ops/.venv/lib/python3.12/site-packages/isolate/connections/grpc/interface.py", line 29, in _
return load_serialized_object(
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/anthonywu/workspace/mflux-ops/.venv/lib/python3.12/site-packages/isolate/connections/common.py", line 78, in load_serialized_object
raise prepare_exc(result, stringized_traceback=stringized_traceback)
File "/opt/venv/lib/python3.12/site-packages/isolate/connections/grpc/agent.py", line 146, in execute_function
File "<string>", line 20, in <module>
File "/root/.pyenv/versions/3.12.4/lib/python3.12/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
^^^^^^^^^^^^^^^^^
subprocess.CalledProcessError: Command '['mksquashfs', '/root/.cache/isolate/virtualenv/12ff153f65b4d8134859e8575c67a14be498c7238407f2d2602664be147d0132', PosixPath('/tmp/env.sqfs'), '-comp', 'lz4']' died with <Signals.SIGKILL: 9>.
the failure happens in the isolate dependency: https://github.com/fal-ai/isolate/blob/main/src/isolate/connections/grpc/agent.py#L146