Skip to content

Commit 8e2f087

Browse files
committed
feat(loader): real byte-progress download (%, MB/s, ETA) + disable xet by default
The first-run auto-download showed huggingface_hub's 'Fetching N files' bar, which counts files not bytes -> it sits at ~5% (1/19) for many minutes while a 14GB shard streams, looking hung. Replace it with an aggregating tqdm that sums all byte-unit bars into one global ProgressBar + a throttled console line (12.3% 4.95/40.30GB 47.2MB/s ETA 12.6min), ignoring the file-count bar. Also default HF_HUB_DISABLE_XET=1: the xet transport can stall on Windows portable; plain HTTPS/LFS is slower but reliable and resumable. Overridable via HF_HUB_DISABLE_XET=0. Space check now uses the repo's real total bytes. v0.3.4.
1 parent 5ebccb6 commit 8e2f087

3 files changed

Lines changed: 81 additions & 25 deletions

File tree

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,11 @@ The **BerniniR · Load Model** node can fetch the weights for you — no manual
4242
- **`neuregex/Bernini-R-fp8 (auto)`** *(default)*[fp8 (e4m3) self-contained bundle](https://huggingface.co/neuregex/Bernini-R-fp8), **~40 GB**, runs the full pipeline in **24 GB**. The fp8 weights are bit-identical to the node's on-the-fly quantization.
4343
- **`ByteDance/Bernini-R-Diffusers (full bf16)`** — original bf16 weights (~126 GB; A100-class, or use on-the-fly `fp8`).
4444
- **`local`** — use the `model_dir` path directly.
45-
- **`auto_download`** *(default on)* — if the chosen repo's weights are missing, downloads them (with a free-space check, a `~40 GB first run` notice, and a progress bar). Turn it off to require a manual download.
45+
- **`auto_download`** *(default on)* — if the chosen repo's weights are missing, downloads them. The first run pulls **~40 GB** (fp8 bundle), so it takes a while; the console shows **real byte progress** (`12.3% 4.95/40.30GB 47.2MB/s ETA 12.6min`), not just a file count. The download is **resumable** — if you cancel (Ctrl-C / Cancel), the next Run picks up where it left off. A free-space check runs first. Turn it off to require a manual download.
4646
- **`download_dir`** *(default `models/bernini`)* — where HF repos are downloaded (relative to ComfyUI, or absolute).
4747

48+
> **The download stalled at "Fetching N files: 5%"?** That old file-count bar (≤0.3.3) looked frozen while a 14 GB shard streamed. 0.3.4+ shows true byte progress instead. The node also disables HF's **xet** transport by default (`HF_HUB_DISABLE_XET=1`), which can hang on Windows portable; it uses plain HTTPS (LFS), which is slower but rock-solid. To re-enable xet, set `HF_HUB_DISABLE_XET=0` before launching ComfyUI.
49+
4850
Manual download (optional), then set `source = local` and `model_dir` to the folder:
4951

5052
```bash

nodes.py

Lines changed: 77 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -72,59 +72,113 @@ def _resolve_dir(p):
7272
return os.path.abspath(p)
7373

7474

75-
def _comfy_tqdm():
76-
"""tqdm que además refleja el progreso en la barra de ComfyUI (best-effort)."""
75+
def _byte_progress(total_bytes):
76+
"""Factory de tqdm que AGREGA todas las barras de bytes de la descarga en UNA
77+
barra global -> progreso REAL (%, GB, MB/s, ETA) en consola + barra de ComfyUI.
78+
79+
IGNORA la barra 'Fetching N files' (cuenta ARCHIVOS, no bytes): esa es la que
80+
daba falsa sensación de cuelgue -> se queda en 5% (=1/19 archivos) durante los
81+
minutos que tarda un shard de 14GB, sin mostrar bytes. Aquí, mientras baja ese
82+
shard, el usuario ve el % real subir y los MB/s."""
7783
try:
78-
from comfy.utils import ProgressBar
7984
from tqdm import tqdm as _tqdm
8085
except Exception:
8186
return None
87+
try:
88+
from comfy.utils import ProgressBar
89+
pb = ProgressBar(total_bytes) if total_bytes else None
90+
except Exception:
91+
pb = None
92+
import threading
93+
import time
94+
st = {"done": 0, "t0": time.time(), "last": 0.0, "lock": threading.Lock()}
8295

83-
class _PBTqdm(_tqdm):
96+
class _AggTqdm(_tqdm):
8497
def __init__(self, *a, **k):
8598
super().__init__(*a, **k)
86-
self._pb = ProgressBar(self.total) if getattr(self, "total", None) else None
99+
# snapshot_download usa unit="B" en las barras de bytes; la de archivos no.
100+
self._bytes = (getattr(self, "unit", "") or "").upper().startswith("B")
101+
self._prev = 0
87102

88103
def update(self, n=1):
89104
super().update(n)
90-
try:
91-
if self._pb and self.total:
92-
self._pb.update_absolute(self.n, self.total)
93-
except Exception:
94-
pass
95-
return _PBTqdm
105+
if not self._bytes:
106+
return
107+
with st["lock"]:
108+
st["done"] += self.n - self._prev
109+
self._prev = self.n
110+
done = st["done"]
111+
now = time.time()
112+
emit = now - st["last"] >= 1.0
113+
if emit:
114+
st["last"] = now
115+
if pb and total_bytes:
116+
try:
117+
pb.update_absolute(min(done, total_bytes), total_bytes)
118+
except Exception:
119+
pass
120+
if emit and total_bytes:
121+
g = 1024 ** 3
122+
el = max(now - st["t0"], 1e-6)
123+
spd = done / el / (1024 ** 2) # MB/s medios
124+
eta = (total_bytes - done) / max(done / el, 1.0) / 60 # min restantes
125+
print(f"[BerniniR] {100 * done / total_bytes:4.1f}% "
126+
f"{done / g:5.2f}/{total_bytes / g:.2f}GB {spd:5.1f}MB/s "
127+
f"ETA {eta:4.1f}min", flush=True)
128+
return _AggTqdm
96129

97130

98131
def _ensure_weights(repo_id, dst, auto_download):
99132
"""Garantiza pesos en `dst`; si faltan y auto_download, los baja de HF con
100-
check de espacio + aviso + progreso (tqdm/ProgressBar). Si no, raise claro."""
133+
progreso REAL por bytes (%, MB/s, ETA) + check de espacio exacto. El transporte
134+
xet va DESACTIVADO por defecto: en Windows/portable puede colgarse; HTTPS clásico
135+
(LFS) es estable y resumible. Si faltan y no hay auto_download, raise con la orden
136+
manual. Es resumible: un Ctrl-C deja los .safetensors a medias y el re-run retoma."""
101137
if _has_weights(dst):
102138
return dst
103139
if not auto_download:
104140
raise FileNotFoundError(
105141
f"[BerniniR] Faltan pesos de '{repo_id}' en {dst}. Activa auto_download, o "
106142
f"descárgalos a mano:\n huggingface-cli download {repo_id} --local-dir \"{dst}\"")
143+
144+
# xet OFF por defecto (evita el cuelgue del transporte xet en Windows/portable).
145+
# Para forzar xet: exporta HF_HUB_DISABLE_XET=0 antes de lanzar ComfyUI.
146+
os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
147+
107148
import shutil as _sh
108149
from huggingface_hub import snapshot_download
109150
parent = os.path.dirname(dst) or "."
110-
os.makedirs(parent, exist_ok=True)
111-
free_gb = _sh.disk_usage(parent).free / (1024 ** 3)
112-
need_gb = 40 if "fp8" in repo_id.lower() else 130
113-
print(f"[BerniniR] descargando '{repo_id}' -> {dst} (~{need_gb}GB el PRIMER run; "
114-
f"{free_gb:.0f}GB libres en {parent})")
115-
if free_gb < need_gb * 1.1:
116-
raise RuntimeError(
117-
f"[BerniniR] espacio insuficiente para '{repo_id}': ~{int(need_gb * 1.1)}GB "
118-
f"necesarios, solo {free_gb:.0f}GB libres en {parent}.")
119151
os.makedirs(dst, exist_ok=True)
152+
g = 1024 ** 3
153+
154+
# Total REAL en bytes desde la metadata del repo -> % fiable + check exacto.
155+
total = 0
156+
try:
157+
from huggingface_hub import HfApi
158+
info = HfApi().model_info(repo_id, files_metadata=True)
159+
total = sum((s.size or 0) for s in info.siblings if not s.rfilename.endswith("/"))
160+
except Exception as e:
161+
print(f"[BerniniR] aviso: no pude leer tamaños del repo ({e}); sigo sin % global.")
162+
163+
need = total if total else (40 * g if "fp8" in repo_id.lower() else 130 * g)
164+
free = _sh.disk_usage(parent).free
165+
xet_off = os.environ.get("HF_HUB_DISABLE_XET", "0") not in ("0", "false", "False")
166+
print(f"[BerniniR] descargando '{repo_id}' -> {dst}", flush=True)
167+
print(f"[BerniniR] total ~{need / g:.1f}GB | libres {free / g:.0f}GB en {parent} | "
168+
f"xet={'off' if xet_off else 'on'} | resumible (Ctrl-C y re-run retoma)", flush=True)
169+
if free < need * 1.05:
170+
raise RuntimeError(
171+
f"[BerniniR] espacio insuficiente para '{repo_id}': ~{need * 1.05 / g:.0f}GB "
172+
f"necesarios, solo {free / g:.0f}GB libres en {parent}.")
173+
120174
kw = {}
121-
tq = _comfy_tqdm()
175+
tq = _byte_progress(total)
122176
if tq is not None:
123177
kw["tqdm_class"] = tq
124178
snapshot_download(repo_id=repo_id, local_dir=dst, **kw)
125179
if not _has_weights(dst):
126180
raise RuntimeError(f"[BerniniR] descarga incompleta: sin transformer/*.safetensors en {dst}")
127-
print(f"[BerniniR] descarga completa: {dst}")
181+
print(f"[BerniniR] descarga completa: {dst}", flush=True)
128182
return dst
129183

130184

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[project]
22
name = "comfyui-berninir"
33
description = "ByteDance Bernini-R (Wan2.2-T2V-A14B + source-id RoPE + APG multi-condition guidance) for ComfyUI: t2v/t2i, image & video editing, reference-to-video."
4-
version = "0.3.3"
4+
version = "0.3.4"
55
license = { file = "LICENSE" }
66
requires-python = ">=3.10"
77
dependencies = [

0 commit comments

Comments
 (0)