Skip to content

Commit 45273bc

Browse files
NookieAIclaude
andcommitted
Release v2.4.9 — transfer safety: prevent data loss on Move + 14 audit fixes
Critical (Move could report success after an incomplete transfer, then delete source): - buildFtpManifest: track dirs that fail to LIST, retry (capped), and throw if a real download walk is incomplete — so an incomplete FTP download can't trigger source delete. - uploadFtpRecursive: follow NTFS junction/symlink dirs (were silently skipped) via stat; verify remote SIZE after each upload (retry short writes); release read-stream fd on retry. - same-server FTP Move: skip when source path === destination path (overwrite no-op that previously download+upload+deleted the just-written folder). - FTP overwrite: abort the item if the pre-overwrite delete fails (was swallowed -> merge). Other fixes: - free-space pre-check now also covers cross-device Move. - FTP download uses toExtendedPath (long Windows paths) + size-based resume skip. - ppsa-only preview includes version suffix (matches check-conflicts/doEnsureAndPopulate). - free-space total ignores the -1 size sentinel. - FTP rename-on-conflict falls back to timestamp when all 100 numbered names exist. - SHA-256 verify abortable mid-hash; go-error clears stale summary stats + resets timer. Found via a 6-finder adversarially-verified transfer-pipeline audit (14 confirmed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 71d1377 commit 45273bc

6 files changed

Lines changed: 173 additions & 31 deletions

File tree

CHANGELOG.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,40 @@ All notable changes to PS5 Vault are documented here.
44

55
---
66

7+
## [2.4.9] — 2026
8+
9+
### Critical — Transfer safety (prevents data loss on Move)
10+
11+
A deep audit of the transfer pipeline found several cases where a **Move** could report
12+
success after an *incomplete* transfer and then delete the source. All are now closed:
13+
14+
- **Incomplete FTP download no longer deletes the source.** If any folder failed to list
15+
during the manifest walk (common under PS5 FTP load), the download silently skipped those
16+
files yet reported success — and a Move then deleted the source. The walk now retries
17+
failed directories and, if any still can't be read, fails the whole transfer so the source
18+
is preserved.
19+
- **Incomplete FTP upload no longer deletes the source.** Directories stored as NTFS
20+
junctions/symlinks were silently skipped during upload. They are now followed and uploaded.
21+
- **FTP uploads are size-verified.** After each file uploads, its size on the PS5 is checked
22+
against the local file; a short/truncated write is retried instead of being accepted.
23+
- **Same-place FTP Move is a no-op.** Moving/restoring a game to the exact path it already
24+
occupies (with Overwrite) no longer risks deleting the just-written folder.
25+
- **Overwrite that can't clear the old folder now aborts** instead of merging the new files
26+
on top of the old version.
27+
28+
### Bug Fixes
29+
30+
- Cross-drive local **Move** now runs the free-space pre-check (previously only Copy did).
31+
- FTP **download** uses extended-length paths, so very long PS5 game paths work on Windows.
32+
- FTP **download resumes**: files already fully present (matching size) are skipped.
33+
- PPSA-only layout **preview** now shows the version suffix, matching the folder actually created.
34+
- Free-space estimate ignores the "size unavailable" marker.
35+
- Speed-limited uploads no longer leak a file handle on a failed retry.
36+
- Cancelling a verified copy now aborts mid-hash promptly.
37+
- A failed transfer no longer shows the previous transfer's summary numbers.
38+
39+
---
40+
741
## [2.4.8] — 2026
842

943
### Bug Fixes — FTP scan UI (information loss)

help.js

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,13 @@
1111
<li><kbd>F1</kbd> opens this help at any time.</li>
1212
</ul>
1313
14-
<h3>What's New (v2.4.8)</h3>
14+
<h3>What's New (v2.4.9)</h3>
15+
<ul>
16+
<li><strong>Transfer safety</strong>: Move is now safe against partial transfers — an incomplete FTP upload or download can no longer report success and delete your only copy. Uploads are size-verified on the PS5, same-place moves are skipped, and overwrite aborts if it can't clear the old folder first.</li>
17+
<li><strong>Transfer fixes</strong>: cross-drive Move checks free space, very long PS5 paths download correctly on Windows, already-downloaded files are skipped on resume, and the PPSA-only destination preview now matches reality.</li>
18+
</ul>
19+
20+
<h3>v2.4.8</h3>
1521
<ul>
1622
<li><strong>FTP scan fixes</strong>: The SIZE column no longer gets stuck on a spinner — every game now resolves to its size, "0 B", or "—" (size unavailable), and the "sizing…" bar always finishes. Covers now appear live as they download, duplicate copies of a game show their own correct sizes, and large FTP libraries save reliably between sessions.</li>
1723
</ul>

main.js

Lines changed: 120 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -749,10 +749,18 @@ async function copyAndVerifyFile(srcPath, dstPath, progressCallback, cancelCheck
749749
await copyFileStream(srcE, dstE, progressCallback, cancelCheck);
750750
const ac = new AbortController();
751751
if (cancelCheck()) ac.abort();
752-
const [hSrc, hDst] = await Promise.all([
753-
hashFile(srcE, ac.signal),
754-
hashFile(dstE, ac.signal),
755-
]);
752+
// Poll cancelCheck during hashing so a mid-hash cancel of a large file aborts both
753+
// reads within ~200ms instead of running two full file reads to completion.
754+
const cancelPoll = setInterval(() => { if (cancelCheck()) ac.abort(); }, 200);
755+
let hSrc, hDst;
756+
try {
757+
[hSrc, hDst] = await Promise.all([
758+
hashFile(srcE, ac.signal),
759+
hashFile(dstE, ac.signal),
760+
]);
761+
} finally {
762+
clearInterval(cancelPoll);
763+
}
756764
if (hSrc === hDst) {
757765
try { const fd = await fs.promises.open(dstE, 'r+'); await fd.sync(); await fd.close(); } catch (_) {}
758766
if (srcStat) {
@@ -1152,6 +1160,11 @@ async function buildFtpManifest(ftpConfig, rootPath, progressCallback, cancelChe
11521160

11531161
const queue = []; // seeded by probe below
11541162
const visited = new Set([rootPath]);
1163+
// Dirs whose LIST never succeeded (even after reconnect/retries). For a real download
1164+
// walk (!sizeOnly) any such dir means the manifest is INCOMPLETE — we fail loudly so a
1165+
// Move never deletes the source after a partial transfer. sizeOnly stays best-effort.
1166+
const failedDirs = [];
1167+
const dirAttempts = new Map(); // dir -> LIST attempts, caps re-enqueues (no infinite loop)
11551168

11561169
// Event-based wake-up: replaces 5ms polling with zero-latency notification.
11571170
// Each time a new dir is pushed (or the last in-flight worker finishes),
@@ -1237,6 +1250,9 @@ async function buildFtpManifest(ftpConfig, rootPath, progressCallback, cancelChe
12371250
e.message.includes('ECONNRESET') ||
12381251
e.message.includes('closed')
12391252
);
1253+
const attempts = (dirAttempts.get(dir) || 0) + 1;
1254+
dirAttempts.set(dir, attempts);
1255+
let recovered = false;
12401256
if (isBrokenState) {
12411257
try { client.close(); } catch (_) {}
12421258
if (accessOpts_w) {
@@ -1248,18 +1264,27 @@ async function buildFtpManifest(ftpConfig, rootPath, progressCallback, cancelChe
12481264
await newClient.access(accessOpts_w);
12491265
client = newClient;
12501266
console.log('[FTP Manifest] Worker reconnected after client error');
1251-
// BUG FIX: re-enqueue the failed dir so its files are counted.
1252-
// Without this, a timed-out dir is permanently lost from the walk.
1253-
enqueueDir(dir);
1267+
recovered = true;
12541268
} catch (reconnErr) {
12551269
console.warn('[FTP Manifest] Worker reconnect failed:', reconnErr.message);
1256-
// BUG FIX: do NOT decrement inFlight here — finally does it.
1257-
// Previous code had an explicit inFlight-- + return which caused
1258-
// double-decrement with finally → inFlight went negative →
1259-
// all workers parked forever and sizing hung indefinitely.
1270+
// Do NOT decrement inFlight here — finally does it (a previous explicit
1271+
// inFlight-- caused a double-decrement → workers parked forever).
12601272
workerDead = true; // signal loop to break after finally runs
12611273
}
12621274
}
1275+
} else {
1276+
recovered = true; // client still usable — retry the dir
1277+
}
1278+
// Re-enqueue for another attempt (capped at 3 to avoid an infinite re-list
1279+
// loop on a persistently-failing dir); otherwise record it as failed so the
1280+
// manifest-incomplete guard below aborts a Move instead of deleting a source.
1281+
// visited.delete is required: enqueueDir() skips already-visited dirs, so the
1282+
// re-enqueue would silently no-op without it.
1283+
if (recovered && attempts < 3) {
1284+
visited.delete(dir);
1285+
enqueueDir(dir);
1286+
} else {
1287+
failedDirs.push(dir);
12631288
}
12641289
// fall through to finally (inFlight--, wakeWorkers)
12651290
}
@@ -1369,6 +1394,13 @@ async function buildFtpManifest(ftpConfig, rootPath, progressCallback, cancelChe
13691394
clients.forEach(c => { try { c.close(); } catch (_) {} });
13701395
}
13711396

1397+
// For a real download walk, a directory we never managed to list means the manifest
1398+
// is incomplete — fail loudly so downloadFtpFolder rethrows and a Move never deletes
1399+
// the source after a partial transfer. sizeOnly walks remain best-effort.
1400+
if (!sizeOnly && failedDirs.length > 0) {
1401+
throw new Error(`Manifest incomplete: failed to list ${failedDirs.length} dir(s), e.g. ${failedDirs[0]}`);
1402+
}
1403+
13721404
const result = { files, totalSize, fileCount: files.length, dirCount, fromCache: false };
13731405
sizeCache.set(cacheKey, result);
13741406
diskSizeCache[cacheKey] = { totalSize, fileCount: files.length, topLevelCount, cachedAt: Date.now() };
@@ -2399,11 +2431,22 @@ async function downloadFtpFolder(ftpConfig, remotePath, localPath, progressCallb
23992431
const localDest = path.join(localPath, ...fileEntry.relPath.split('/'));
24002432
await fs.promises.mkdir(toExtendedPath(path.dirname(localDest)), { recursive: true });
24012433

2434+
// Resume skip: if the destination already exists at the exact expected size,
2435+
// it was fully downloaded on a prior run — skip it (partial files differ in size
2436+
// and are still re-fetched). Mirrors the local copy resume behavior.
2437+
const _existing = await fs.promises.stat(toExtendedPath(localDest)).catch(() => null);
2438+
if (_existing && _existing.size === fileEntry.size && fileEntry.size > 0) {
2439+
bytesCopied += fileEntry.size;
2440+
progressCallback?.({ type: 'go-file-progress', fileRel: path.basename(fileEntry.remotePath), totalBytesCopied: bytesCopied, totalBytes: totalSize });
2441+
continue;
2442+
}
2443+
24022444
// Retry up to 5 attempts on transient FTP errors — also auto-reconnects on disconnect
24032445
let downloaded = false;
24042446
for (let attempt = 1; attempt <= 5 && !downloaded; attempt++) {
24052447
try {
2406-
await client.downloadTo(localDest, fileEntry.remotePath);
2448+
// Extended path so long PS5 game trees (>260 chars) don't fail on Windows.
2449+
await client.downloadTo(toExtendedPath(localDest), fileEntry.remotePath);
24072450
downloaded = true;
24082451
} catch (e) {
24092452
if (cancelCheck?.()) throw new Error('Cancelled');
@@ -2576,21 +2619,49 @@ async function uploadFtpRecursive(client, localPath, remotePath, progressCallbac
25762619
if (cancelCheck()) throw new Error('Cancelled');
25772620
const localItem = path.join(localPath, ent.name);
25782621
const remoteItem = path.posix.join(remotePath, ent.name);
2579-
if (ent.isFile()) {
2622+
// NTFS junctions/symlinks (PS5 Sc0/Sc1/-app game-data folders) classify as
2623+
// neither isFile() nor isDirectory() under withFileTypes on Windows. stat()
2624+
// follows the junction (as findAllParamJsons/listAllFilesWithStats do) — without
2625+
// this the whole subtree is silently skipped, producing an incomplete upload that
2626+
// is then reported as success (and the local source deleted on a Move).
2627+
const _st = (ent.isFile() || ent.isDirectory()) ? null : await fs.promises.stat(localItem).catch(() => null);
2628+
const isFile = ent.isFile() || (_st && _st.isFile());
2629+
const isDir = ent.isDirectory() || (_st && _st.isDirectory());
2630+
if (isFile) {
25802631
let uploaded = false;
25812632
let lastErr = null;
25822633
for (let attempt = 1; attempt <= 5 && !uploaded; attempt++) {
25832634
try {
25842635
if (speedLimitBps > 0) {
2585-
const rs = fs.createReadStream(localItem, { highWaterMark: bufBytes });
2586-
const throttle = new ThrottledStream(speedLimitBps);
2587-
// Forward stream errors so the transfer fails cleanly rather than hanging
2588-
rs.on('error', (e) => throttle.destroy(e));
2589-
rs.pipe(throttle);
2590-
await client.uploadFrom(throttle, remoteItem);
2636+
let rs, throttle;
2637+
try {
2638+
rs = fs.createReadStream(localItem, { highWaterMark: bufBytes });
2639+
throttle = new ThrottledStream(speedLimitBps);
2640+
// Forward stream errors so the transfer fails cleanly rather than hanging
2641+
rs.on('error', (e) => throttle.destroy(e));
2642+
rs.pipe(throttle);
2643+
await client.uploadFrom(throttle, remoteItem);
2644+
} finally {
2645+
// Release the fd/stream on every attempt (success or failure) so a failed
2646+
// retry doesn't leak a file handle or keep a detached pipe alive.
2647+
try { rs && rs.destroy(); } catch (_) {}
2648+
try { throttle && throttle.destroy(); } catch (_) {}
2649+
}
25912650
} else {
25922651
await client.uploadFrom(localItem, remoteItem);
25932652
}
2653+
// Verify the remote byte count matches local before declaring success.
2654+
// basic-ftp resolves uploadFrom on the server's 226 reply; PS5 FTP can 226 a
2655+
// short write, so confirm SIZE. If the server lacks SIZE (<0/throws), trust
2656+
// the 226 rather than failing forever.
2657+
{
2658+
const localSize = (await fs.promises.stat(localItem).catch(() => ({ size: -1 }))).size;
2659+
let remoteSize = -1;
2660+
try { remoteSize = await client.size(remoteItem); } catch (_) { remoteSize = -1; }
2661+
if (remoteSize >= 0 && localSize >= 0 && remoteSize !== localSize) {
2662+
throw new Error(`size mismatch for ${ent.name}: local ${localSize} remote ${remoteSize}`);
2663+
}
2664+
}
25942665
uploaded = true;
25952666
} catch (e) {
25962667
lastErr = e;
@@ -2622,7 +2693,7 @@ async function uploadFtpRecursive(client, localPath, remotePath, progressCallbac
26222693
if (!uploaded) throw new Error(`FTP upload failed for file: ${ent.name}${lastErr?.message || 'unknown error'}`);
26232694
const size = (await fs.promises.stat(localItem).catch(() => ({ size: 0 }))).size || 0;
26242695
progressCallback?.({ type: 'go-file-complete', fileRel: ent.name, totalBytesCopied: size, totalBytes: totalSize });
2625-
} else if (ent.isDirectory()) {
2696+
} else if (isDir) {
26262697
await uploadFtpRecursive(client, localItem, remoteItem, progressCallback, cancelCheck, totalSize, speedLimitBps, bufBytes, accessConfig);
26272698
}
26282699
}
@@ -2898,10 +2969,12 @@ async function doEnsureAndPopulate(event, opts) {
28982969

28992970
const results = [];
29002971
// Pre-compute once so progressFn doesn't call .reduce() on every IPC event
2901-
const _grandTotalBytes = items.reduce((s, x) => s + (x.totalSize || 0), 0);
2972+
// Sum only positive sizes — the -1 "size unavailable" sentinel (and 0) must not
2973+
// subtract from or pollute the free-space estimate.
2974+
const _grandTotalBytes = items.reduce((s, x) => s + (x.totalSize > 0 ? x.totalSize : 0), 0);
29022975
try {
2903-
// ── Free-space pre-check ─────────────────────────────────────────────────
2904-
if ((action === 'copy' || action === 'copy-fast') && _grandTotalBytes > 0) {
2976+
// ── Free-space pre-check (also covers Move: a cross-device move is a full copy) ──
2977+
if ((action === 'copy' || action === 'copy-fast' || action === 'move') && _grandTotalBytes > 0) {
29052978
try {
29062979
if (!ftpConfig && !ftpDestConfig) {
29072980
// Local-to-local: use OS free space API
@@ -3167,7 +3240,13 @@ async function doEnsureAndPopulate(event, opts) {
31673240
try {
31683241
await dc.access({ host: ftpDestConfig.host, port: parseInt(ftpDestConfig.port), user: ftpDestConfig.user || 'anonymous', password: ftpDestConfig.pass || '', secure: false });
31693242
await ftpDeleteRecursive(dc, ftpDestRemotePath);
3170-
} catch (e) { console.warn('[FTP] Pre-overwrite delete failed:', e.message); }
3243+
} catch (e) {
3244+
// ftpDeleteRecursive no-ops on a missing target, so a thrown error here is
3245+
// a real failure (shallow/protected path or remove error). Abort the item
3246+
// rather than uploading on top of the old version (silent merge/corruption).
3247+
console.warn('[FTP] Pre-overwrite delete failed:', e.message);
3248+
throw new Error('Overwrite aborted — could not clear existing folder: ' + e.message);
3249+
}
31713250
finally { dc.close(); }
31723251
}
31733252
} else {
@@ -3186,11 +3265,15 @@ async function doEnsureAndPopulate(event, opts) {
31863265
applyFtpPassive(rc, ftpDestConfig);
31873266
try {
31883267
await rc.access({ host: ftpDestConfig.host, port: parseInt(ftpDestConfig.port), user: ftpDestConfig.user || 'anonymous', password: ftpDestConfig.pass || '', secure: false });
3268+
let renamed = false;
31893269
for (let n = 1; n <= 100; n++) {
31903270
const tryR = ftpDestRemotePath + ` (${n})`;
31913271
try { await rc.cd(tryR); await rc.list(); }
3192-
catch (_) { ftpDestRemotePath = tryR; finalTarget = finalTarget + ` (${n})`; break; }
3272+
catch (_) { ftpDestRemotePath = tryR; finalTarget = finalTarget + ` (${n})`; renamed = true; break; }
31933273
}
3274+
// All 100 numbered candidates exist — fall back to a timestamp suffix so we
3275+
// never merge into the original conflicting folder (matches the catch below).
3276+
if (!renamed) { const ts = Date.now(); ftpDestRemotePath += ` (${ts})`; finalTarget += ` (${ts})`; }
31943277
} catch (e) {
31953278
const ts = Date.now(); ftpDestRemotePath += ` (${ts})`; finalTarget += ` (${ts})`;
31963279
} finally { rc.close(); }
@@ -3228,6 +3311,18 @@ async function doEnsureAndPopulate(event, opts) {
32283311
if (ftpConfig && action === 'move' &&
32293312
ftpConfig.host === ftpDestConfig.host &&
32303313
String(ftpConfig.port) === String(ftpDestConfig.port)) {
3314+
// Guard: when the destination resolves to the SAME server path as the
3315+
// source (e.g. moving/restoring a game in-place with overwrite), a
3316+
// server-side rename is a no-op and the download+upload+delete fallback
3317+
// would delete the just-restored game. Treat as already-satisfied.
3318+
const _nSrc = ('/' + String(srcFolder).replace(/^\/+/, '')).replace(/\/\/+/g, '/').replace(/\/$/, '');
3319+
const _nDst = ('/' + String(remotePath).replace(/^\/+/, '')).replace(/\/\/+/g, '/').replace(/\/$/, '');
3320+
if (_nSrc === _nDst) {
3321+
console.warn('[FTP] Same-server move: source === destination — skipping (no-op):', _nDst);
3322+
results.push({ item: safeGameName, target: finalTarget, moved: true, skipped: true, source: originalSrcFolder, safeGameName, totalSize: itemTotalBytes });
3323+
progressFn({ type: 'go-file-complete', fileRel: path.basename(finalTarget), totalBytesCopied: 0, totalBytes: 0 });
3324+
continue;
3325+
}
32313326
const client = new ftp.Client(15000); // 15s: rename is fast but needs round-trips
32323327
applyFtpPassive(client, ftpConfig);
32333328
let renameOk = false;

package-lock.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "ps5-vault",
3-
"version": "2.4.8",
3+
"version": "2.4.9",
44
"description": "PS5 game library manager \u2014 scan, organise and transfer games via USB or FTP",
55
"main": "main.js",
66
"author": "PS5 Vault",

0 commit comments

Comments
 (0)