Skip to content

Commit 3c631dc

Browse files
committed
state, internal/dao, util: revert hot-path fsync, fix leak and doc inaccuracy
- util.AtomicWriteFile: drop the fsync added last round. On macOS, Go's File.Sync issues fcntl(F_FULLFSYNC), which measured ~140x slower than the plain write here (confirmed on this machine: ~4.3ms vs ~30µs/op) -- and this helper runs on Hermit's "exec" hot path via dao.UpdatePackage. The data it protects is a regenerable cache (etag + check timestamp), so losing it to a crash just costs one extra upstream check; that's not worth paying this cost on every invocation. Documented as a deliberate omission. - state.extract: the "copy manifest referred files" loop can also fail after archive.Extract has already published p.Dest, the same condition the previous commit fixed for the EventUnpack trigger a few lines below it -- missed because it wasn't the line called out by review. Now cleaned up with util.RemoveAllAtomic here too, otherwise a retry of the same package is permanently wedged behind archive.Extract's "destination already exists". - internal/dao.UpdatePackage: correct a doc comment claiming the etag/ checked-at interleave risk was "not introduced by the two-file split" -- the single-JSON-file format it replaced wrote both fields in one atomic rename, so this specific mismatched pairing is in fact newly possible. Still benign: worst case is a one-cycle-stale check time that self-corrects.
1 parent 3a56197 commit 3c631dc

3 files changed

Lines changed: 25 additions & 18 deletions

File tree

internal/dao/dao.go

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -140,10 +140,12 @@ func (d *DAO) readCheckedAt(pkgRef string) (time.Time, error) {
140140
// so two UpdatePackage calls for the same package racing each other can
141141
// interleave: caller A's etag write can be immediately followed by caller
142142
// B's checked-at write, leaving a GetPackage that reads in between with A's
143-
// etag paired with B's checked-at time. This is a pre-existing risk carried
144-
// over from the single-JSON-file format this replaced (which had the same
145-
// last-writer-wins exposure across the two logical fields, just within one
146-
// file); it is not introduced by the two-file split.
143+
// etag paired with B's checked-at time. The single-JSON-file format this
144+
// replaced wrote both fields in one atomic rename, so this specific
145+
// mismatched pairing is newly possible with the two-file split -- but it's
146+
// still benign: at worst it under- or over-estimates how recently a
147+
// concurrently-updated package was checked by one update cycle, which
148+
// self-corrects on the next check.
147149
func (d *DAO) UpdatePackage(pkgRef string, pkg *Package) error {
148150
checkedAt := pkg.UpdateCheckedAt
149151
if checkedAt.IsZero() {

state/state.go

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -454,18 +454,22 @@ func (s *State) extract(b *ui.Task, p *manifest.Package) error {
454454
if err != nil {
455455
return errors.WithStack(err)
456456
}
457+
// From here on, p.Dest is already published: archive.Extract renames it
458+
// into place before returning, not after finalise() runs. That means an
459+
// unlocked reader could be looking at it, and archive.Extract itself
460+
// refuses to extract into a p.Dest that already exists -- so any failure
461+
// below must clean it up the same reader-safe way as everywhere else in
462+
// this package, or a retry of this package is permanently wedged behind
463+
// "destination already exists".
457464
// Copy manifest referred files
458465
for _, file := range p.Files {
459466
err = vfs.CopyFile(file.FS, file.FromPath, file.ToPath)
460467
if err != nil {
468+
_ = util.RemoveAllAtomic(p.Dest)
461469
return errors.WithStack(err)
462470
}
463471
}
464472
if _, err = p.Trigger(b, manifest.EventUnpack); err != nil {
465-
// p.Dest is already published (archive.Extract renames it into place
466-
// before returning), so an unlocked reader could be looking at it --
467-
// remove it the same reader-safe way as everywhere else in this
468-
// package rather than deleting it out from under them entry by entry.
469473
_ = util.RemoveAllAtomic(p.Dest)
470474
return errors.WithStack(err)
471475
}

util/atomicfile.go

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,17 @@ import (
1111
// file in the same directory, then renamed into place. Unlike os.WriteFile,
1212
// which truncates the existing file before writing, a concurrent reader can
1313
// never observe an empty or partially-written file.
14+
//
15+
// This deliberately does not fsync the temp file before renaming it: doing so
16+
// closes a narrow crash-durability gap (a crash between a successful-looking
17+
// write and the underlying data actually reaching disk could otherwise leave
18+
// path pointing at a zero-length or truncated file), but on macOS, Go's
19+
// File.Sync issues fcntl(F_FULLFSYNC), which is roughly two orders of
20+
// magnitude slower than a plain write -- and this is called from
21+
// internal/dao.UpdatePackage on Hermit's "exec" hot path. The data this
22+
// protects (a cached etag and check timestamp) is not authoritative state:
23+
// losing it to a crash just costs one extra upstream check on the next run,
24+
// which doesn't justify that cost on every invocation.
1425
func AtomicWriteFile(path string, data []byte, perm os.FileMode) error {
1526
dir := filepath.Dir(path)
1627
tmp, err := os.CreateTemp(dir, filepath.Base(path)+".tmp-*")
@@ -22,20 +33,10 @@ func AtomicWriteFile(path string, data []byte, perm os.FileMode) error {
2233
defer os.Remove(tmpPath)
2334

2435
_, writeErr := tmp.Write(data)
25-
var syncErr error
26-
if writeErr == nil {
27-
// Without this, a crash shortly after Rename can leave path pointing
28-
// at a temp file the filesystem never flushed, ie. a zero-length or
29-
// truncated file, despite the rename itself being durable.
30-
syncErr = tmp.Sync()
31-
}
3236
closeErr := tmp.Close()
3337
if writeErr != nil {
3438
return errors.WithStack(writeErr)
3539
}
36-
if syncErr != nil {
37-
return errors.WithStack(syncErr)
38-
}
3940
if closeErr != nil {
4041
return errors.WithStack(closeErr)
4142
}

0 commit comments

Comments
 (0)