Skip to content

Resource leak: createTSDB does not close TSDB when db.Compact() fails #7557

@sandy2008

Description

@sandy2008

Summary

In pkg/ingester/ingester.go, the createTSDB function opens a TSDB via tsdb.Open() (line 2936) and later calls db.Compact() (line 2966). If Compact() returns an error, the function returns immediately without calling db.Close(), leaking the head, WAL, and mmap file handles.

Code path

// line 2936 – TSDB opened, resources acquired
db, err := tsdb.Open(udir, ...)
if err != nil {
    return nil, errors.Wrapf(err, "failed to open TSDB: %s", udir)
}
db.DisableCompactions()

// line 2966 – Compact fails → return without Close → leak
err = db.Compact(context.TODO())
if err != nil {
    return nil, errors.Wrapf(err, "failed to compact TSDB: %s", udir)  // ← db not closed
}

Impact

Each leaked TSDB holds:

  • Head in-memory data structures
  • Open WAL segment file descriptors
  • mmap'd chunk file descriptors

In a multi-tenant ingester under compaction pressure, repeated failures on different tenant TSDBs could exhaust file descriptors (hitting ulimit -n) and memory, eventually crashing the ingester process.

Proposed fix

Call db.Close() before returning the error from the Compact() failure path:

err = db.Compact(context.TODO())
if err != nil {
    _ = db.Close()
    return nil, errors.Wrapf(err, "failed to compact TSDB: %s", udir)
}

This follows the established resource-cleanup pattern used elsewhere in the codebase (e.g., closeAllTSDB).

Acceptance criteria

  • db.Close() is called on the Compact() error path
  • No existing tests regress
  • Unit test or review confirms no other early-return paths between tsdb.Open and userDB.db = db leak the handle

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions