Soft and hard storage quota

[Raubzeug]

Context

Frontend issue: ydb-platform/ydb-embedded-ui#4239

The Embedded UI needs to display:

• current quota-accounted storage usage;
• the hard quota that activates write blocking;
• the soft recovery threshold below which blocking is cleared;
• whether database writes are currently blocked.

The active blocked state cannot be inferred from usage alone. While usage is between the soft and hard thresholds, writes may be either allowed or blocked depending on the previous state.

Problem

Most database UI surfaces receive their data from /viewer/tenantinfo.

/viewer/describe is not requested on the default Database view. It is currently used only by the optional new Storage view after the internal Storage tab is selected and the required feature gates are enabled. Relying on describe therefore does not cover:

• the default Database overview;
• the legacy Storage view;
• the databases list.

Issuing a separate describe request for every database in the list would also create an undesirable N+1 request pattern.

/viewer/tenantinfo already exposes storage usage and quota-related fields, including:

• TablesStorage[].Size;
• TablesStorage[].SoftQuota;
• TablesStorage[].HardQuota;
• DatabaseQuotas.

SchemeShard also maintains the authoritative hysteresis state in DomainState.DiskQuotaExceeded, but tenantinfo does not appear to expose an equivalent field.

Requested investigation and changes

Active blocked state

Verify whether /viewer/tenantinfo already exposes an authoritative equivalent of DomainState.DiskQuotaExceeded.

If it does not, expose an optional blocked-state field for every returned tenant. The field should represent the persisted SchemeShard state, not a value recomputed from current usage.

The contract should let clients distinguish:

• usage between soft and hard with writes allowed;
• usage between soft and hard with writes still blocked;
• usage above hard with blocking active;
• recovery after usage drops below soft.

Please preserve compatibility with older backends. Clients should be able to distinguish an unsupported or unavailable field from a known false value, either through field presence or an explicit API version/capability.

Soft and hard quota values

Verify that TablesStorage[].SoftQuota and TablesStorage[].HardQuota contain the effective thresholds used by quota enforcement.

In particular, verify the response when separate storage-kind quotas are enabled. The current implementation appears to construct TablesStorage before populating storageQuotasByType from DatabaseQuotas.storage_quotas.

If this affects the returned values, fix the ordering or otherwise make the per-kind thresholds reliable.

Quota-accounted usage

Verify that TablesStorage[].Size represents the same usage value that SchemeShard compares with the quota in every supported mode:

• overall database quota;
• separate storage-kind quotas;
• tables and topics.

Viewer appears to use topic DataSize in at least one fallback path, while quota enforcement may use AccountSize. If these values can differ, expose an explicit quota-accounted usage value rather than requiring clients to reproduce SchemeShard accounting rules.

Effective semantics

Verify that clients can determine which thresholds are effective when:

• only the hard quota is configured;
• only the soft quota is configured;
• both are configured;
• neither is configured;
• overall and per-storage-kind quota fields are present.

If the existing fields already provide an unambiguous contract, document their semantics and add regression coverage rather than introducing duplicate fields. Otherwise, expose the smallest backward-compatible set of effective values required by clients.

Acceptance criteria

• A single /viewer/tenantinfo request provides, for each tenant:
  • quota-accounted usage in bytes;
  • the applicable soft recovery threshold;
  • the applicable hard write-block threshold;
  • the authoritative active blocked state.
• The blocked state follows SchemeShard hysteresis and is not recomputed by Viewer from current usage.
• Per-storage-kind thresholds and usage are correct when separate quotas are enabled.
• Topic usage follows the same accounting semantics as quota enforcement.
• Unsupported or unavailable values are distinguishable from real zero and false values.
• Existing clients remain compatible.
• Viewer API version or capability metadata is updated if necessary.
• Tests cover:
  • overall quotas;
  • separate storage-kind quotas;
  • soft-only, hard-only, and no-quota configurations;
  • usage below soft, between thresholds, and above hard;
  • both blocked and unblocked states between the thresholds;
  • quota-accounted topic storage.