Expose subgraph logs via subgraph GraphQL

[fordN]

This PR introduces a subgraph log storage and querying system for Graph Node. Subgraph logs can be queried through the GraphQL API via a new _logs field. The implementation supports multiple storage backends (File, Elasticsearch, Loki) with a consistent query interface.

What's new

GraphQL Query API

• New _logs query field on all subgraph deployments
• Filter by log level, timestamp range, and text search
• Structured log entries with metadata (source location, arguments)
• Pagination via first/skip and sort order via orderDirection

Storage Backends

• File: JSON Lines format, one file per subgraph (good for development)
• Elasticsearch: Enterprise search and analytics (production)
• Loki: Grafana log aggregation, with basic auth support (production)
• Disabled: Default when no [log_store] section is configured

Configuration via graph-node.toml

Log store is configured through a [log_store] section in the TOML config file, following the same pattern as [store], [chains], and [deployment].

Architecture

• LogDrain: Write-side sink for each backend (File, Loki, Elasticsearch)
• LogStore: Read-side query interface for each backend
• LoggerFactory: Refactored for multi-backend log routing
• Uses slog::Level directly (no custom LogLevel enum)
• Parse failures are logged as warnings, not silently dropped

Examples

Querying logs

{
  _logs(
    level: ERROR
    search: "timeout"
    from: "2024-01-15T00:00:00Z"
    to: "2024-01-16T00:00:00Z"
    first: 100
    orderDirection: desc
  ) {
    id
    timestamp
    level
    text
    arguments { key value }
    meta { module line column }
  }
}

Configuring log store backends

File-based (development):

[log_store]
backend = "file"
directory = "/var/log/graph-node/subgraph-logs"
retention_hours = 72

Loki (production):

[log_store]
backend = "loki"
url = "https://loki.example.com"
tenant_id = "my-tenant"
username = "user"
password = "secret"

Elasticsearch (production):

[log_store]
backend = "elasticsearch"
url = "http://localhost:9200"
username = "elastic"
password = "changeme"
index = "subgraph"
timeout_secs = 10

[lutter]

One thing I wonder about: should this be configured via environment variables or through graph-node.toml? I would lean towards the latter since it's a configuration that is unlikely to change often, and having it in the config file would let's us express more complicated configuration (though admittedly, right now the config is not very complex)

[dwerner]

Lgtm! I have a few questions/suggestions, but yolo

[DaMandal0rian]

Great work on this! We're planning to use the Loki drain in production as part of our ES decommission.

One blocker: our production Loki endpoint (loki-logs.thegraph.com) requires HTTP basic auth. The current LokiDrain implementation only supports GRAPH_LOG_STORE_LOKI_URL and GRAPH_LOG_STORE_LOKI_TENANT_ID — no credentials.

Could we add support for basic auth? Something like:

GRAPH_LOG_STORE_LOKI_USER=<username>
GRAPH_LOG_STORE_LOKI_PASSWORD=<password>

This would allow the reqwest client to attach an Authorization: Basic ... header on push requests. Without this, we'd need to deploy a sidecar proxy or an unauthenticated push endpoint, which adds operational complexity.

[fordN]

Great work on this! We're planning to use the Loki drain in production as part of our ES decommission.

One blocker: our production Loki endpoint (loki-logs.thegraph.com) requires HTTP basic auth. The current LokiDrain implementation only supports GRAPH_LOG_STORE_LOKI_URL and GRAPH_LOG_STORE_LOKI_TENANT_ID — no credentials.

Could we add support for basic auth? Something like:

GRAPH_LOG_STORE_LOKI_USER=<username>
GRAPH_LOG_STORE_LOKI_PASSWORD=<password>

This would allow the reqwest client to attach an Authorization: Basic ... header on push requests. Without this, we'd need to deploy a sidecar proxy or an unauthenticated push endpoint, which adds operational complexity.

Added! ✔️

[fordN]

One thing I wonder about: should this be configured via environment variables or through graph-node.toml? I would lean towards the latter since it's a configuration that is unlikely to change often, and having it in the config file would let's us express more complicated configuration (though admittedly, right now the config is not very complex)

Yeah, might as well keep configs in the toml file, easier to manage. I've made the change to that effect in commit 64fa5278.