This page explains how to configure Summoner.server in a way that is understandable to readers who do not want to study the implementation. Each setting is described by its purpose, behavior, default value, and the practical consequences of changing it.
A Summoner server is a TCP relay: clients connect to a host and port, send newline-delimited messages, and the server forwards messages to other connected clients. Most configuration exists to answer three operational questions:
- Where does the server listen (
host,port), and which backend runs it (version)? - What is recorded for observability (
logger)? - How does the server stay stable under load (
hyper_parametersandhyper_parameters.backpressure_policy)?
Note
Loading & precedence
-
You can pass a configuration dictionary directly to
server.run(...)asconfig_dict, or provide a JSON file path viaconfig_path. -
The server runs using either the Python backend (asyncio) or the Rust backend (Tokio wrapped by Python).
-
Host/Port precedence
- Rust backend: prefers the values in the config file (
host,port) and falls back to the arguments ofrun(host, port)when not present. - Python backend: binds to
run(host, port); top-levelhostandportin the config are not used for binding in the Python path.
- Rust backend: prefers the values in the config file (
-
- What hyper parameters control
hyper_parameters.connection_buffer_sizehyper_parameters.command_buffer_sizehyper_parameters.control_channel_capacityhyper_parameters.queue_monitor_capacityhyper_parameters.rate_limit_msgs_per_minutehyper_parameters.client_timeout_secshyper_parameters.timeout_check_interval_secshyper_parameters.accept_error_backoff_mshyper_parameters.quarantine_cooldown_secshyper_parameters.quarantine_cleanup_interval_secshyper_parameters.throttle_delay_mshyper_parameters.flow_control_delay_mshyper_parameters.worker_threads
-
- How backpressure protects the server
hyper_parameters.backpressure_policy.enable_throttlehyper_parameters.backpressure_policy.throttle_thresholdhyper_parameters.backpressure_policy.enable_flow_controlhyper_parameters.backpressure_policy.flow_control_thresholdhyper_parameters.backpressure_policy.enable_disconnecthyper_parameters.backpressure_policy.disconnect_threshold
Purpose: The parameter host is the network address the server listens on.
"127.0.0.1"means only local clients can connect (development on one machine)."0.0.0.0"means the server accepts connections from other machines that can reach it (LAN or cloud deployment).
If host is not provided, the default is "127.0.0.1".
{ "host": "0.0.0.0", "port": 8888 }Purpose: The parameter port is the TCP port number the server listens on.
If port is not provided, the default is 8888.
Purpose: The parameter version selects which server backend runs.
"python"selects the Python asyncio server."rust"selects the newest available Rust backend installed in the environment."rust_vX.Y.Z"pins to a specific Rust backend version when that module is installed (for example"rust_v1.1.0").
On Windows, the server runs the Python backend.
Logging is how the server answers basic production questions such as: "Who connected?", "What load did the server observe?", "Which clients are being throttled?", and "Why did a client disconnect?".
The configuration controls two destinations:
- Console logs (visible in terminal output or container logs)
- File logs (persisted on disk)
A key operational property is that the Rust backend initializes the logger once per process. Changing the config values requires restarting the server process to change logging behavior.
The Rust backend formats console logs as readable lines containing:
- a timestamp
- the server name
- the log level
- the message
For file logs, the Rust backend supports:
- plain text lines, or
- JSON Lines (one JSON object per line), where the
messagefield is itself either JSON (if the logged string parses as JSON) or a string (if it does not).
This makes it possible to ingest logs into standard enterprise tooling while also keeping console output readable.
Purpose: The parameter log_level sets the minimum severity that is recorded.
Config path: logger.log_level.
The parameter log_level determines how "chatty" the server is allowed to be in logs. Higher verbosity is helpful during debugging, while lower verbosity keeps production logs smaller.
We recommend choosing the lowest verbosity that still answers your operational questions.
- Use
"INFO"for most deployments (connections, disconnections, warnings, backpressure actions). - Use
"DEBUG"for short investigations (it can generate a lot of volume under load). - Use
"WARNING"or"ERROR"only if you already have good observability elsewhere and want to minimize log volume.
If not provided, the default is "DEBUG".
"DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"
Purpose: The parameter enable_console_log turns console logging on or off.
Config path: logger.enable_console_log.
The parameter enable_console_log determines whether the server prints logs to the terminal (useful for local runs and container logs).
Keep this enabled in local development and in containerized environments where stdout/stderr are collected by the platform. In long-running services with dedicated file shipping, you may choose to disable console logs to reduce noise.
If not provided, the default is true.
Purpose: The parameter console_log_format controls the console log line format for the Python backend.
Config path: logger.console_log_format.
The parameter console_log_format determines how console logs are formatted on the Python backend (colors, timestamp layout, and so on).
If you are using the Python backend, this is where you tune readability (colors, fields, timestamp layout). If you are using the Rust backend, this setting is typically left in the config for consistency across environments, even though it does not affect Rust console output.
If omitted, the Python backend uses its default formatting.
The Rust backend prints a fixed console format (timestamp, name, level, message). The console_log_format string does not change Rust console formatting.
Purpose: The parameter enable_file_log turns file logging on or off.
Config path: logger.enable_file_log.
The parameter enable_file_log determines whether the server also writes logs to disk (useful for long-running deployments where terminal logs are not retained).
Enable file logging when you want durable logs on disk (for example, for post-incident review, auditing, or environments where stdout is not retained). If you already rely on centralized logging from console output, you may keep this disabled.
If not provided, the default is false.
Purpose: The parameter log_file_path is the directory where log files are written when enable_file_log is true.
Config path: logger.log_file_path.
The parameter log_file_path determines where log files are written if file logging is enabled. If this is empty, the server writes the log file next to where it was started.
Use an explicit path in production so logs end up where your tooling expects them (for example, a mounted volume or a standard system log directory). If you run multiple servers on one machine, prefer separate directories per service or per environment.
If omitted or set to an empty string, the Rust backend writes a log file in the current working directory.
The file name is derived from the server name, with dots replaced by underscores.
Purpose: The parameter enable_json_log selects whether file logs are written as JSON Lines or as plain text.
Config path: logger.enable_json_log.
The parameter enable_json_log determines whether file logs are written as structured JSON lines instead of plain text. JSON logs are easier to search, parse, and ingest into centralized logging systems.
Enable JSON logs when logs are intended for ingestion (ELK, Datadog, Splunk, CloudWatch, etc.). Keep plain text logs when logs are primarily read by humans on the host.
If not provided, the default is false.
JSON logs are the preferred format when logs are ingested into a centralized logging system, because fields like timestamp, level, and server name become structured data.
Purpose: The parameter log_format controls the file log line format for the Python backend when JSON logging is not enabled.
Config path: logger.log_format.
The parameter log_format determines how file logs are formatted on the Python backend when JSON logging is not enabled.
Tune this only if you are using the Python backend and you want file logs to match an existing organization standard. For the Rust backend, keep this value for config consistency across backends.
If omitted, the Python backend uses its default formatting.
The Rust backend uses a fixed format for plain text file logs.
Purpose: The parameter date_format controls timestamp formatting for console and file logs.
Config path: logger.date_format.
The parameter date_format determines how timestamps look in logs (for example, including milliseconds).
If you correlate server logs with other systems (load balancers, clients, infrastructure), consider using a timestamp format with millisecond precision and stable ordering. If you rely on log ingestion, choose a format that your pipeline parses reliably.
If not provided, the default is "%Y-%m-%d %H:%M:%S.%3f" (date, time, and milliseconds).
Purpose: The parameter max_file_size configures log rotation size for the Python backend.
Config path: logger.max_file_size.
The parameter max_file_size determines the maximum size of a log file before rotation on the Python backend.
Set this based on how quickly logs grow in your environment. For high-traffic services, a smaller rotation size avoids very large files but increases the number of rotated files created.
If omitted, the Python backend uses its default rotation size.
The Rust backend writes to a single file per server name using the configured path. Rotation by file size is not applied by the Rust backend.
Purpose: The parameter backup_count configures how many rotated log files are kept by the Python backend.
Config path: logger.backup_count.
The parameter backup_count determines how many rotated log files are kept on the Python backend.
Choose a retention count that matches your operational needs (for example, keeping enough history to cover the typical time between incidents and detection). If you ship logs elsewhere, you can keep this low.
If omitted, the Python backend uses its default retention count.
The Rust backend does not apply size-based rotation or retention counts.
Purpose: The parameter log_keys reduces the amount of message content written to logs by keeping only selected fields.
Config path: logger.log_keys.
The parameter log_keys controls how much of the message content is written to disk when JSON file logging is enabled. This is commonly used to keep logs lightweight or avoid writing large or sensitive payloads.
Use this setting to balance observability with volume and confidentiality:
- If payloads can be large,
log_keyskeeps logs small and stable. - If payloads can contain sensitive data,
log_keys: []can prevent accidental persistence of content. - If you need request tracing, include stable identifiers such as message IDs, user IDs, or types.
If not provided, the default is null, which means "do not filter".
[!TIP] [For developers] How filtering works
When filtering is enabled, the server keeps:
_version(if present)- selected keys inside
_payloadand_typeThis preserves stable identifiers and message classification while avoiding large or sensitive payload fields in logs.
"log_keys": nullLogs full content.
"log_keys": []Logs only minimal structure (useful when payloads are sensitive or very large).
"log_keys": ["msg_id", "type", "user_id"]Logs only the listed keys inside _payload and _type, plus _version.
The hyper_parameters section contains the operational limits and timing constants that keep the server responsive under normal traffic and predictable under stress.
In practice, these settings control four things:
- How much short-term burstiness the server can absorb (buffers).
- How much one client is allowed to consume (rate limits and idle timeouts).
- How the server recovers from abnormal conditions (accept backoff and quarantine).
- How strongly the server slows clients down when it needs to protect itself (throttle and flow-control delays, plus runtime parallelism).
Most deployments can start with defaults. You typically adjust hyper parameters when you observe one of the following: dropped messages due to bursts, clients that are too noisy, high CPU usage, or frequent backpressure actions in otherwise expected usage.
Purpose: The parameter connection_buffer_size sets the capacity of the global buffer that stores per-client load reports flowing into the backpressure monitoring pipeline.
Config path: hyper_parameters.connection_buffer_size.
The parameter connection_buffer_size determines how many "this client is getting busy" updates the server can temporarily store while it decides whether to slow someone down. If this buffer fills up, the server may stop recording some of these busy updates.
If you expect large rooms or frequent bursts, increase this buffer so the server can continue "seeing" load changes while it is busy. If you keep this small, the backpressure monitor may miss some short spikes because updates are dropped when the buffer is full.
If omitted, it defaults to the value 128.
Higher values allow more clients to report load at once during bursts. Lower values cause reports to be dropped earlier when the system is busy.
Purpose: The parameter command_buffer_size sets the capacity of the global buffer that stores backpressure actions emitted by the monitor before the main loop applies them.
Config path: hyper_parameters.command_buffer_size.
The parameter command_buffer_size determines how many "server reactions" can be queued up at once (examples: "slow this client down", "pause this client", "disconnect this client"). If this buffer fills up, some reactions may be delayed or not queued.
If you notice that backpressure actions are slow to take effect during bursts, increasing this buffer can help the server queue more reactions at once. If you keep it small, the monitor may be forced to skip or delay some actions during extreme spikes.
If omitted, it defaults to the value 32.
Higher values allow more pending "slow down" or "disconnect" actions to queue during bursts. Lower values constrain how many actions can accumulate.
Purpose: The parameter control_channel_capacity sets the capacity of the per-client buffer used to deliver control signals to a specific client session task (throttle and flow control).
Config path: hyper_parameters.control_channel_capacity.
The parameter control_channel_capacity determines, per client, how many "slow down / pause" signals the server can stack up for that one client. If this fills up for a client, additional slow-down signals for that client may not get through until earlier ones are processed.
If a client oscillates between normal and overloaded conditions, a slightly larger buffer can keep control signals flowing smoothly. If this is too small, control signals may fail to enqueue when the client is already under pressure, which makes the response less consistent.
If omitted, it defaults to the value 8.
Higher values allow multiple control signals to queue for a client under unstable load. Lower values make the control channel more immediate, but signals may be skipped if the client is already saturated.
Purpose: The parameter queue_monitor_capacity sets the capacity of the per-client buffer used to stage local load measurements before forwarding them into the global reporting buffer.
Config path: hyper_parameters.queue_monitor_capacity.
The parameter queue_monitor_capacity determines, per client, how many "how busy am I?" measurements can be temporarily stored before being forwarded to the server's global slowdown logic. If this fills up, some measurements may be skipped until the buffer drains.
This buffer mainly matters during short bursts. If you expect clients to broadcast to many peers in bursts, increasing it can help preserve a smoother stream of load measurements. If it is too small, the server may skip some measurements while the client is busy, which reduces the monitor's visibility into short spikes.
If omitted, it defaults to the value 100.
Higher values allow each client task to record short spikes in load without blocking itself. Lower values reduce memory use, but measurements may be skipped during spikes.
Purpose: The parameter rate_limit_msgs_per_minute sets a per-client message rate limit; if a client exceeds this rate, the server warns the client and does not broadcast excess messages.
Config path: hyper_parameters.rate_limit_msgs_per_minute.
The parameter rate_limit_msgs_per_minute determines how many messages one client is allowed to send per minute. If a client goes above this, the server warns them and ignores extra messages instead of letting them flood everyone.
Set this based on the expected client behavior. For interactive chat, users can legitimately send bursts. For automated clients, a misconfiguration can generate sustained high rates. This limit provides a predictable ceiling on how much one client can consume.
If omitted, it defaults to the value 300.
Purpose: The parameter client_timeout_secs disconnects clients that have been inactive for the specified number of seconds.
Config path: hyper_parameters.client_timeout_secs.
The parameter client_timeout_secs determines how long a client can stay silent before the server assumes it is idle and drops the connection (to avoid dead connections piling up).
Choose a timeout that matches your environment's connection patterns. In production, this prevents dead or abandoned connections from accumulating. If you have receive-only clients, ensure they send an occasional heartbeat, or set this to null if idle connections are expected and safe.
If omitted, it defaults to the value 300.
If set to null, inactivity timeout is disabled.
Purpose: The parameter timeout_check_interval_secs controls how often the server checks whether clients have been idle for too long.
Config path: hyper_parameters.timeout_check_interval_secs.
The parameter timeout_check_interval_secs determines how often the server checks for idle clients. A smaller value detects dead connections faster; a larger value reduces background checking work.
If you need faster cleanup of dead connections, lower this value. If you have extremely large numbers of clients and want to minimize periodic overhead, a higher value can be acceptable, at the cost of slower timeout enforcement.
If omitted, it defaults to the value 30.
Purpose: The parameter accept_error_backoff_ms controls how long the server waits after an accept error before retrying, to avoid a tight retry loop under system resource pressure.
Config path: hyper_parameters.accept_error_backoff_ms.
The parameter accept_error_backoff_ms determines how long the server waits after a failed "accept new connection" attempt before trying again. This prevents the server from spinning in a tight loop if the OS temporarily refuses new connections.
In normal operation, this value is rarely visible. It becomes relevant when the OS is under stress (file descriptor exhaustion, transient network issues). Keeping a small backoff prevents CPU burn during such events while still allowing recovery.
If omitted, it defaults to the value 100.
Purpose: The parameter quarantine_cooldown_secs controls how long a client address is blocked after a forced disconnect.
Config path: hyper_parameters.quarantine_cooldown_secs.
The parameter quarantine_cooldown_secs determines how long (in seconds) a disconnected client is temporarily blocked from reconnecting. This prevents a misbehaving client from instantly reconnecting and repeating the same overload.
If you are defending against buggy or abusive clients that reconnect immediately, increase this cooldown so the server has time to recover. If clients are trusted and disconnects are usually accidental, a shorter cooldown improves recovery time.
If omitted, it defaults to the value 300.
Purpose: The parameter quarantine_cleanup_interval_secs controls how often the server removes expired entries from the quarantine list.
Config path: hyper_parameters.quarantine_cleanup_interval_secs.
The parameter quarantine_cleanup_interval_secs determines how often (in seconds) the server cleans up expired blocks so clients can reconnect once their cooldown expires.
This is primarily an operational hygiene setting. A shorter interval removes expired entries more promptly; a longer interval reduces background work. Most deployments can keep the default.
If omitted, it defaults to the value 60.
Purpose: The parameter throttle_delay_ms controls the time added as a delay when throttling is applied; this is a gentle slowdown intended to smooth bursts.
Config path: hyper_parameters.throttle_delay_ms.
The parameter throttle_delay_ms determines how much delay (in milliseconds) the server adds when it throttles a client. This is the "speed bump" amount.
If you want the server to apply a light speed bump rather than a noticeable pause, keep this small. If clients are generating aggressive bursts, increasing this delay can reduce churn and help the system stabilize with less need for stronger actions.
If omitted, it defaults to the value 200.
Purpose: The parameter flow_control_delay_ms controls the time added as a pause when flow control is applied; this is a stronger slowdown intended to stop a client from overwhelming the server during high load.
Config path: hyper_parameters.flow_control_delay_ms.
The parameter flow_control_delay_ms determines how long (in milliseconds) the server pauses a client when flow control is triggered. This is a stronger brake than throttling.
This should generally be longer than throttle_delay_ms. If it is too short, flow control may not meaningfully reduce pressure. If it is too long, legitimate clients may experience noticeable latency spikes when the server enters protection mode.
If omitted, it defaults to the value 1000.
Purpose: The parameter worker_threads controls the number of worker threads used by the Rust runtime.
Config path: hyper_parameters.worker_threads.
The parameter worker_threads (default: num_cores - 1) determines how many CPU worker threads the Rust server uses to process work. More threads can help on busy machines, but it is usually best to keep this close to the default unless you are tuning with real load tests.
The default is usually appropriate. Increase this only if you have evidence that the server is CPU-bound and under-utilizing cores. Decrease it if you intentionally want the server to use fewer CPU resources on a shared machine.
It defaults to "number of CPU cores minus one", with a minimum of 1.
The backpressure policy lives in the config at hyper_parameters.backpressure_policy.
Backpressure is the server's "automatic stability system". Its goal is to prevent a situation where one overloaded broadcast causes the server to accumulate too many pending tasks and memory allocations.
In plain terms, the server continuously estimates how much work it is about to create when forwarding a message. A message that must be forwarded to many other clients creates more work than a message forwarded to a few clients. When that estimated work exceeds configured thresholds, the server takes steps of increasing severity:
- Throttle: inject a small delay for the sending client to slow the rate of work creation.
- Flow control: inject a larger pause to more strongly limit the client.
- Disconnect: as a last resort, remove the client and quarantine its address for a cooldown period.
In practice, the thresholds decide when each step becomes acceptable, and the delays decide how strong the slowdown feels.
The thresholds and the choice of steps are controlled by the parameters below.
Purpose: The parameter enable_throttle turns on the "gentle slowdown" step.
Config path: hyper_parameters.backpressure_policy.enable_throttle.
The parameter enable_throttle determines whether the server is allowed to use the "slow down" step. If this is on, the server can tell a client to slow down before doing anything harsher.
Enable this in most deployments. Throttling is a low-friction way to smooth bursts without disrupting client sessions. If you disable it, the server will rely more quickly on stronger measures (flow control and disconnect).
If omitted, it defaults to the value true.
Purpose: The parameter throttle_threshold controls the load threshold at which throttling begins.
Config path: hyper_parameters.backpressure_policy.throttle_threshold.
The parameter throttle_threshold determines when the server considers a client "too busy" and starts slowing them down. Higher means the server tolerates more burstiness; lower means it reacts sooner.
Set this based on expected room size or fan-out. If it is too low, clients may be slowed down during normal operation. If it is too high, the server may postpone throttling until pressure is already severe.
If omitted, it defaults to the value 100.
Purpose: The parameter enable_flow_control turns on the stronger pause step.
Config path: hyper_parameters.backpressure_policy.enable_flow_control.
The parameter enable_flow_control determines whether the server is allowed to use the "temporary pause" step. If this is on, the server can tell a client to pause briefly so the system can catch up.
Enable this when you want a firm protection layer that prevents runaway load during spikes. If you disable it, the server will either keep throttling (if enabled) or jump directly to disconnect (if enabled) when thresholds are exceeded.
If omitted, it defaults to the value true.
Purpose: The parameter flow_control_threshold controls the load threshold at which flow control begins.
Config path: hyper_parameters.backpressure_policy.flow_control_threshold.
The parameter flow_control_threshold determines when the server starts applying flow control (the pause step). This is usually set higher than the throttle threshold because it is a stronger response.
This is typically higher than throttle_threshold. A common approach is to treat throttling as early smoothing and flow control as hard braking. If these two thresholds are too close, the server will enter pause mode frequently, which can feel abrupt to clients.
If omitted, it defaults to the value 300.
Purpose: The parameter enable_disconnect turns on the last-resort disconnect step.
Config path: hyper_parameters.backpressure_policy.enable_disconnect.
The parameter enable_disconnect determines whether the server is allowed to disconnect a client when things get extreme. If this is on, the server can disconnect a client to protect itself and other clients.
Enable this if you need strong protection against extreme overload or abusive clients. If you disable disconnect, the server will try to recover using throttle and flow control alone, which can be acceptable in trusted environments but provides less isolation when a client is persistently disruptive.
If omitted, it defaults to the value true.
Purpose: The parameter disconnect_threshold controls the load threshold at which the server disconnects the client and places it in quarantine.
Config path: hyper_parameters.backpressure_policy.disconnect_threshold.
The parameter disconnect_threshold determines when the server decides a client is so overloaded or disruptive that it should be disconnected. This is the last-resort threshold.
Treat this as the safety fuse. If it is too low, legitimate clients may be disconnected during real traffic spikes. If it is too high, the server may spend too long in a degraded state trying to recover without removing the source of pressure.
If omitted, it defaults to the value 500.
This section gives a starting point for common deployment goals. The intent is to provide safe defaults, not to replace measurement and load testing.
Unless stated otherwise, names below refer to hyper_parameters.*. Backpressure policy settings live at hyper_parameters.backpressure_policy.*.
Focus on allowing frequent small messages without aggressively slowing users:
- Raise
rate_limit_msgs_per_minuteif users legitimately send bursts. - Keep
throttle_delay_msmodest. - Keep
flow_control_delay_msmoderate so the system can recover quickly from spikes. - Set thresholds based on expected room sizes.
Focus on preventing the server from creating too many concurrent sends:
- Increase
connection_buffer_sizeandqueue_monitor_capacityso monitoring signals can keep up. - Choose thresholds proportional to the expected number of recipients.
- Use JSON file logs and
log_keysto keep logs compact.
Focus on limiting damage from malicious or buggy clients:
- Lower
rate_limit_msgs_per_minute. - Lower thresholds and keep disconnect enabled.
- Increase
quarantine_cooldown_secsif repeated reconnect attempts are part of the threat model. - Consider disabling detailed payload logging via
log_keys: []if payloads may contain sensitive content.
{
"host": "127.0.0.1",
"port": 8888,
"version": "rust",
"logger": {
"log_level": "INFO",
"enable_console_log": true,
"console_log_format": "\u001b[92m%(asctime)s\u001b[0m - \u001b[94m%(name)s\u001b[0m - %(levelname)s - %(message)s",
"enable_file_log": true,
"enable_json_log": true,
"log_file_path": "logs/",
"log_format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s",
"max_file_size": 1000000,
"backup_count": 3,
"date_format": "%Y-%m-%d %H:%M:%S.%3f",
"log_keys": null
},
"hyper_parameters": {
"connection_buffer_size": 256,
"command_buffer_size": 64,
"control_channel_capacity": 8,
"queue_monitor_capacity": 100,
"client_timeout_secs": 600,
"rate_limit_msgs_per_minute": 1000,
"timeout_check_interval_secs": 30,
"accept_error_backoff_ms": 100,
"quarantine_cooldown_secs": 600,
"quarantine_cleanup_interval_secs": 60,
"throttle_delay_ms": 200,
"flow_control_delay_ms": 1000,
"worker_threads": 4,
"backpressure_policy": {
"enable_throttle": true,
"throttle_threshold": 50,
"enable_flow_control": true,
"flow_control_threshold": 150,
"enable_disconnect": true,
"disconnect_threshold": 300
}
}
}
« Previous: Summoner.server.server
| Next: Summoner.server »