feat(BA-3023): Add unified multi agent etcd client handling

[hhoikoo]

resolves #6720 (BA-3023)

This change introduces EtcdClientRegistry, which is a class that can contain etcd clients for multiple agents, each with its own prefix information at scaling group and individual node level.

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue
• Installer updates including:
  • Fixtures for db schema changes
  • New mandatory config options
• Update of end-to-end CLI integration tests in ai.backend.test
• API server-client counterparts (e.g., manager API -> client SDK)
• Test case(s) to:
  • Demonstrate the difference of before/after
  • Demonstrate the flow of abstract/conceptual models with a concrete implementation
• Documentation
  • Contents in the docs directory
  • docstrings in public interfaces and type annotations

[Copilot]

Pull Request Overview

This PR introduces EtcdClientRegistry, a new class designed to manage etcd clients for multiple agents within a single process. This enables clean separation of etcd client instances, each with appropriate scope prefixes for accessing agent-specific, scaling-group, and global configurations.

Key changes:

• Created EtcdClientRegistry class to centralize etcd client management with support for multiple agents
• Refactored AgentRPCServer to use the registry instead of a single etcd client
• Updated all etcd access patterns to use either global_etcd for shared configuration or agent-specific clients

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/ai/backend/agent/etcd.py	New file introducing the EtcdClientRegistry class that manages etcd clients for multiple agents with proper scoping
src/ai/backend/agent/server.py	Refactored to use EtcdClientRegistry instead of a single AsyncEtcd instance; updated all etcd access to use appropriate client (global vs agent-specific)
tests/agent/test_agent.py	Updated test fixtures and mocks to use the new etcd_client_registry structure with global_etcd access pattern
changes/.feature.md	Added changelog entry documenting the new feature

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.