MAINT: Unify scenario initialize_async's into common parameter bag (#2132)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Author: rlundeen2

.env_example

@@ -30,7 +30,7 @@
 
 PLATFORM_OPENAI_CHAT_ENDPOINT="https://api.openai.com/v1"
 PLATFORM_OPENAI_CHAT_KEY="sk-xxxxx"
-PLATFORM_OPENAI_CHAT_GPT4O_MODEL="gpt-4o"
+PLATFORM_OPENAI_CHAT_MODEL="gpt-4o"
 
 # Note: For Azure OpenAI endpoints, use the new format with /openai/v1 and specify the model separately
 # Example: https://xxxx.openai.azure.com/openai/v1
@@ -113,7 +113,7 @@ AZURE_FOUNDRY_DEEPSEEK_MODEL=""
 
 AZURE_FOUNDRY_PHI4_ENDPOINT="https://xxxxx.models.ai.azure.com"
 AZURE_CHAT_PHI4_KEY="xxxxx"
-AZURE_FOUNDRY_PHI4_MODEL=""
+AZURE_CHAT_PHI4_MODEL=""
 
 AZURE_FOUNDRY_MISTRAL_LARGE_ENDPOINT="https://xxxxx.services.ai.azure.com/openai/v1/"
 AZURE_FOUNDRY_MISTRAL_LARGE_KEY="xxxxx"
@@ -141,7 +141,7 @@ DEFAULT_OPENAI_FRONTEND_MODEL = "gpt-4o"
 
 OPENAI_CHAT_ENDPOINT=${PLATFORM_OPENAI_CHAT_ENDPOINT}
 OPENAI_CHAT_KEY=${PLATFORM_OPENAI_CHAT_KEY}
-OPENAI_CHAT_MODEL=${PLATFORM_OPENAI_CHAT_GPT4O_MODEL}
+OPENAI_CHAT_MODEL=${PLATFORM_OPENAI_CHAT_MODEL}
 # The following line can be populated if using an Azure OpenAI deployment
 # where the deployment name differs from the actual underlying model
 OPENAI_CHAT_UNDERLYING_MODEL=""

.github/instructions/scenarios.instructions.md

@@ -13,9 +13,9 @@ Scenarios orchestrate multi-attack security testing campaigns. Each scenario gro
 All scenarios inherit from `Scenario` (ABC) and must:
 
 1. **Define `VERSION`** as a class constant (increment on breaking changes)
-2. **Optionally declare `BASELINE_ATTACK_POLICY`** (defaults to `BaselineAttackPolicy.Enabled` — a baseline `PromptSendingAttack` is prepended and callers can opt out per run via `initialize_async(include_baseline=False)`):
+2. **Optionally declare `BASELINE_ATTACK_POLICY`** (defaults to `BaselineAttackPolicy.Enabled` — a baseline `PromptSendingAttack` is prepended and callers can opt out per run by setting `include_baseline=False` in the run params, see "Run Parameters" below):
    - `BaselineAttackPolicy.Disabled` — baseline supported but off by default (e.g. `Jailbreak`, where templates dominate the run).
-   - `BaselineAttackPolicy.Forbidden` — baseline is meaningless for this scenario's comparison axis (e.g. `AdversarialBenchmark`, which compares against gold-standard answers). Explicit `include_baseline=True` raises `ValueError`.
+   - `BaselineAttackPolicy.Forbidden` — baseline is meaningless for this scenario's comparison axis (e.g. `AdversarialBenchmark`, which compares against gold-standard answers). Supplying `include_baseline=True` raises `ValueError`.
 3. **Pass `strategy_class`, `default_strategy`, and `default_dataset_config` to `super().__init__()`:**
 
 ```python
@@ -81,6 +81,34 @@ Requirements:
 - `super().__init__()` called with `version`, `strategy_class`, `default_strategy`, `default_dataset_config`, `objective_scorer`
 - complex objects like `adversarial_chat` or `objective_scorer` should be passed into the constructor.
 
+## Run Parameters
+
+Run-time inputs (target, strategies, dataset config, concurrency, labels, baseline flag) are **not** arguments to `initialize_async`. They flow through a single parameter bag (`self.params`), populated by `set_params_from_args` from the merged CLI / config / programmatic arguments. `initialize_async` takes no arguments and reads everything from the bag:
+
+```python
+scenario.set_params_from_args(args={"objective_target": target, "max_concurrency": 8})
+await scenario.initialize_async()
+```
+
+The base `Scenario` declares the common run inputs once in `_common_scenario_parameters()`: `objective_target` (a `RegistryReference` — resolved by name or supplied as an instance), the `opaque` live objects `scenario_strategies` / `strategy_converters` / `dataset_config` / `memory_labels` (passed by identity, never coerced or deep-copied), and the scalars `max_concurrency` / `max_retries` / `include_baseline`.
+
+### Declaring custom parameters — add via `additional_parameters`
+
+The base `Scenario` composes `supported_parameters()` as `_common_scenario_parameters() + additional_parameters()`. To add your own parameters, override **`additional_parameters()`** and return just your extras — the common inputs are included for you, so there's no `super()` call to forget:
+
+```python
+@classmethod
+def additional_parameters(cls) -> list[Parameter]:
+    return [
+        Parameter(name="max_turns", description="...", param_type=int, default=5),
+    ]
+```
+
+- **Add (common case):** override `additional_parameters` and return `[Parameter(...)]`
+- **Remove / replace a common input (rare):** override `supported_parameters` directly and compose against `super()`, e.g. `return [p for p in super().supported_parameters() if p.name != "dataset_config"]`
+
+Dropping a common input is not silent: `set_params_from_args` rejects any value supplied for an undeclared parameter, so the registry/CLI/programmatic path fails loudly. If a scenario resolves its strategies differently (e.g. pairing attacks with converters), override the `_resolve_scenario_strategies` hook rather than `initialize_async` (see `RedTeamAgent`).
+
 ## Dataset Loading
 
 Datasets are read from `CentralMemory`.
@@ -272,6 +300,8 @@ New scenarios must be registered in `pyrit/scenario/__init__.py` as virtual pack
 ## Common Review Issues
 
 - Accessing `self._objective_target` or `self._scenario_strategies` before `initialize_async()`
+- Overriding `supported_parameters()` without composing against `super()` (silently drops the common run inputs)
+- Adding arguments back onto `initialize_async` instead of declaring them via `supported_parameters()` and reading from `self.params`
 - Forgetting `@apply_defaults` on `__init__`
 - Empty `seed_groups` passed to `AtomicAttack`
 - Missing `VERSION` class constant

doc/code/registry/1_class_registry.ipynb

@@ -137,8 +137,9 @@
     "encoding_class = registry.get_class(\"garak.encoding\")\n",
     "scenario = encoding_class()  # type: ignore\n",
     "\n",
-    "# Pass dataset configuration to initialize_async\n",
-    "await scenario.initialize_async(objective_target=target)  # type: ignore\n",
+    "# Set the objective target, then initialize\n",
+    "scenario.set_params_from_args(args={\"objective_target\": target})  # type: ignore\n",
+    "await scenario.initialize_async()  # type: ignore\n",
     "\n",
     "# Option 2: Use create_instance() shortcut\n",
     "# scenario = registry.create_instance(\"garak.encoding\", objective_target=my_target, ...)\n",

doc/code/registry/1_class_registry.py

@@ -56,8 +56,9 @@
 encoding_class = registry.get_class("garak.encoding")
 scenario = encoding_class()  # type: ignore
 
-# Pass dataset configuration to initialize_async
-await scenario.initialize_async(objective_target=target)  # type: ignore
+# Set the objective target, then initialize
+scenario.set_params_from_args(args={"objective_target": target})  # type: ignore
+await scenario.initialize_async()  # type: ignore
 
 # Option 2: Use create_instance() shortcut
 # scenario = registry.create_instance("garak.encoding", objective_target=my_target, ...)