infrawatch · ayefimov-1 · Feb 3, 2026 · Feb 16, 2026 · Feb 17, 2026 · Feb 17, 2026
diff --git a/.yamllint b/.yamllint
diff --git a/roles/telemetry_chargeback/.yamllint b/roles/telemetry_chargeback/.yamllint
@@ -0,0 +1,18 @@
+---
+# Ansible-lint compatible yamllint config for this role only.
+# See: https://ansible.readthedocs.io/projects/lint/rules/yaml/
+extends: default
+
+rules:
+  comments:
+    min-spaces-from-content: 1
+  comments-indentation: false
+  braces:
+    min-spaces-inside: 0
+    max-spaces-inside: 1
+  octal-values:
+    forbid-implicit-octal: true
+    forbid-explicit-octal: true
+  line-length:
+    max: 160
+    level: warning
diff --git a/roles/telemetry_chargeback/README.md b/roles/telemetry_chargeback/README.md
@@ -5,7 +5,7 @@ The **`telemetry_chargeback`** role is designed to test the **RHOSO Cloudkitty**
 The role performs two main functions:
 
 1. **CloudKitty Validation** - Enables and configures the CloudKitty hashmap rating module, then validates its state.
-2. **Synthetic Data Generation** - Generates synthetic Loki log data for testing chargeback scenarios using a Python script and Jinja2 template.
+2. **Synthetic Data Generation & Analysis** - Generates synthetic Loki log data for testing chargeback scenarios and calculates metric totals. The role automatically discovers and processes all scenario files matching `test_*.yml` in the `files/` directory. For each scenario it runs: generate synthetic data, compute syn-totals, ingest to Loki, flush Loki ingester memory, and get cost via CloudKitty rating summary (using begin/end from syn-totals). Retrieve-from-Loki is included in the load_loki_data flow. After all scenarios, the role runs cleanup (`cleanup_ck.yml`) to remove the local flush cert directory.
 
 Requirements
 ------------
@@ -15,14 +15,15 @@ It relies on the following being available on the target or control host:
 * The **OpenStack CLI client** must be installed and configured with administrative credentials.
 * Required Python libraries for the `openstack` CLI (e.g., `python3-openstackclient`).
 * Connectivity to the OpenStack API endpoint.
-* **Python 3** with the following libraries for synthetic data generation:
+* **Python 3** with the following libraries for synthetic data generation and analysis:
   * `PyYAML`
   * `Jinja2`
 
 It is expected to be run **after** a successful deployment and configuration of the following components:
 
 * **OpenStack:** A functional OpenStack cloud (RHOSO) environment.
 * **Cloudkitty:** The Cloudkitty service must be installed, configured, and running.
+* **Loki / OpenShift (for ingest and flush):** When using ingest and flush tasks, the control host must have `oc` CLI access, and the Cloudkitty Loki stack (route, certificates, ingester) must be deployed. The role sets Loki push/query URLs and extracts certificates via `setup_loki_env.yml`.
 
 Role Variables
 --------------
@@ -40,23 +41,97 @@ These variables are used internally by the role and typically do not need to be
 
 | Variable | Default Value | Description |
 |----------|---------------|-------------|
-| `logs_dir_zuul` | `/home/zuul/ci-framework-data/logs` | Remote directory for log files. |
-| `artifacts_dir_zuul` | `/home/zuul/ci-framework-data/artifacts` | Directory for generated artifacts. |
-| `ck_synth_script` | `{{ role_path }}/files/gen_synth_loki_data.py` | Path to the synthetic data generation script. |
-| `ck_data_template` | `{{ role_path }}/template/loki_data_templ.j2` | Path to the Jinja2 template for Loki data format. |
-| `ck_data_config` | `{{ role_path }}/files/test_static.yml` | Path to the scenario configuration file. |
-| `ck_output_file_local` | `{{ artifacts_dir_zuul }}/loki_synth_data.json` | Local path for generated synthetic data. |
-| `ck_output_file_remote` | `{{ logs_dir_zuul }}/gen_loki_synth_data.log` | Remote destination for synthetic data. |
+| `logs_dir_zuul` | `{{ ansible_env.HOME }}/ci-framework-data/logs` | Remote directory for log files. |
+| `artifacts_dir_zuul` | `{{ ansible_env.HOME }}/ci-framework-data/artifacts` | Directory for generated artifacts. |
+| `cloudkitty_scenario_dir` | `{{ role_path }}/files` | Directory containing scenario files (`test_*.yml`). |
+| `cloudkitty_synth_data_suffix` | `-synth_data.json` | Suffix for generated synthetic data files. |
+| `cloudkitty_loki_data_suffix` | `-loki_data.json` | Suffix for Loki query result JSON files. |
+| `cloudkitty_synth_totals_suffix` | `-synth_metrics_totals.yml` | Suffix for generated metric totals files (from synthetic data). |
+| `cloudkitty_loki_totals_suffix` | `-loki_totals.yml` | Suffix for CloudKitty rating summary output files (from loki_rate task). |
+| `cloudkitty_loki_totals_metrics_suffix` | `-loki_metrics_totals.yml` | Suffix for metric totals computed from Loki-retrieved JSON (retrieve_loki_data task). |
+| `cloudkitty_synth_script` | `{{ role_path }}/files/gen_synth_loki_data.py` | Path to the synthetic data generation script. |
+| `cloudkitty_data_template` | `{{ role_path }}/templates/loki_data_templ.j2` | Path to the Jinja2 template for Loki data format. |
+| `cloudkitty_totals_script` | `{{ role_path }}/files/gen_synth_loki_metrics_totals.py` | Path to the metric totals calculation script. |
+
+### Loki / OpenShift Variables (vars/main.yml)
+
+Used by setup, ingest, flush, and retrieve tasks when running against Loki on OpenShift:
+
+| Variable | Default Value | Description |
+|----------|---------------|-------------|
+| `cert_secret_name` | `cert-cloudkitty-client-internal` | OpenShift secret name for client certificates. |
+| `cert_dir` | `{{ ansible_user_dir }}/ck-certs` | Local directory for extracted ingest/query certs. |
+| `client_secret` | `secret/cloudkitty-lokistack-gateway-client-http` | Secret for flush client certs. |
+| `ca_configmap` | `cm/cloudkitty-lokistack-ca-bundle` | ConfigMap for CA bundle. |
+| `remote_cert_dir` | `osp-certs` | Directory inside the OpenStack pod for certs. |
+| `local_cert_dir` | `{{ ansible_env.HOME }}/ci-framework-data/flush_certs` | Local directory for flush certs (removed by cleanup_ck.yml after the run). |
+| `logql_query` | `{service="cloudkitty"}` (overridable via `loki_query`) | LogQL query for Loki. |
+| `cloudkitty_namespace` | `openstack` | OpenShift namespace for Cloudkitty/Loki resources. |
+| `openstackpod` | `openstackclient` | OpenStack client pod name for exec/cp. |
+| `lookback` | `6` | Days lookback for Loki query time range. |
+| `limit` | `50` | Limit for Loki query results. |
+
+Loki push/query URLs are set dynamically in `setup_loki_env.yml` from the Cloudkitty Loki route.
+
+### Synthetic Data Scripts
+
+**gen_synth_loki_data.py** — Generates Loki-format JSON from a scenario YAML and template. The role invokes it with `-r` so that timestamps in the output are in **reverse** order (youngest first, oldest last). When run manually you can omit `-r` for chronological order (oldest first, youngest last).
+
+| Option | Description |
+|--------|--------------|
+| `--tmpl` | Path to the Jinja2 template (e.g. `loki_data_templ.j2`). |
+| `-t`, `--test` | Path to the scenario YAML (e.g. `test_static_basic.yml`). |
+| `-o`, `--output` | Path to the output JSON file. |
+| `-p`, `--project-id` | Optional; overrides `groupby.project_id` in every log entry. |
+| `-u`, `--user-id` | Optional; overrides `groupby.user_id` in every log entry. |
+| `-r`, `--reverse` | Reverse timestamp order in JSON output (youngest first, oldest last). |
+| `--debug` | Enable debug logging. |
+
+**gen_synth_loki_metrics_totals.py** — Reads the synthetic (or Loki-retrieved) JSON and writes a YAML with aggregated metrics and time bounds. The output is used by the role for validation and for the Loki query time range.
+
+Output YAML structure:
+
+* **time** — `begin`, `end` (ISO strings), `begin_nano`, `end_nano` (nanosecond timestamps for the first and last time step; used by the Loki query in `retrieve_loki_data.yml`).
+* **data_log** — `total_time_steps`, `metrics_per_step`, `log_count`.
+* **synth_rate** — Per-metric rates and `total_rate`.
+
+### Dynamically Set Variables
+
+Set in **main.yml** from the OpenStack CLI (`openstack project show admin` / `openstack user show admin`):
+
+| Variable | Description |
+|----------|-------------|
+| `cloudkitty_project_id` | ID of the OpenStack project named `admin` (empty string if not found). Passed as `-p` to the synthetic data generator when non-empty. |
+| `cloudkitty_user_id` | ID of the OpenStack user named `admin` (empty string if not found). Passed as `-u` to the synthetic data generator when non-empty. |
+
+Set in **gen_synth_loki_data.yml** for each scenario file during the loop:
+
+| Variable | Description |
+|----------|-------------|
+| `cloudkitty_data_file` | Local path for generated JSON data (`{{ artifacts_dir_zuul }}/{{ scenario_name }}-synth_data.json`) |
+| `cloudkitty_synth_totals_file` | Local path for calculated metric totals (`{{ artifacts_dir_zuul }}/{{ scenario_name }}{{ cloudkitty_synth_totals_suffix }}`) |
+| `cloudkitty_test_file` | Path to the scenario configuration file (`{{ cloudkitty_scenario_dir }}/{{ scenario_name }}.yml`) |
 
 Scenario Configuration
 ----------------------
-The synthetic data generation is controlled by a YAML configuration file (`files/test_static.yml`). This file defines:
+The synthetic data generation is controlled by YAML configuration files in the `files/` directory. Any file matching `test_*.yml` will be automatically discovered and processed. Files whose names start with an underscore (e.g. `_test_*.yml`) are **not** discovered by the role; they can be used as reference or for manual runs.
+
+Each scenario file defines:
+
+* **generation** — Time range configuration (days, step_seconds).
+* **log_types** — List of log type definitions. Each entry has **type** (identifier and value in output), unit, description, qty, price, groupby, and metadata. The **groupby** dict typically includes dimension keys (e.g. id, user_id, project_id, tenant_id); the generator merges **date_fields** into groupby at run time.
+* **required_fields** — Top-level keys required for each log type (e.g. type, unit, qty, price, groupby, metadata).
+* **date_fields** — Date field names to merge into groupby (week_of_the_year, day_of_the_year, month, year).
+* **loki_stream** — Loki stream configuration (service name).
+
+**groupby.id** should be consistent by metric type across all scenario files so that the same type always uses the same id. The reference mapping is defined in `_test_all_qty_zero.yml` (e.g. `ceilometer_cpu_num` → me1, `ceilometer_image_size` → me2, `ceilometer_ip_floating` → me7).
+
+Example scenario files:
 
-* **generation** - Time range configuration (days, step_seconds)
-* **log_types** - List of log type definitions with name, type, unit, qty, price, groupby, and metadata
-* **required_fields** - Fields required for validation
-* **date_fields** - Date fields to add to groupby (week_of_the_year, day_of_the_year, month, year)
-* **loki_stream** - Loki stream configuration (service name)
+* `test_static_basic.yml` — Basic static values for qty and price.
+* `test_static_basic_gid.yml` — Same as above with explicit groupby ids.
+* `test_dyn_basic.yml` — Dynamic values distributed across time steps.
+* `_test_all_qty_zero.yml` — Reference scenario (all quantities zero); defines the standard groupby.id mapping. Not auto-discovered.
 
 Dependencies
 ------------