Commit a99c65a
Perfpub to use breakdow.csv for filtering
Summary:
**Overview:** This diff enhances the perfpub tool to filter metrics based on the start time and end time found in the `breakdown.csv` file, which is a file inside the input directory. This is essential for DCPerf mini where the benchmark is only a few seconds and we need to filter metrics for the exact benchmark time. We are changing DCPerf mini to genereate the breakdown.csv and put it the metric directory.
So, with the current diff, here is how we filter metrics:
1. If --last-secs and --skip-last-secs are provided by user, they will be used for filtering metrics
2. If --last-secs and --skip-last-secs are not provided and breakdown.csv is present in the metric directory, breakdown.csv will be used to claculate the start and end time
3. if breakdown.csv is not present, the default values for --last-secs and --skip-last-secs will be used for filtering metrics
**Key Changes:**
1. **Added `parse_breakdown_csv()` function**: Extracts the `main_benchmark` start and end timestamps from `breakdown.csv` which records operation timing information during benchmark execution.
2. **Added `parse_metric_timestamps()` function**: Parses timestamp columns from metric files (like `mpstat.csv`, `mem-stat.csv`) which typically contain time-only values in formats like "05:01:49 PM".
3. **Added `find_closest_timestamp_index()` function**: Finds the closest timestamp index in metric files that matches the benchmark start/end times, handling edge cases like day boundary transitions.
4. **Added `calculate_filtering_params()` function**: Orchestrates the above functions to compute optimal `last_secs` and `skip_last_secs` values based on:
* The actual benchmark execution window from `breakdown.csv`
* The metrics collection intervals
* The alignment between benchmark times and metric file timestamps
5. **Updated perfpub main logic**: When default filtering parameters are used (`--last-secs=300` and `--skip-last-secs=0`), the tool now automatically calculates better values from `breakdown.csv` if available, with informative output showing the calculated values.
Reviewed By: charles-typ
Differential Revision: D84877609
fbshipit-source-id: 5c952d42a73c14518551ea45ce60856dfe27d0511 parent fdcfc05 commit a99c65a
1 file changed
+262
-31
lines changed
0 commit comments