Skip to content

Conversation

@gautschimi
Copy link
Contributor

@gautschimi gautschimi commented Oct 23, 2025

This PR:

  • fixes DV for the flash controller (There was a mismatch of byte and word addresses in the function that generates the scrambling tweak)
  • introduces one new test that performs basic random read/write operations to the data partition of the flash with ECC/scrambling randomly enabled

The flash_controller DV dropped to ~75% success rate flash_ctrl_dv_results_march_2025 during this year. With these fixes, the flash_controller DV should achieve ~97% success rate again (similar to earlier this year: flash_ctrl_dv_results_jan_2025)

@gautschimi gautschimi requested a review from a team as a code owner October 23, 2025 13:13
@gautschimi gautschimi requested review from rswarbrick and vogelpi and removed request for a team October 23, 2025 13:13
@gautschimi gautschimi force-pushed the flash_ctrl_dv branch 3 times, most recently from 95acc90 to f1e73ae Compare October 23, 2025 16:47
@gautschimi gautschimi requested a review from nasahlpa October 24, 2025 06:55
Copy link
Contributor

@vogelpi vogelpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @gautschimi for looking into this. I have some minor comments and a question.

Would you mind running a full regression (with VCS) with coverage enabled on this PR to verify it doesn't break other tests please?

Comment on lines 38 to 39
localparam uint FLASH_DATA_BYTE_WIDTH = $clog2(flash_ctrl_top_specific_pkg::DataByteWidth);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look write to me. DataByteWidth is vbits(8) = 3. If you take another $clog2 of that you get 2. Then below, you are using this to compute the base word address of different pages. Are you sure you end up with the right addresses?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, that looks wrong. This function is only used for toplevel software tests, I need to run a few of them

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment on lines 331 to 345
update_mp_region_cfg_mubifalse(region_cfg);
data = get_csr_val_with_updated_field(ral.mp_region_cfg[index].en, data,
region_cfg.en);
data = data | get_csr_val_with_updated_field(ral.mp_region_cfg[index].rd_en, data,
region_cfg.read_en);
data = data | get_csr_val_with_updated_field(ral.mp_region_cfg[index].prog_en, data,
region_cfg.program_en);
data = data | get_csr_val_with_updated_field(ral.mp_region_cfg[index].erase_en, data,
region_cfg.erase_en);
data = data | get_csr_val_with_updated_field(ral.mp_region_cfg[index].scramble_en,
data, region_cfg.scramble_en);
data = data | get_csr_val_with_updated_field(ral.mp_region_cfg[index].ecc_en, data,
region_cfg.ecc_en);
data = data | get_csr_val_with_updated_field(ral.mp_region_cfg[index].he_en, data,
region_cfg.he_en);
data = get_csr_val_with_updated_field(ral.mp_region_cfg[index].rd_en, data,
region_cfg.read_en);
data = get_csr_val_with_updated_field(ral.mp_region_cfg[index].prog_en, data,
region_cfg.program_en);
data = get_csr_val_with_updated_field(ral.mp_region_cfg[index].erase_en, data,
region_cfg.erase_en);
data = get_csr_val_with_updated_field(ral.mp_region_cfg[index].scramble_en,
data, region_cfg.scramble_en);
data = get_csr_val_with_updated_field(ral.mp_region_cfg[index].ecc_en, data,
region_cfg.ecc_en);
data = get_csr_val_with_updated_field(ral.mp_region_cfg[index].he_en, data,
region_cfg.he_en);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious, can you explain why these changes are required and don't break other tests please? Because I can imagine that these tasks for the base sequence are quite fundamental and that they get used in almost all sequences.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I don't know what the purpose of this function is. I would say it should set all the CSRs with the updated values provided by the variable region_cfg. The function get_csr_val_with_updated_field() does exactly this. There is no need for an additional OR with the current value.

Infact, an additional OR will make mubi encodings invalid. (MuBi4True (0x6) | MuBi4False (0x9) = 0xF (invalid encoding)
-> the region encodings will always be invalid -> scrambling, ecc was mostly disabled

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for explaining, this makes a lot of sense.

@gautschimi gautschimi force-pushed the flash_ctrl_dv branch 2 times, most recently from eb9d025 to 58691fd Compare October 24, 2025 14:05
Adds a basic test that 1. configures scramble, ecc_en 2. programs
the flash, 3. read back memory over the host interface,
4. read backs the memory over the controller interface

Signed-off-by: Michael Gautschi <[email protected]>
FlashAddrWidth is in flash words (not bytes). Passing a byte address to
a function which expects word-addresses leads to a truncation of the three
most significant address bits. As a result, when accessing the upper part
of each flash bank the scrambling tweak was wrong which lead to a wrong
data encryption and hence also to a wrong ICV calculation for the expected
responses.

Signed-off-by: Michael Gautschi <[email protected]>
@gautschimi
Copy link
Contributor Author

results of the latest coverage run: (corresponds to d3aa4e9)

### GitHub Revision: [`58691fd3c4`](https://github.com/lowrisc/opentitan/tree/58691fd3c4db68a8c54dba5d058fb4cb4b238cd6)                                                                                                   12:48:42 [186/611]
### Branch: flash_ctrl_dv                                                                                                                                                                                                                   
### [Testplan](https://opentitan.org/book/hw/top_earlgrey/ip_autogen/flash_ctrl/data/flash_ctrl_testplan.html)                                                                                                                              
### Simulator: VCS                                                                                                                                                                                                                          
                                                                                                                                                                                                                                            
### Test Results                                                                                                                                                                                                                            
|  Stage  |                   Name                    | Tests                                    |  Max Job Runtime  |  Simulated Time  |  Passing  |  Total  |  Pass Rate  |                                                               
|:-------:|:-----------------------------------------:|:-----------------------------------------|:-----------------:|:----------------:|:---------:|:-------:|:-----------:|                                                               
|   V1    |                   smoke                   | flash_ctrl_smoke                         |      52.560s      |     1.438ms      |    50     |   50    |  100.00 %   |                                                               
|   V1    |                 smoke_hw                  | flash_ctrl_smoke_hw                      |      4.850s       |    164.627us     |     5     |    5    |  100.00 %   |                                                               
|   V1    |               csr_hw_reset                | flash_ctrl_csr_hw_reset                  |      8.360s       |     97.531us     |     5     |    5    |  100.00 %   |                                                               
|   V1    |                  csr_rw                   | flash_ctrl_csr_rw                        |      4.100s       |    816.086us     |    20     |   20    |  100.00 %   |                                                               
|   V1    |               csr_bit_bash                | flash_ctrl_csr_bit_bash                  |      31.020s      |     9.086ms      |     5     |    5    |  100.00 %   |                                                               
|   V1    |               csr_aliasing                | flash_ctrl_csr_aliasing                  |      23.520s      |     6.345ms      |     5     |    5    |  100.00 %   |                                                               
|   V1    |        csr_mem_rw_with_rand_reset         | flash_ctrl_csr_mem_rw_with_rand_reset    |      4.920s       |    116.869us     |    20     |   20    |  100.00 %   |                                                               
|   V1    | regwen_csr_and_corresponding_lockable_csr | flash_ctrl_csr_rw                        |      4.100s       |    816.086us     |    20     |   20    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_csr_aliasing                  |      23.520s      |     6.345ms      |     5     |    5    |  100.00 %   |                                                               
|   V1    |                 mem_walk                  | flash_ctrl_mem_walk                      |      2.650s       |     69.609us     |     5     |    5    |  100.00 %   |                                                               
|   V1    |            mem_partial_access             | flash_ctrl_mem_partial_access            |      2.730s       |    101.922us     |     5     |    5    |  100.00 %   |                                                               
|   V1    |                                           | **TOTAL**                                |                   |                  |    120    |   120   |  100.00 %   |                                                               
|   V2    |                   sw_op                   | flash_ctrl_sw_op                         |      5.080s       |     38.814us     |     5     |    5    |  100.00 %   |                                                               
|   V2    |             host_read_direct              | flash_ctrl_host_dir_rd                   |      12.970s      |     48.110us     |     5     |    5    |  100.00 %   |                                                               
|   V2    |                 rma_hw_if                 | flash_ctrl_hw_rma                        |      16.122m      |      1.608s      |     3     |    3    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_hw_rma_reset                  |      6.684m       |    160.183ms     |    20     |   20    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_lcmgr_intg                    |      4.840s       |     25.799us     |    20     |   20    |  100.00 %   |                                                               
|   V2    |            host_controller_arb            | flash_ctrl_host_ctrl_arb                 |      13.979m      |    250.491ms     |     5     |    5    |  100.00 %   |                                                               
|   V2    |               erase_suspend               | flash_ctrl_erase_suspend                 |      2.037m       |     4.249ms      |     5     |    5    |  100.00 %   |                                                               
|   V2    |               program_reset               | flash_ctrl_prog_reset                    |      2.145m       |     3.301ms      |    30     |   30    |  100.00 %   |                                                               
|   V2    |            full_memory_access             | flash_ctrl_full_mem_access               |      35.550m      |    106.630ms     |     5     |    5    |  100.00 %   |                                                               
|   V2    |             rd_buff_eviction              | flash_ctrl_rd_buff_evict                 |      1.037m       |     15.025ms     |     5     |    5    |  100.00 %   |                                                               
|   V2    |          rd_buff_eviction_w_ecc           | flash_ctrl_rw_evict                      |      8.050s       |     97.050us     |    38     |   40    |   95.00 %   |                                                               
|         |                                           | flash_ctrl_rw_evict_all_en               |      10.210s      |     27.627us     |    37     |   40    |   92.50 %   |                                                               
|         |                                           | flash_ctrl_re_evict                      |      13.170s      |    104.740us     |    20     |   20    |  100.00 %   |                                                               
|   V2    |                 host_arb                  | flash_ctrl_phy_arb                       |      1.673m       |     1.639ms      |    20     |   20    |  100.00 %   |                                                               
|   V2    |              host_interleave              | flash_ctrl_phy_arb                       |      1.673m       |     1.639ms      |    20     |   20    |  100.00 %   |                                                               
|   V2    |             memory_protection             | flash_ctrl_mp_regions                    |      7.236m       |     84.710ms     |    20     |   20    |  100.00 %   |                                                               
|   V2    |                fetch_code                 | flash_ctrl_fetch_code                    |      9.430s       |     1.835ms      |    10     |   10    |  100.00 %   |                                                               
|   V2    |              all_partitions               | flash_ctrl_rand_ops                      |      4.216m       |    815.421us     |    20     |   20    |  100.00 %   |                                                               
|   V2    |                 error_mp                  | flash_ctrl_error_mp                      |      6.294m       |     6.480ms      |    10     |   10    |  100.00 %   |                                                               
|   V2    |              error_prog_win               | flash_ctrl_error_prog_win                |      3.156m       |     3.890ms      |    10     |   10    |  100.00 %   |                                                               
|   V2    |              error_prog_type              | flash_ctrl_error_prog_type               |      8.628m       |    632.477us     |     5     |    5    |  100.00 %   |                                                               
|   V2    |              error_read_seed              | flash_ctrl_hw_read_seed_err              |      3.990s       |     25.400us     |    20     |   20    |  100.00 %   |                                                               
|   V2    |            read_write_overflow            | flash_ctrl_oversize_error                |      1.694m       |     7.664ms      |     5     |    5    |  100.00 %   |                                                               
|   V2    |            flash_ctrl_disable             | flash_ctrl_disable                       |      6.720s       |     53.824us     |    50     |   50    |  100.00 %   |                                                               
|   V2    |            flash_ctrl_connect             | flash_ctrl_connect                       |      5.220s       |     40.891us     |    80     |   80    |  100.00 %   |                                                               
|   V2    |                stress_all                 | flash_ctrl_stress_all                    |      5.324m       |    277.525us     |     5     |    5    |  100.00 %   |                                                               
|   V2    |             secret_partition              | flash_ctrl_hw_sec_otp                    |      2.166m       |     34.183ms     |    50     |   50    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_otp_reset                     |      23.660s      |     41.742us     |    77     |   80    |   96.25 %   |                                                               
|   V2    |            isolation_partition            | flash_ctrl_hw_rma                        |      16.122m      |      1.608s      |     3     |    3    |  100.00 %   |                                                               
|   V2    |                interrupts                 | flash_ctrl_intr_rd                       |      1.740m       |     2.643ms      |    37     |   40    |   92.50 %   |                                                               
|         |                                           | flash_ctrl_intr_wr                       |      40.980s      |     11.245ms     |    10     |   10    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_intr_rd_slow_flash            |      2.948m       |     63.669ms     |    40     |   40    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_intr_wr_slow_flash            |      1.816m       |    138.207ms     |     9     |   10    |   90.00 %   |                                                               
|   V2    |                invalid_op                 | flash_ctrl_invalid_op                    |      39.290s      |     30.131ms     |    19     |   20    |   95.00 %   |                                                               
|   V2    |                mid_op_rst                 | flash_ctrl_mid_op_rst                    |      20.760s      |     1.342ms      |     5     |    5    |  100.00 %   |                                                               
|   V2    |              double_bit_err               | flash_ctrl_read_word_sweep_derr          |      5.160s       |     88.767us     |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_ro_derr                       |      1.262m       |     4.651ms      |    10     |   10    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_rw_derr                       |      2.393m       |     2.114ms      |    10     |   10    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_derr_detect                   |      1.199m       |     1.397ms      |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_integrity                     |      5.473m       |     10.323ms     |     5     |    5    |  100.00 %   |                                                               
|   V2    |              single_bit_err               | flash_ctrl_read_word_sweep_serr          |      5.190s       |     51.242us     |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_ro_serr                       |      1.132m       |     3.435ms      |    10     |   10    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_rw_serr                       |      2.066m       |     2.290ms      |    10     |   10    |  100.00 %   |                                                               
|   V2    |           singlebit_err_counter           | flash_ctrl_serr_counter                  |      47.280s      |     3.914ms      |     5     |    5    |  100.00 %   |                                                               
|   V2    |           singlebit_err_address           | flash_ctrl_serr_address                  |      46.650s      |     1.283ms      |     5     |    5    |  100.00 %   |                                                               
|   V2    |                 scramble                  | flash_ctrl_wo                            |      2.412m       |     13.501ms     |    20     |   20    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_write_word_sweep              |      3.770s       |     40.584us     |     1     |    1    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_read_word_sweep               |      3.200s       |     89.973us     |     1     |    1    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_ro                            |      57.690s      |    600.704us     |    20     |   20    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_rw                            |      5.466m       |     20.133ms     |    19     |   20    |   95.00 %   |                                                               
|   V2    |            filesystem_support             | flash_ctrl_fs_sup                        |      17.280s      |    690.937us     |     5     |    5    |  100.00 %   |                                                               
|   V2    |          rma_write_process_error          | flash_ctrl_rma_err                       |      7.206m       |    223.384ms     |     3     |    3    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_hw_prog_rma_wipe_err          |      1.200m       |     10.034ms     |    20     |   20    |  100.00 %   |                                                               
|   V2    |                alert_test                 | flash_ctrl_alert_test                    |      4.670s       |    126.442us     |    50     |   50    |  100.00 %   |                                                               
|   V2    |                 intr_test                 | flash_ctrl_intr_test                     |      2.780s       |     17.722us     |    50     |   50    |  100.00 %   |                                                               
|   V2    |           tl_d_oob_addr_access            | flash_ctrl_tl_errors                     |      5.940s       |    122.601us     |    20     |   20    |  100.00 %   |                                                               
|   V2    |            tl_d_illegal_access            | flash_ctrl_tl_errors                     |      5.940s       |    122.601us     |    20     |   20    |  100.00 %   |
|   V2    |          tl_d_outstanding_access          | flash_ctrl_csr_hw_reset                  |      8.360s       |     97.531us     |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_csr_rw                        |      4.100s       |    816.086us     |    20     |   20    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_csr_aliasing                  |      23.520s      |     6.345ms      |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_same_csr_outstanding          |      8.440s       |    906.884us     |    20     |   20    |  100.00 %   |                                                               
|   V2    |            tl_d_partial_access            | flash_ctrl_csr_hw_reset                  |      8.360s       |     97.531us     |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_csr_rw                        |      4.100s       |    816.086us     |    20     |   20    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_csr_aliasing                  |      23.520s      |     6.345ms      |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_same_csr_outstanding          |      8.440s       |    906.884us     |    20     |   20    |  100.00 %   |                                                               
|   V2    |                                           | **TOTAL**                                |                   |                  |    999    |  1013   |   98.62 %   |
|   V2S   |          shadow_reg_update_error          | flash_ctrl_shadow_reg_errors             |      18.570s      |     67.228us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |    shadow_reg_read_clear_staged_value     | flash_ctrl_shadow_reg_errors             |      18.570s      |     67.228us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |         shadow_reg_storage_error          | flash_ctrl_shadow_reg_errors             |      18.570s      |     67.228us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |           shadowed_reset_glitch           | flash_ctrl_shadow_reg_errors             |      18.570s      |     67.228us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |    shadow_reg_update_error_with_csr_rw    | flash_ctrl_shadow_reg_errors_with_csr_rw |      20.850s      |     1.144ms      |    20     |   20    |  100.00 %   |                                                               
|   V2S   |                tl_intg_err                | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_tl_intg_err                   |      2.684m       |     1.567ms      |    20     |   20    |  100.00 %   |                                                               
|   V2S   |         sec_cm_reg_bus_integrity          | flash_ctrl_tl_intg_err                   |      2.684m       |     1.567ms      |    20     |   20    |  100.00 %   | 
|   V2S   |         sec_cm_host_bus_integrity         | flash_ctrl_tl_intg_err                   |      2.684m       |     1.567ms      |    20     |   20    |  100.00 %   |
|   V2S   |         sec_cm_reg_bus_integrity          | flash_ctrl_tl_intg_err                   |      2.684m       |     1.567ms      |    20     |   20    |  100.00 %   |                                              12:48:42 [93/611]
|   V2S   |         sec_cm_host_bus_integrity         | flash_ctrl_tl_intg_err                   |      2.684m       |     1.567ms      |    20     |   20    |  100.00 %   |                                                               
|   V2S   |         sec_cm_mem_bus_integrity          | flash_ctrl_rd_intg                       |      7.580s       |    214.463us     |     3     |    3    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_wr_intg                       |      3.530s       |    174.248us     |     3     |    3    |  100.00 %   |                                                               
|   V2S   |       sec_cm_scramble_key_sideload        | flash_ctrl_smoke                         |      52.560s      |     1.438ms      |    50     |   50    |  100.00 %   |                                                               
|   V2S   |       sec_cm_lc_ctrl_intersig_mubi        | flash_ctrl_otp_reset                     |      23.660s      |     41.742us     |    77     |   80    |   96.25 %   |                                                               
|         |                                           | flash_ctrl_disable                       |      6.720s       |     53.824us     |    50     |   50    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_sec_info_access               |      38.880s      |     16.487ms     |    50     |   50    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_connect                       |      5.220s       |     40.891us     |    80     |   80    |  100.00 %   |                                                               
|   V2S   |         sec_cm_ctrl_config_regwen         | flash_ctrl_config_regwen                 |      3.050s       |    139.060us     |     5     |    5    |  100.00 %   |                                                               
|   V2S   |     sec_cm_data_regions_config_regwen     | flash_ctrl_csr_rw                        |      4.100s       |    816.086us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |     sec_cm_data_regions_config_shadow     | flash_ctrl_shadow_reg_errors             |      18.570s      |     67.228us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |     sec_cm_info_regions_config_regwen     | flash_ctrl_csr_rw                        |      4.100s       |    816.086us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |     sec_cm_info_regions_config_shadow     | flash_ctrl_shadow_reg_errors             |      18.570s      |     67.228us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |         sec_cm_bank_config_regwen         | flash_ctrl_csr_rw                        |      4.100s       |    816.086us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |         sec_cm_bank_config_shadow         | flash_ctrl_shadow_reg_errors             |      18.570s      |     67.228us     |    20     |   20    |  100.00 %   |                                                               
|   V2S   |        sec_cm_mem_ctrl_global_esc         | flash_ctrl_disable                       |      6.720s       |     53.824us     |    50     |   50    |  100.00 %   |                                                               
|   V2S   |         sec_cm_mem_ctrl_local_esc         | flash_ctrl_rd_intg                       |      7.580s       |    214.463us     |     3     |    3    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_access_after_disable          |      2.920s       |     37.140us     |     3     |    3    |  100.00 %   |                                                               
|   V2S   |         sec_cm_mem_addr_infection         | flash_ctrl_host_addr_infection           |      6.710s       |     49.996us     |     3     |    3    |  100.00 %   |                                                               
|   V2S   |      sec_cm_mem_disable_config_mubi       | flash_ctrl_disable                       |      6.720s       |     53.824us     |    50     |   50    |  100.00 %   |                                                               
|   V2S   |         sec_cm_exec_config_redun          | flash_ctrl_fetch_code                    |      9.430s       |     1.835ms      |    10     |   10    |  100.00 %   |                                                               
|   V2S   |            sec_cm_mem_scramble            | flash_ctrl_rw                            |      5.466m       |     20.133ms     |    19     |   20    |   95.00 %   |                                                               
|   V2S   |           sec_cm_mem_integrity            | flash_ctrl_rw_serr                       |      2.066m       |     2.290ms      |    10     |   10    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_rw_derr                       |      2.393m       |     2.114ms      |    10     |   10    |  100.00 %   |                                                               
|         |                                           | flash_ctrl_integrity                     |      5.473m       |     10.323ms     |     5     |    5    |  100.00 %   |                                                               
|   V2S   |       sec_cm_rma_entry_mem_sec_wipe       | flash_ctrl_hw_rma                        |      16.122m      |      1.608s      |     3     |    3    |  100.00 %   |                                                               
|   V2S   |          sec_cm_ctrl_fsm_sparse           | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|   V2S   |           sec_cm_phy_fsm_sparse           | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|   V2S   |        sec_cm_phy_prog_fsm_sparse         | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|   V2S   |             sec_cm_ctr_redun              | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|   V2S   |       sec_cm_phy_arbiter_ctrl_redun       | flash_ctrl_phy_arb_redun                 |      6.200s       |    752.503us     |     5     |    5    |  100.00 %   |                                                               
|   V2S   |  sec_cm_phy_host_grant_ctrl_consistency   | flash_ctrl_phy_host_grant_err            |      3.160s       |     39.294us     |     5     |    5    |  100.00 %   |                                                               
|   V2S   |      sec_cm_phy_ack_ctrl_consistency      | flash_ctrl_phy_ack_consistency           |      3.100s       |     69.823us     |     5     |    5    |  100.00 %   |                                                               
|   V2S   |           sec_cm_fifo_ctr_redun           | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|   V2S   |     sec_cm_mem_tl_lc_gate_fsm_sparse      | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|   V2S   |     sec_cm_prog_tl_lc_gate_fsm_sparse     | flash_ctrl_sec_cm                        |      11.591m      |     1.209ms      |     5     |    5    |  100.00 %   |                                                               
|   V2S   |                                           | **TOTAL**                                |                   |                  |    147    |   147   |  100.00 %   |                                                               
|   V3    |           asymmetric_read_path            | flash_ctrl_rd_ooo                        |      10.730s      |    487.177us     |     1     |    1    |  100.00 %   |                                                               
|   V3    |                                           | **TOTAL**                                |                   |                  |     1     |    1    |  100.00 %   |                                                               
|         |              Unmapped tests               | flash_ctrl_basic_rw                      |      2.096m       |     1.904ms      |     3     |    3    |  100.00 %   |                                                               
|         |                                           | **TOTAL**                                |                   |                  |   1270    |  1284   |   98.91 %   |                                                               
                                                                                                                                                                                                                                            
## Coverage Results                                                                                                                                                                                                                         
                                                                                                                                                                                                                                            
### [Coverage Dashboard](/scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/cov_report/dashboard.html)                                                                                                                   
                                                                                                                                                                                                                                            
|  SCORE  |  LINE   |  COND   |  TOGGLE  |   FSM   |  BRANCH  |  ASSERT  |  GROUP  |                                                                                                                                                        
|:-------:|:-------:|:-------:|:--------:|:-------:|:--------:|:--------:|:-------:|                                                                                                                                                        
| 96.36 % | 96.09 % | 94.75 % | 98.66 %  | 92.52 % | 97.45 %  | 96.62 %  | 98.42 % |                                                                                                                                                        
## Failure Buckets                                                                                                                                                                                                                          
                                                                                                                                                                                                                                            
* `UVM_ERROR (csr_utils_pkg.sv:456) [csr_utils_pkg::csr_rd_check.isolation_fork.unnamed$$_*] Check failed obs == exp (* [*] vs * [*]) Regname: flash_ctrl_core_reg_block.op_status.err reset value: *` has 3 failures:                      
    * Test flash_ctrl_rw_evict_all_en has 3 failures.                                                                                                                                                                                       
        * 12.flash_ctrl_rw_evict_all_en.41850039488337179876107989228238592237500695477635775571050807225729839015950\                                                                                                                      
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/12.flash_ctrl_rw_evict_all_en/latest/run.log                                                                                               
                                                                                                                                                                                                                                            
                UVM_ERROR @ 9917.2 ns: (csr_utils_pkg.sv:456) [csr_utils_pkg::csr_rd_check.isolation_fork.unnamed$$_0] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: flash_ctrl_core_reg_block.op_status.err reset value: 0x0       
                UVM_INFO @ 9917.2 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                     
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
        * 15.flash_ctrl_rw_evict_all_en.112885328337305673308373456732814432187435365906293461974044791457243178371392\                                                                                                                     
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/15.flash_ctrl_rw_evict_all_en/latest/run.log                                                                                               
                                                                                                                                                                                                                                            
                UVM_ERROR @ 10585.4 ns: (csr_utils_pkg.sv:456) [csr_utils_pkg::csr_rd_check.isolation_fork.unnamed$$_0] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: flash_ctrl_core_reg_block.op_status.err reset value: 0x0      
                UVM_INFO @ 10585.4 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                    
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
        * ... and 1 more failures.                                                                                                                                                                                                          
* `Offending 'dst_req_o'` has 3 failures:                                                                                                                                                                                                   
    * Test flash_ctrl_otp_reset has 3 failures.                                                                                                                                                                                             
        * 24.flash_ctrl_otp_reset.54061782225587340258574464281349173675970399103045698965651111163282616304216\                                                                                                                            
          Line 175, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/24.flash_ctrl_otp_reset/latest/run.log                                      
                                                                                                                                                                                                                                            
                        Offending 'dst_req_o'                                                                                                                                                                                               
                UVM_ERROR @ 15458.5 ns: (prim_sync_reqack.sv:354) [ASSERT FAILED] SyncReqAckAckNeedsReq                                                                                                                                     
                UVM_INFO @ 15458.5 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                    
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
        * 63.flash_ctrl_otp_reset.5156759873797471302473051582638234416712726395587958542135678660224409805221\                                                                                                                             
          Line 147, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/63.flash_ctrl_otp_reset/latest/run.log                                      
                                                                                                                                                                                                                                            
                        Offending 'dst_req_o'                                                                                                                                                                                               
                UVM_ERROR @ 34203.1 ns: (prim_sync_reqack.sv:354) [ASSERT FAILED] SyncReqAckAckNeedsReq                                                                                                                                     
                UVM_INFO @ 34203.1 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                    
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
          * ... and 1 more failures.                                                                                                                                                                                                   [0/611]
* `UVM_ERROR (cip_base_scoreboard.sv:267) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert recov_err triggered unexpectedly` has 2 failures:                                                       
    * Test flash_ctrl_invalid_op has 1 failures.                                                                                                                                                                                            
        * 6.flash_ctrl_invalid_op.46125747228589740268811019697759299862600757600528979633160370560549531240635\                                                                                                                            
          Line 2040, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/6.flash_ctrl_invalid_op/latest/run.log                                                                                                    
                                                                                                                                                                                                                                            
                UVM_ERROR @ 3851829.8 ns: (cip_base_scoreboard.sv:267) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert recov_err triggered unexpectedly                      
                UVM_INFO @ 3851829.8 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                  
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
    * Test flash_ctrl_rw has 1 failures.                                                                                                                                                                                                    
        * 11.flash_ctrl_rw.107527281823836188647429466420770973083312482339230381988284978392287581539932\                                                                                                                                  
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/11.flash_ctrl_rw/latest/run.log                                                                                                            
                                                                                                                                                                                                                                            
                UVM_ERROR @ 9219174.6 ns: (cip_base_scoreboard.sv:267) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert recov_err triggered unexpectedly                      
                UVM_INFO @ 9219174.6 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                  
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
* `UVM_ERROR (csr_utils_pkg.sv:456) [csr_utils_pkg::csr_rd_check.isolation_fork.unnamed$$_*] Check failed obs == exp (* [*] vs * [*]) Regname: flash_ctrl_core_reg_block.err_code.rd_err reset value: *` has 2 failures:                    
    * Test flash_ctrl_rw_evict has 2 failures.                                                                                                                                                                                              
        * 6.flash_ctrl_rw_evict.114435821909560506958293945584600443068715926172144892491622412951646091980078\                                                                                                                             
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/6.flash_ctrl_rw_evict/latest/run.log                                                                                                       
                                                                                                                                                                                                                                            
                UVM_ERROR @ 86332.4 ns: (csr_utils_pkg.sv:456) [csr_utils_pkg::csr_rd_check.isolation_fork.unnamed$$_0] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: flash_ctrl_core_reg_block.err_code.rd_err reset value: 0x0    
                UVM_INFO @ 86332.4 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                    
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
        * 31.flash_ctrl_rw_evict.16036934559355294699539854679365243513442407076947871083640426682202452213112\                                                                                                                             
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/31.flash_ctrl_rw_evict/latest/run.log                                                                                                      
                                                                                                                                                                                                                                            
                UVM_ERROR @ 55185.2 ns: (csr_utils_pkg.sv:456) [csr_utils_pkg::csr_rd_check.isolation_fork.unnamed$$_0] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: flash_ctrl_core_reg_block.err_code.rd_err reset value: 0x0    
                UVM_INFO @ 55185.2 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                    
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
* `Job timed out after * minutes` has 1 failures:                                                                                                                                                                                           
    * Test flash_ctrl_intr_wr_slow_flash has 1 failures.                                                                                                                                                                                    
        * 5.flash_ctrl_intr_wr_slow_flash.92907951380700627916655513997512323762035165095444027789143307036902015929430\                                                                                                                    
         Log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/5.flash_ctrl_intr_wr_slow_flash/latest/run.log                                                                                                           
                                                                                                                                                                                                                                            
                Job timed out after 60 minutes                                                                                                                                                                                              
                                                                                                                                                                                                                                            
* `UVM_ERROR (flash_ctrl_otf_scoreboard.sv:376) [rdata_comp_bank1] *: obs:exp *cfa94e_bf5a8cd0:ffffffff_bf5a8cd* mismatch!!` has 1 failures:                                                                                                
    * Test flash_ctrl_intr_rd has 1 failures.                                                                                                                                                                                               
        * 11.flash_ctrl_intr_rd.19266094210998586745905270800642793949044652447060381476698276599545836420318\                                                                                                                              
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/11.flash_ctrl_intr_rd/latest/run.log                                                                                                       
                                                                                                                                                                                                                                            
                UVM_ERROR @ 1631082.6 ns: (flash_ctrl_otf_scoreboard.sv:376) [rdata_comp_bank1]    5: obs:exp    36cfa94e_bf5a8cd0:ffffffff_bf5a8cd0  mismatch!!                                                                            
                UVM_INFO @ 1631082.6 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                  
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
* `UVM_ERROR (flash_ctrl_otf_scoreboard.sv:376) [rdata_comp_bank1] *: obs:exp dbcc06bd_17dd3955:ffffffff_17dd* mismatch!!` has 1 failures:                                                                                                  
    * Test flash_ctrl_intr_rd has 1 failures.                                                                                                                                                                                               
        * 16.flash_ctrl_intr_rd.67908634499172563700750950043456855371716542109010340183537815228167198333696\                                                                                                                              
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/16.flash_ctrl_intr_rd/latest/run.log                                                                                                       
                                                                                                                                                                                                                                            
                UVM_ERROR @ 5568169.2 ns: (flash_ctrl_otf_scoreboard.sv:376) [rdata_comp_bank1]    5: obs:exp    dbcc06bd_17dd3955:ffffffff_17dd3955  mismatch!!                                                                            
                UVM_INFO @ 5568169.2 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                  
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
* `UVM_ERROR (flash_ctrl_otf_scoreboard.sv:376) [rdata_comp_bank1] *: obs:exp *d442bf1_61a8ed0f:ffffffff_61a8ed0f mismatch!!` has 1 failures:                                                                                               
    * Test flash_ctrl_intr_rd has 1 failures.                                                                                                                                                                                               
        * 38.flash_ctrl_intr_rd.50985353224349900357728681755204562808354352972017623287159296192101935750857\                                                                                                                              
          Line 103, in log /scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs/38.flash_ctrl_intr_rd/latest/run.log                                                                                                       
                                                                                                                                                                                                                                            
                UVM_ERROR @ 14678051.1 ns: (flash_ctrl_otf_scoreboard.sv:376) [rdata_comp_bank1]    6: obs:exp    5d442bf1_61a8ed0f:ffffffff_61a8ed0f  mismatch!!            
                UVM_INFO @ 14678051.1 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]                                                                                                                                                 
                --- UVM Report catcher Summary ---                                                                                                                                                                                          
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
INFO: [FlowCfg] [scratch_path]: [flash_ctrl] [/scratch/mgautschi/opentitan/scratch/flash_ctrl_dv/flash_ctrl-sim-vcs]                                                                                                                        
ERROR: [dvsim] Errors were encountered in this run.                                                                                                                                                                                         
                                                                                                                                                                                                                                            
          [   legend    ]: [Q: queued, D: dispatched, P: passed, F: failed, K: killed, T: total]                                                                                                                                            
00:00:27  [    build    ]: [Q: 0, D: 0, P: 2, F: 0, K: 0, T: 2] 100%                                                                                                                                                                        
01:42:11  [     run     ]: [Q: 0, D: 0, P: 1270, F: 13, K: 1, T: 1284] 100%                                                                                                                                                                 
01:42:36  [  cov_merge  ]: [Q: 0, D: 0, P: 1, F: 0, K: 0, T: 1] 100%                                                                                                                                                                        
01:42:51  [ cov_report  ]: [Q: 0, D: 0, P: 1, F: 0, K: 0, T: 1] 100%                                                                                                                                                                                                                                                              
 

@vogelpi
Copy link
Contributor

vogelpi commented Oct 27, 2025

Thanks for updating the PR and for providing the regressions results. This all makes sense now and I see that not only pass rate increased quite substantially but also the functional coverage went slightly up once more. This looks good to me!

@vogelpi vogelpi added the CherryPick:earlgrey_1.0.0 This PR should be cherry-picked to earlgrey_1.0.0 label Oct 27, 2025
@vogelpi vogelpi added this pull request to the merge queue Oct 27, 2025
Merged via the queue into lowRISC:master with commit cb622e0 Oct 27, 2025
49 checks passed
@lowrisc-ci
Copy link

lowrisc-ci bot commented Oct 27, 2025

Backport failed for earlgrey_1.0.0, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin earlgrey_1.0.0
git worktree add -d .worktree/backport-28555-to-earlgrey_1.0.0 origin/earlgrey_1.0.0
cd .worktree/backport-28555-to-earlgrey_1.0.0
git switch --create backport-28555-to-earlgrey_1.0.0
git cherry-pick -x 9581b9b742922756ced43cc2ceed907df26b46c4 d3aa4e9764fb730a0a1f61cf7b98ac3856914f22

@lowrisc-ci lowrisc-ci bot added the Manually CherryPick This PR should be manually cherry picked. label Oct 27, 2025
@lowrisc-ci
Copy link

lowrisc-ci bot commented Oct 27, 2025

Backport failed for earlgrey_1.0.0, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin earlgrey_1.0.0
git worktree add -d .worktree/backport-28555-to-earlgrey_1.0.0 origin/earlgrey_1.0.0
cd .worktree/backport-28555-to-earlgrey_1.0.0
git switch --create backport-28555-to-earlgrey_1.0.0
git cherry-pick -x 9581b9b742922756ced43cc2ceed907df26b46c4 d3aa4e9764fb730a0a1f61cf7b98ac3856914f22

@vogelpi vogelpi mentioned this pull request Oct 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CherryPick:earlgrey_1.0.0 This PR should be cherry-picked to earlgrey_1.0.0 Manually CherryPick This PR should be manually cherry picked.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants