|
| 1 | +What: /sys/kernel/debug/iommu/intel/iommu_regset |
| 2 | +Date: December 2023 |
| 3 | +Contact: Jingqi Liu < [email protected]> |
| 4 | +Description: |
| 5 | + This file dumps all the register contents for each IOMMU device. |
| 6 | + |
| 7 | + Example in Kabylake: |
| 8 | + |
| 9 | + :: |
| 10 | + |
| 11 | + $ sudo cat /sys/kernel/debug/iommu/intel/iommu_regset |
| 12 | + |
| 13 | + IOMMU: dmar0 Register Base Address: 26be37000 |
| 14 | + |
| 15 | + Name Offset Contents |
| 16 | + VER 0x00 0x0000000000000010 |
| 17 | + GCMD 0x18 0x0000000000000000 |
| 18 | + GSTS 0x1c 0x00000000c7000000 |
| 19 | + FSTS 0x34 0x0000000000000000 |
| 20 | + FECTL 0x38 0x0000000000000000 |
| 21 | + |
| 22 | + [...] |
| 23 | + |
| 24 | + IOMMU: dmar1 Register Base Address: fed90000 |
| 25 | + |
| 26 | + Name Offset Contents |
| 27 | + VER 0x00 0x0000000000000010 |
| 28 | + GCMD 0x18 0x0000000000000000 |
| 29 | + GSTS 0x1c 0x00000000c7000000 |
| 30 | + FSTS 0x34 0x0000000000000000 |
| 31 | + FECTL 0x38 0x0000000000000000 |
| 32 | + |
| 33 | + [...] |
| 34 | + |
| 35 | + IOMMU: dmar2 Register Base Address: fed91000 |
| 36 | + |
| 37 | + Name Offset Contents |
| 38 | + VER 0x00 0x0000000000000010 |
| 39 | + GCMD 0x18 0x0000000000000000 |
| 40 | + GSTS 0x1c 0x00000000c7000000 |
| 41 | + FSTS 0x34 0x0000000000000000 |
| 42 | + FECTL 0x38 0x0000000000000000 |
| 43 | + |
| 44 | + [...] |
| 45 | + |
| 46 | +What: /sys/kernel/debug/iommu/intel/ir_translation_struct |
| 47 | +Date: December 2023 |
| 48 | +Contact: Jingqi Liu < [email protected]> |
| 49 | +Description: |
| 50 | + This file dumps the table entries for Interrupt |
| 51 | + remapping and Interrupt posting. |
| 52 | + |
| 53 | + Example in Kabylake: |
| 54 | + |
| 55 | + :: |
| 56 | + |
| 57 | + $ sudo cat /sys/kernel/debug/iommu/intel/ir_translation_struct |
| 58 | + |
| 59 | + Remapped Interrupt supported on IOMMU: dmar0 |
| 60 | + IR table address:100900000 |
| 61 | + |
| 62 | + Entry SrcID DstID Vct IRTE_high IRTE_low |
| 63 | + 0 00:0a.0 00000080 24 0000000000040050 000000800024000d |
| 64 | + 1 00:0a.0 00000001 ef 0000000000040050 0000000100ef000d |
| 65 | + |
| 66 | + Remapped Interrupt supported on IOMMU: dmar1 |
| 67 | + IR table address:100300000 |
| 68 | + Entry SrcID DstID Vct IRTE_high IRTE_low |
| 69 | + 0 00:02.0 00000002 26 0000000000040010 000000020026000d |
| 70 | + |
| 71 | + [...] |
| 72 | + |
| 73 | + **** |
| 74 | + |
| 75 | + Posted Interrupt supported on IOMMU: dmar0 |
| 76 | + IR table address:100900000 |
| 77 | + Entry SrcID PDA_high PDA_low Vct IRTE_high IRTE_low |
| 78 | + |
| 79 | +What: /sys/kernel/debug/iommu/intel/dmar_translation_struct |
| 80 | +Date: December 2023 |
| 81 | +Contact: Jingqi Liu < [email protected]> |
| 82 | +Description: |
| 83 | + This file dumps Intel IOMMU DMA remapping tables, such |
| 84 | + as root table, context table, PASID directory and PASID |
| 85 | + table entries in debugfs. For legacy mode, it doesn't |
| 86 | + support PASID, and hence PASID field is defaulted to |
| 87 | + '-1' and other PASID related fields are invalid. |
| 88 | + |
| 89 | + Example in Kabylake: |
| 90 | + |
| 91 | + :: |
| 92 | + |
| 93 | + $ sudo cat /sys/kernel/debug/iommu/intel/dmar_translation_struct |
| 94 | + |
| 95 | + IOMMU dmar1: Root Table Address: 0x103027000 |
| 96 | + B.D.F Root_entry |
| 97 | + 00:02.0 0x0000000000000000:0x000000010303e001 |
| 98 | + |
| 99 | + Context_entry |
| 100 | + 0x0000000000000102:0x000000010303f005 |
| 101 | + |
| 102 | + PASID PASID_table_entry |
| 103 | + -1 0x0000000000000000:0x0000000000000000:0x0000000000000000 |
| 104 | + |
| 105 | + IOMMU dmar0: Root Table Address: 0x103028000 |
| 106 | + B.D.F Root_entry |
| 107 | + 00:0a.0 0x0000000000000000:0x00000001038a7001 |
| 108 | + |
| 109 | + Context_entry |
| 110 | + 0x0000000000000000:0x0000000103220e7d |
| 111 | + |
| 112 | + PASID PASID_table_entry |
| 113 | + 0 0x0000000000000000:0x0000000000800002:0x00000001038a5089 |
| 114 | + |
| 115 | + [...] |
| 116 | + |
| 117 | +What: /sys/kernel/debug/iommu/intel/invalidation_queue |
| 118 | +Date: December 2023 |
| 119 | +Contact: Jingqi Liu < [email protected]> |
| 120 | +Description: |
| 121 | + This file exports invalidation queue internals of each |
| 122 | + IOMMU device. |
| 123 | + |
| 124 | + Example in Kabylake: |
| 125 | + |
| 126 | + :: |
| 127 | + |
| 128 | + $ sudo cat /sys/kernel/debug/iommu/intel/invalidation_queue |
| 129 | + |
| 130 | + Invalidation queue on IOMMU: dmar0 |
| 131 | + Base: 0x10022e000 Head: 20 Tail: 20 |
| 132 | + Index qw0 qw1 qw2 |
| 133 | + 0 0000000000000014 0000000000000000 0000000000000000 |
| 134 | + 1 0000000200000025 0000000100059c04 0000000000000000 |
| 135 | + 2 0000000000000014 0000000000000000 0000000000000000 |
| 136 | + |
| 137 | + qw3 status |
| 138 | + 0000000000000000 0000000000000000 |
| 139 | + 0000000000000000 0000000000000000 |
| 140 | + 0000000000000000 0000000000000000 |
| 141 | + |
| 142 | + [...] |
| 143 | + |
| 144 | + Invalidation queue on IOMMU: dmar1 |
| 145 | + Base: 0x10026e000 Head: 32 Tail: 32 |
| 146 | + Index qw0 qw1 status |
| 147 | + 0 0000000000000004 0000000000000000 0000000000000000 |
| 148 | + 1 0000000200000025 0000000100059804 0000000000000000 |
| 149 | + 2 0000000000000011 0000000000000000 0000000000000000 |
| 150 | + |
| 151 | + [...] |
| 152 | + |
| 153 | +What: /sys/kernel/debug/iommu/intel/dmar_perf_latency |
| 154 | +Date: December 2023 |
| 155 | +Contact: Jingqi Liu < [email protected]> |
| 156 | +Description: |
| 157 | + This file is used to control and show counts of |
| 158 | + execution time ranges for various types per DMAR. |
| 159 | + |
| 160 | + Firstly, write a value to |
| 161 | + /sys/kernel/debug/iommu/intel/dmar_perf_latency |
| 162 | + to enable sampling. |
| 163 | + |
| 164 | + The possible values are as follows: |
| 165 | + |
| 166 | + * 0 - disable sampling all latency data |
| 167 | + |
| 168 | + * 1 - enable sampling IOTLB invalidation latency data |
| 169 | + |
| 170 | + * 2 - enable sampling devTLB invalidation latency data |
| 171 | + |
| 172 | + * 3 - enable sampling intr entry cache invalidation latency data |
| 173 | + |
| 174 | + Next, read /sys/kernel/debug/iommu/intel/dmar_perf_latency gives |
| 175 | + a snapshot of sampling result of all enabled monitors. |
| 176 | + |
| 177 | + Examples in Kabylake: |
| 178 | + |
| 179 | + :: |
| 180 | + |
| 181 | + 1) Disable sampling all latency data: |
| 182 | + |
| 183 | + $ sudo echo 0 > /sys/kernel/debug/iommu/intel/dmar_perf_latency |
| 184 | + |
| 185 | + 2) Enable sampling IOTLB invalidation latency data |
| 186 | + |
| 187 | + $ sudo echo 1 > /sys/kernel/debug/iommu/intel/dmar_perf_latency |
| 188 | + |
| 189 | + $ sudo cat /sys/kernel/debug/iommu/intel/dmar_perf_latency |
| 190 | + |
| 191 | + IOMMU: dmar0 Register Base Address: 26be37000 |
| 192 | + <0.1us 0.1us-1us 1us-10us 10us-100us 100us-1ms |
| 193 | + inv_iotlb 0 0 0 0 0 |
| 194 | + |
| 195 | + 1ms-10ms >=10ms min(us) max(us) average(us) |
| 196 | + inv_iotlb 0 0 0 0 0 |
| 197 | + |
| 198 | + [...] |
| 199 | + |
| 200 | + IOMMU: dmar2 Register Base Address: fed91000 |
| 201 | + <0.1us 0.1us-1us 1us-10us 10us-100us 100us-1ms |
| 202 | + inv_iotlb 0 0 18 0 0 |
| 203 | + |
| 204 | + 1ms-10ms >=10ms min(us) max(us) average(us) |
| 205 | + inv_iotlb 0 0 2 2 2 |
| 206 | + |
| 207 | + 3) Enable sampling devTLB invalidation latency data |
| 208 | + |
| 209 | + $ sudo echo 2 > /sys/kernel/debug/iommu/intel/dmar_perf_latency |
| 210 | + |
| 211 | + $ sudo cat /sys/kernel/debug/iommu/intel/dmar_perf_latency |
| 212 | + |
| 213 | + IOMMU: dmar0 Register Base Address: 26be37000 |
| 214 | + <0.1us 0.1us-1us 1us-10us 10us-100us 100us-1ms |
| 215 | + inv_devtlb 0 0 0 0 0 |
| 216 | + |
| 217 | + >=10ms min(us) max(us) average(us) |
| 218 | + inv_devtlb 0 0 0 0 |
| 219 | + |
| 220 | + [...] |
| 221 | + |
| 222 | +What: /sys/kernel/debug/iommu/intel/<bdf>/domain_translation_struct |
| 223 | +Date: December 2023 |
| 224 | +Contact: Jingqi Liu < [email protected]> |
| 225 | +Description: |
| 226 | + This file dumps a specified page table of Intel IOMMU |
| 227 | + in legacy mode or scalable mode. |
| 228 | + |
| 229 | + For a device that only supports legacy mode, dump its |
| 230 | + page table by the debugfs file in the debugfs device |
| 231 | + directory. e.g. |
| 232 | + /sys/kernel/debug/iommu/intel/0000:00:02.0/domain_translation_struct. |
| 233 | + |
| 234 | + For a device that supports scalable mode, dump the |
| 235 | + page table of specified pasid by the debugfs file in |
| 236 | + the debugfs pasid directory. e.g. |
| 237 | + /sys/kernel/debug/iommu/intel/0000:00:02.0/1/domain_translation_struct. |
| 238 | + |
| 239 | + Examples in Kabylake: |
| 240 | + |
| 241 | + :: |
| 242 | + |
| 243 | + 1) Dump the page table of device "0000:00:02.0" that only supports legacy mode. |
| 244 | + |
| 245 | + $ sudo cat /sys/kernel/debug/iommu/intel/0000:00:02.0/domain_translation_struct |
| 246 | + |
| 247 | + Device 0000:00:02.0 @0x1017f8000 |
| 248 | + IOVA_PFN PML5E PML4E |
| 249 | + 0x000000008d800 | 0x0000000000000000 0x00000001017f9003 |
| 250 | + 0x000000008d801 | 0x0000000000000000 0x00000001017f9003 |
| 251 | + 0x000000008d802 | 0x0000000000000000 0x00000001017f9003 |
| 252 | + |
| 253 | + PDPE PDE PTE |
| 254 | + 0x00000001017fa003 0x00000001017fb003 0x000000008d800003 |
| 255 | + 0x00000001017fa003 0x00000001017fb003 0x000000008d801003 |
| 256 | + 0x00000001017fa003 0x00000001017fb003 0x000000008d802003 |
| 257 | + |
| 258 | + [...] |
| 259 | + |
| 260 | + 2) Dump the page table of device "0000:00:0a.0" with PASID "1" that |
| 261 | + supports scalable mode. |
| 262 | + |
| 263 | + $ sudo cat /sys/kernel/debug/iommu/intel/0000:00:0a.0/1/domain_translation_struct |
| 264 | + |
| 265 | + Device 0000:00:0a.0 with pasid 1 @0x10c112000 |
| 266 | + IOVA_PFN PML5E PML4E |
| 267 | + 0x0000000000000 | 0x0000000000000000 0x000000010df93003 |
| 268 | + 0x0000000000001 | 0x0000000000000000 0x000000010df93003 |
| 269 | + 0x0000000000002 | 0x0000000000000000 0x000000010df93003 |
| 270 | + |
| 271 | + PDPE PDE PTE |
| 272 | + 0x0000000106ae6003 0x0000000104b38003 0x0000000147c00803 |
| 273 | + 0x0000000106ae6003 0x0000000104b38003 0x0000000147c01803 |
| 274 | + 0x0000000106ae6003 0x0000000104b38003 0x0000000147c02803 |
| 275 | + |
| 276 | + [...] |
0 commit comments