PCIe 3.0 (x8 lanes) support for ADRV9009ZU11EG/ADRV2CRR_FMC

[cristianmihaipopa]

PR Description

The ADRV9009ZU11EG/ADRV2CRR_FMC design gains a new data offload pipeline, in addition to the NIC enabled by Corundum.

PCIe 3.0 with 8 differential data lanes (8× RX_P/TX_P and 8× RX_N/TX_N) is now supported using the Xilinx XDMA IP, on the ADRV9009ZU11EG/ADRV2CRR_FMC variant only. It cannot be built alongside the Corundum variant in the same HDL design.

The new PCIe design introduces the following changes:

• The XDMA IP acts as a PCIe transaction-translation bridge between the FPGA and the host PC. By default the XDMA has its own DMA engines; in this design they are disabled and we use our own AXI DMAC engines and connect them directly to the XDMA bridge
• The base HDL design for ADRV9009ZU11EG/ADRV2CRR_FMC has been significantly changed. The FPGA's DDR4 memory is bypassed entirely: all data coming from the JESD204 IPs is streamed directly into the host PC's memory through the existing AXI DMAC IPs, via the XDMA IP, when the host requests a transaction
• Many IPs and signals were rerouted from the PS to the XDMA, so the PS gives up most of its control in favor of host control over the IPs. All AXI DMAC and JESD204 IPs are now controlled by the host through the XDMA bridge. The SPI and GPIO signals were rerouted to new AXI SPI and AXI GPIO controllers, which are likewise connected to the XDMA bridge
• To satisfy the MSI/MSI-X interrupt handling of the XDMA IP, which is essential to the operation of the PCIe bridge, a new custom AXI IP was implemented: library/axi_pcie_intc. It bridges the level-sensitive fabric interrupt lines to the endpoint's usr_irq_req/usr_irq_ack handshake required by PG195 (XDMA XIli), giving each source its own MSI/MSI-X vector
• New PCIe-related .tcl methods were added to projects/scripts/adi_board.tcl to support future PCIe projects:
  • ad_pcie_interconnect: connects a peripheral's AXI-Lite slave to the XDMA M_AXI_B (BAR0) master through the pcie_ctrl_sc SmartConnect. Creates the SmartConnect on the first call and grows NUM_MI on each subsequent call; detaches the IP from any previous CPU-side wiring, rewires it to pcie_axi_clk, and assigns its address segment within BAR0
  • ad_pcie_saxi_interconnect: connects an AXI master port (DMA data, scatter-gather, or transceiver eye-scan) to the XDMA S_AXI_B slave through the pcie_saxi_sc SmartConnect, so the DMAC writes straight to host RAM. Creates the SmartConnect on the first call and grows NUM_SI afterwards, adding clock slots as new master-side clock domains appear
  • ad_pcie_interrupt: connects a peripheral IRQ line to the axi_pcie_intc controller. Creates the controller on the first call, maps it into the XDMA M_AXI_B space, and grows NUM_VECTORS/xdma_num_usr_irq as more IPs attach. By default each source gets its own vector (SRC_PER_VEC=1), so the host handler needs no dispatch read; an optional argument pins a source to a fixed vector to keep hwirq numbering stable across build variants
• To provide network access to the PS from the host over PCIe, a bidirectional PS <-> host network link was added using four axi_dmac instances (axi_host_net_tx/rx_dma on the PCIe side, axi_ps_net_tx/rx_dma on the PS side) and two asynchronous AXI-Stream FIFOs bridging the 250 MHz PCIe and 100 MHz PS clock domains. This carries IP traffic (SSH, remote IIO, firmware updates) over the PCIe link without a separate Ethernet cable.
• Fixes were made to library/axi_dmac to support the network link: a new HAS_AXIS_TKEEP parameter wires m_axis_keep/s_axis_keep to the real byte strobe (previously hardwired to all-ones), dropping DMA_LENGTH_ALIGN to zero so transfer lengths are byte-exact, required for arbitrary-size network packets. This also fixes a zero-width replication bug in the dest_burst_len_data initialization. In addition, axi_dmac_constr.ttcl gains two set_false_path constraints for the CDC synchronizer pairs introduced when scatter-gather mode (DMA_SG_TRANSFER=1) is enabled.

Both the PCIe project implementation (project documentation + README.md) and the custom interrupt controller have been documented. To build the project, run the following in projects/adrv9009zu11eg/adrv2crr_fmc:

make PCIE=1

PR Type

• Bug fix (change that fixes an issue)
• New feature (change that adds new functionality)
• Breaking change (has dependencies in other repos or will cause CI to fail)
• Documentation

PR Checklist

• I have followed the code style guidelines
• I have performed a self-review of changes
• I have compiled all hdl projects and libraries affected by this PR
• I have tested in hardware affected projects, at least on relevant boards
• I have commented my code, at least hard-to-understand parts
• I have signed off all commits from this PR
• I have updated the documentation (wiki pages, ReadMe files, Copyright etc)
• I have not introduced new Warnings/Critical Warnings on compilation
• I have added new hdl testbenches or updated existing ones

[cristianmihaipopa]

V1: Fixed check_guideline script highlighted issues (comments, copyright years, indentation).

[cristianmihaipopa]

V2:

• GPIO[31:0] were rerouted back to the PS. Previously, they were routed to the XDMA bridge. These GPIOs are carrier specific (HMC7044 carrier, leds, buttons etc.), no need for them to be controlled from the Host.
• removed axi_gpio2, now we use only axi_gpio1, in dual mode
• moved axi_spi from address 0x84040000 to 0x84030000 (old address of axi_gpio2)