Support Fully Asynchronous DMAs (#114)

This pull request introduces improvements to the DMA code generation for several backends (`SnitchDma` and `Mchan`), to enable proper double-buffering by overlapping DMA transfers with kernel calls. Additionally, it refactors the profiling infrastructure for Snitch tiling and improves the readability of the generated code by adding some helpful comments.

### Added
- Profiling-aware tiling mixins: `ProfilingDoubleBufferingTilingMixIn` and `ProfilingSingleBufferingTilingMixIn` integrated into the Snitch and PULP tiling generators.  
- Optional comments injected into generated code (DMA templates `_initTemplate`, `_allocTemplate`, `_waitTemplate`) for improved readability and traceability.  
- Profiling instrumentation for tile-level DMA and kernel execution integrated into the tiling passes for Snitch backends.

### Changed
- Refactored DMA code-generation in the backends (`SnitchDma`, `Mchan`) to enable full overlap of DMA and compute for double-buffering, replacing the earlier (incorrect) synchronization scheme.  
- Simplified tiling generator logic by leveraging the profiling mix-ins and consolidating redundant template assignments, improving maintainability and code generation clarity.  
- Improved the waiting-strategy architecture: introduced `PerTensorWaitingStrategy` alongside existing `TensorGroupWaitingStrategy`, enabling finer-grained control of DMA futures in DB mode.  

### Fixed
- Corrected DMA synchronization bug that previously prevented effective overlapping of transfer and compute in DB mode, especially noticeable for memory-bound kernels.

Author: Xeratec

CHANGELOG.md

@@ -4,6 +4,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 ## Unreleased (Planned Release Target: v0.2.1)
 
 ### List of Pull Requests
+- Support Fully Asynchronous DMAs [#114](https://github.com/pulp-platform/Deeploy/pull/114)
 - Disallow shape inference [#128](https://github.com/pulp-platform/Deeploy/pull/128)
 - Remove memory-aware node bindings [#123](https://github.com/pulp-platform/Deeploy/pull/123)
 - Fix missing const's layout transformation and refactor NCHWtoNHWC passes [#122](https://github.com/pulp-platform/Deeploy/pull/122)
@@ -55,6 +56,8 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - RequantHelpers.py for Neureka's TileConstraints
 - Added assertion that all the graph tensors after lowering have a shape annotated
 - Added testFloatGEMMnobias
+- Profiling support and optional comments in generated DMA code for better traceability
+- Added new waiting-strategy logic with fine-grained `PerTensorWaitingStrategy`
 
 ### Changed
 - Replaced platform-specific tags (`*-amd64`, `*-arm64`) with direct digest references in `Noelware/docker-manifest-action`.
@@ -91,6 +94,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Removed Wmem variants of bindings and tile constraints from Neureka
 - Disabled ICCT_ITA_8 MemPool test because it was using a lowering that created shapeless tensors
 - Added missing shape annotation to the testTypeInferenceDifferentTypes
+- Refactored DMA code generation (`SnitchDma`, `Mchan`) to correctly overlap transfers and compute in double-buffering mode
 
 ### Fixed
 - Prevent node duplication for graphs generated via GraphSurgeon
@@ -105,6 +109,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Missing layout transformation of the const's (bias, mul, add, shift in Conv/RequantizedConv)
 - Keep mul/add rank of requantized Neureka tile constraints
 - Fix bias hoisting in generic GEMM with no bias
+- DMA synchronization bug causing reduced DB performance on memory-bound kernels.
 
 ### Removed
 - Delete outdated and unused `.gitlab-ci.yml` file

Deeploy/Targets/PULPOpen/CodeTransformationPasses/PULPClusterTiling.py

@@ -7,22 +7,20 @@
 from Deeploy.DeeployTypes import CodeGenVerbosity, CodeTransformationPass, ExecutionBlock, NetworkContext, _NoVerbosity
 from Deeploy.TilingExtension.AsyncDma import AsyncDma
 from Deeploy.TilingExtension.CodeTransformationPasses.DoubleBufferingTilingCodeGeneration import \
-    DoubleBufferingTilingCodeGeneration
+    DoubleBufferingTilingCodeGeneration, ProfilingDoubleBufferingTilingMixIn
 from Deeploy.TilingExtension.CodeTransformationPasses.SingleBufferingTilingCodeGeneration import \
-    SingleBufferingTilingCodeGeneration
-from Deeploy.TilingExtension.CodeTransformationPasses.TilingPrototypes import DoubleBufferingTilingMixIn, \
-    ProfilingDoubleBufferingTilingMixIn, ProfilingSingleBufferingTilingMixIn, SingleBufferingTilingMixIn
+    ProfilingSingleBufferingTilingMixIn, SingleBufferingTilingCodeGeneration
 
 
-class PULPClusterTilingGenerationSB(SingleBufferingTilingCodeGeneration, SingleBufferingTilingMixIn):
+class PULPClusterTilingGenerationSB(SingleBufferingTilingCodeGeneration):
     pass
 
 
 class ProfilingPULPClusterTilingGenerationSB(SingleBufferingTilingCodeGeneration, ProfilingSingleBufferingTilingMixIn):
     pass
 
 
-class PULPClusterTilingGenerationDB(DoubleBufferingTilingCodeGeneration, DoubleBufferingTilingMixIn):
+class PULPClusterTilingGenerationDB(DoubleBufferingTilingCodeGeneration):
     pass
 
 

Deeploy/Targets/PULPOpen/CodeTransformationPasses/PULPL3Tiling.py

@@ -7,22 +7,20 @@
 from Deeploy.DeeployTypes import CodeGenVerbosity, CodeTransformationPass, ExecutionBlock, NetworkContext, _NoVerbosity
 from Deeploy.TilingExtension.AsyncDma import AsyncDma
 from Deeploy.TilingExtension.CodeTransformationPasses.DoubleBufferingTilingCodeGeneration import \
-    DoubleBufferingTilingCodeGeneration
+    DoubleBufferingTilingCodeGeneration, ProfilingDoubleBufferingTilingMixIn
 from Deeploy.TilingExtension.CodeTransformationPasses.SingleBufferingTilingCodeGeneration import \
-    SingleBufferingTilingCodeGeneration
-from Deeploy.TilingExtension.CodeTransformationPasses.TilingPrototypes import DoubleBufferingTilingMixIn, \
-    ProfilingDoubleBufferingTilingMixIn, ProfilingSingleBufferingTilingMixIn, SingleBufferingTilingMixIn
+    ProfilingSingleBufferingTilingMixIn, SingleBufferingTilingCodeGeneration
 
 
-class PULPL3TilingGenerationSB(SingleBufferingTilingCodeGeneration, SingleBufferingTilingMixIn):
+class PULPL3TilingGenerationSB(SingleBufferingTilingCodeGeneration):
     pass
 
 
 class ProfilingPULPL3TilingGenerationSB(SingleBufferingTilingCodeGeneration, ProfilingSingleBufferingTilingMixIn):
     pass
 
 
-class PULPL3TilingGenerationDB(DoubleBufferingTilingCodeGeneration, DoubleBufferingTilingMixIn):
+class PULPL3TilingGenerationDB(DoubleBufferingTilingCodeGeneration):
     pass
 
 

Deeploy/Targets/PULPOpen/DMA/L3Dma.py

@@ -12,9 +12,16 @@
 
 class L3DmaFuture(Future):
 
-    _initTemplate = NodeTemplate("pi_cl_ram_req_t ${name};")
+    _initTemplate = NodeTemplate("pi_cl_ram_req_t ${name} = {0};")
+
     _deinitTemplate = NodeTemplate("")
-    _waitTemplate = NodeTemplate("pi_cl_ram_copy_wait(&${name});")
+
+    _allocTemplate = NodeTemplate("")
+
+    _waitTemplate = NodeTemplate("""
+    if (${name}.size != 0) {
+        pi_cl_ram_copy_wait(&${name});
+    }""")
 
 
 class L3Dma(AsyncDma):

Deeploy/Targets/PULPOpen/DMA/MchanDma.py

@@ -6,14 +6,23 @@
 from typing import Dict, Tuple
 
 from Deeploy.DeeployTypes import NetworkContext, NodeTemplate, OperatorRepresentation, VariableBuffer
-from Deeploy.TilingExtension.AsyncDma import AsyncDma, DmaDirection, Future, TensorGroupWaitingStrategy
+from Deeploy.TilingExtension.AsyncDma import AsyncDma, DirectionWaitingStrategy, DmaDirection, Future
 
 
 class MchanChannelFuture(Future):
 
-    _initTemplate = NodeTemplate("uint32_t ${name} = mchan_channel_alloc();")
-    _deinitTemplate = NodeTemplate("mchan_channel_free(${name});")
-    _waitTemplate = NodeTemplate("mchan_channel_wait(${name});")
+    _initTemplate = NodeTemplate("uint32_t ${name} = (uint32_t) -1;")
+
+    _deinitTemplate = NodeTemplate("")
+
+    _allocTemplate = NodeTemplate("${name} = mchan_channel_alloc();")
+
+    _waitTemplate = NodeTemplate("""
+if (${name} <= MCHAN_CHANNEL_ID_MAX) {
+    mchan_channel_wait(${name});
+    mchan_channel_free(${name});
+}
+""")
 
 
 class MchanDma(AsyncDma):
@@ -22,7 +31,7 @@ class MchanDma(AsyncDma):
         1: NodeTemplate("mchan_transfer_1d(${cmd}, ${loc}, ${ext});"),
         2: NodeTemplate("mchan_transfer_2d_ext_strided(${cmd}, ${loc}, ${ext}, ${size_1d}, ${stride_2d});"),
     }
-    _waitingStrategy = TensorGroupWaitingStrategy(MchanChannelFuture, "channel_id")
+    _waitingStrategy = DirectionWaitingStrategy(MchanChannelFuture, "channel")
 
     def __init__(self, transferTemplates: Dict[int, NodeTemplate] = _transferTemplates) -> None:
         super().__init__(transferTemplates)

Deeploy/Targets/Snitch/Bindings.py

@@ -14,7 +14,7 @@
 from Deeploy.Targets.Generic.Templates import iNoNormTemplate
 from Deeploy.Targets.Generic.TypeCheckers import AddChecker, GEMMChecker, RQAddChecker, SoftmaxChecker, iNoNormChecker
 from Deeploy.Targets.Snitch.CodeTransformationPasses import SnitchClusterTiling, SnitchCoreFilterPass, \
-    SnitchProfileExecutionBlockPass, SnitchSynchCoresPass
+    SnitchSynchCoresPass
 from Deeploy.Targets.Snitch.DMA.SnitchDma import SnitchDma
 from Deeploy.Targets.Snitch.Templates import AddTemplate, FloatGemmTemplate, RQAddTemplate, iSoftmaxTemplate
 from Deeploy.Targets.Snitch.Templates.FloatSoftmaxTemplate import FloatSoftmax_Template
@@ -37,7 +37,6 @@
 
 TiledTransformer = CodeTransformation([
     SnitchCoreFilterPass("compute"),
-    SnitchProfileExecutionBlockPass(),
     TilingVariableReplacement("L1"),
     TilingCallClosure(writeback = False),
     SnitchSynchCoresPass(),

Deeploy/Targets/Snitch/CodeTransformationPasses/SnitchClusterTiling.py

@@ -4,38 +4,55 @@
 
 from typing import Tuple
 
-from Deeploy.DeeployTypes import CodeGenVerbosity, CodeTransformationPass, ExecutionBlock, NetworkContext, _NoVerbosity
+from Deeploy.DeeployTypes import CodeGenVerbosity, CodeTransformationPass, ExecutionBlock, NetworkContext, \
+    NodeTemplate, _NoVerbosity
 from Deeploy.TilingExtension.AsyncDma import AsyncDma
 from Deeploy.TilingExtension.CodeTransformationPasses.DoubleBufferingTilingCodeGeneration import \
-    DoubleBufferingTilingCodeGeneration
+    DoubleBufferingTilingCodeGeneration, ProfilingDoubleBufferingTilingMixIn
 from Deeploy.TilingExtension.CodeTransformationPasses.SingleBufferingTilingCodeGeneration import \
-    SingleBufferingTilingCodeGeneration
-from Deeploy.TilingExtension.CodeTransformationPasses.TilingPrototypes import DoubleBufferingTilingMixIn, \
-    SingleBufferingTilingMixIn
+    ProfilingSingleBufferingTilingMixIn, SingleBufferingTilingCodeGeneration
 
 
-class SnitchClusterTilingSB(SingleBufferingTilingCodeGeneration, SingleBufferingTilingMixIn):
+class SnitchClusterTilingSB(SingleBufferingTilingCodeGeneration):
     pass
 
 
-class SnitchClusterTilingDB(DoubleBufferingTilingCodeGeneration, DoubleBufferingTilingMixIn):
+class SnitchClusterTilingDB(DoubleBufferingTilingCodeGeneration):
     pass
 
 
+class ProfilingSnitchClusterTilingSB(SingleBufferingTilingCodeGeneration, ProfilingSingleBufferingTilingMixIn):
+    _printCycleDifference = NodeTemplate(r"""
+    printf("%s%u][Core %d] %s%u%s", ${prefixStr}, ${profileIdxVar}, snrt_global_core_idx(), "${flavorStr}", \
+    ${measurementsEnd}[${profileIdxVar}] - ${measurementsStart}[${profileIdxVar}], ${suffixStr});
+    """)
+
+
+class ProfilingSnitchClusterTilingDB(DoubleBufferingTilingCodeGeneration, ProfilingDoubleBufferingTilingMixIn):
+    _printCycleDifference = NodeTemplate(r"""
+    printf("%s%u][Core %d] %s%u%s", ${prefixStr}, ${profileIdxVar}, snrt_global_core_idx(), "${flavorStr}", \
+    ${measurementsEnd}[${profileIdxVar}] - ${measurementsStart}[${profileIdxVar}], ${suffixStr});
+    """)
+
+
 class SnitchClusterTiling(CodeTransformationPass):
 
     def __init__(self, externalMemory: str, localMemory: str, dma: AsyncDma):
         self.SB = SnitchClusterTilingSB(externalMemory, localMemory, dma)
+        self.profilingSB = ProfilingSnitchClusterTilingSB(externalMemory, localMemory, dma)
+
         self.DB = SnitchClusterTilingDB(externalMemory, localMemory, dma)
+        self.profilingDB = ProfilingSnitchClusterTilingDB(externalMemory, localMemory, dma)
 
     def apply(self,
               ctxt: NetworkContext,
               executionBlock: ExecutionBlock,
               name: str,
               verbose: CodeGenVerbosity = _NoVerbosity) -> Tuple[NetworkContext, ExecutionBlock]:
         if verbose.tilingProfiling:
-            raise NotImplementedError("Profiling not implemented for L2")
-
-        ctxt, executionBlock = self.SB.apply(ctxt, executionBlock, name)
-        ctxt, executionBlock = self.DB.apply(ctxt, executionBlock, name)
+            ctxt, executionBlock = self.profilingSB.apply(ctxt, executionBlock, name)
+            ctxt, executionBlock = self.profilingDB.apply(ctxt, executionBlock, name)
+        else:
+            ctxt, executionBlock = self.SB.apply(ctxt, executionBlock, name)
+            ctxt, executionBlock = self.DB.apply(ctxt, executionBlock, name)
         return ctxt, executionBlock

Deeploy/Targets/Snitch/DMA/SnitchDma.py

@@ -5,31 +5,41 @@
 from typing import Dict, Tuple
 
 from Deeploy.DeeployTypes import NetworkContext, NodeTemplate, OperatorRepresentation, VariableBuffer
-from Deeploy.TilingExtension.AsyncDma import AsyncDma, DmaDirection, Future, TensorGroupWaitingStrategy
+from Deeploy.TilingExtension.AsyncDma import AsyncDma, DmaDirection, Future, PerTensorWaitingStrategy
 
 
 class SnitchBarrierFuture(Future):
     _initTemplate = NodeTemplate("")
     _deinitTemplate = NodeTemplate("")
+    _allocTemplate = NodeTemplate("")
     _waitTemplate = NodeTemplate("if (snrt_is_dm_core()) snrt_dma_wait_all();")
 
 
 # LMACAN: TODO: Add single transfer waiting
 class SnitchFuture(Future):
-    _initTemplate = NodeTemplate("uint16_t ${name};")
+    _initTemplate = NodeTemplate("snrt_dma_txid_t ${name} = (snrt_dma_txid_t) -1;")
+
     _deinitTemplate = NodeTemplate("")
-    _waitTemplate = NodeTemplate("if (snrt_is_dm_core()) snrt_dma_wait(${name});")
+
+    _allocTemplate = NodeTemplate("")
+
+    _waitTemplate = NodeTemplate(
+        "if ( (${name} != ( (snrt_dma_txid_t) -1) ) && snrt_is_dm_core() ) snrt_dma_wait(${name});")
 
 
 class SnitchDma(AsyncDma):
 
     _transferTemplates = {
         2:
-            NodeTemplate(
-                "if (snrt_is_dm_core()) snrt_dma_start_2d(${dest}, ${src}, ${size}, ${stride_dest}, ${stride_src}, ${repeat});"
-            ),
+            NodeTemplate("""
+            if (snrt_is_dm_core()) {
+                ${future} = snrt_dma_start_2d(${dest}, ${src}, ${size}, ${stride_dest}, ${stride_src}, ${repeat});
+                // WIESEP: Hack as otherwise the last commited DMA transaction ID can never be resolved.
+                snrt_dma_start_2d(${dest}, ${dest}, 1, 0, 0, 0);
+            }
+            """),
     }
-    _waitingStrategy = TensorGroupWaitingStrategy(SnitchBarrierFuture, "")
+    _waitingStrategy = PerTensorWaitingStrategy(SnitchFuture)
 
     def __init__(self, transferTemplates: Dict[int, NodeTemplate] = _transferTemplates) -> None:
         super().__init__(transferTemplates)
@@ -43,13 +53,13 @@ def checkTransfer(self, ctxt: NetworkContext, externalBuffer: VariableBuffer, lo
     def transferOpRepr(self, externalBuffer: VariableBuffer, localBuffer: VariableBuffer, shape: Tuple[int, ...],
                        strideExt: Tuple[int, ...], strideLoc: Tuple[int, ...], direction: DmaDirection,
                        future: Future) -> OperatorRepresentation:
-        _ = future
         operatorRepresentation: OperatorRepresentation = {
             "dest": localBuffer.name if direction == "ExternalToLocal" else externalBuffer.name,
             "src": externalBuffer.name if direction == "ExternalToLocal" else localBuffer.name,
             "repeat": shape[0],
             "size": shape[1],
             "stride_dest": strideLoc[0] if direction == "ExternalToLocal" else strideExt[0],
             "stride_src": strideExt[0] if direction == "ExternalToLocal" else strideLoc[0],
+            "future": future.name
         }
         return operatorRepresentation