[Liger-Kernel][LAYER NORM]Performance Discrepancy of Triton vs. Torch on PVC Compared to Nvidia

[Tarakarevu1]

Describe the issue

###Describe the issue
For [layer_norm] we collected benchmark data in

Triton
Torch
Torch.compile with three different modes: default, reduce-overhead, and max-autotune
Our findings indicate a notable performance discrepancy between Triton and Torch on different hardware platforms, specifically PVC and Nvidia H100 GPUs.

Let me know if you need more details.

Environment details

Triton: 3.2
GPU: Intel GPU Max 1550, H-100

[aregm]

@Tarakarevu1 - can you please be less generic and provide a reproducer?
@whitneywhtsang should be the same issue with the fp conversion in LayerNorm as we see in tutorials. Can you please confirm?

[Tarakarevu1]

This is the code file https://github.com/intel-innersource/applications.benchmarking.oneapi.ligerkernel/blob/main/benchmark/scripts/benchmark_layer_norm.py

[whitneywhtsang]

Functional regression: #3723

[whitneywhtsang]

This is the code file https://github.com/intel-innersource/applications.benchmarking.oneapi.ligerkernel/blob/main/benchmark/scripts/benchmark_layer_norm.py

@Tarakarevu1 is there any differences from the one in https://github.com/linkedin/Liger-Kernel?

[whitneywhtsang]

Latest performance result with public Liger-Kernel after fixing functional failure:

[Tarakarevu1]

There are notable differences between the benchmark_layer_norm.py script found in Intel's internal repository for Liger Kernel benchmarking and the offerings in LinkedIn's public Liger Kernel repository

[whitneywhtsang]

There are notable differences between the benchmark_layer_norm.py script found in Intel's internal repository for Liger Kernel benchmarking and the offerings in LinkedIn's public Liger Kernel repository

Right, Triton performance is better in the internal repo. What's the upstreaming plan to https://github.com/linkedin/Liger-Kernel? As we are now doing regular measurement using the public repo, it would be great to measure with the improved performance.

[vlad-penkin]

We'll continue to do measurements with the public repo.

Closing the ticket. @Tarakarevu1 please reopen the ticket when the changes will be upstreamed.