Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
5dc2f10
Auto-merge updates from master branch
invalid-email-address Feb 19, 2025
63f87e0
[Automated Commit] Format Codebase
github-actions[bot] Feb 19, 2025
a629034
Auto-merge updates from master branch
invalid-email-address Feb 20, 2025
bf8c1c2
Auto-merge updates from master branch
invalid-email-address Feb 21, 2025
ec284d2
Updated tags for submission checker command in docs
arjunsuresh Feb 24, 2025
5335553
Update mobilenets docs
arjunsuresh Feb 25, 2025
b9767aa
Update main.py
arjunsuresh Feb 25, 2025
f42aeeb
Update main.py
arjunsuresh Feb 25, 2025
d377848
Auto-merge updates from master branch
invalid-email-address Feb 25, 2025
c699ce3
update dataset download commands - waymo calib (#2130)
anandhu-eng Feb 25, 2025
c35459b
Auto-merge updates from master branch
invalid-email-address Feb 25, 2025
7c09a73
Merge branch 'master' into dev
arjunsuresh Feb 25, 2025
74ed21c
Merge branch 'master' into dev
arjunsuresh Feb 25, 2025
0eee795
Merge branch 'master' into dev
nathanwasson Feb 25, 2025
6a90ae4
Auto-merge updates from master branch
invalid-email-address Feb 27, 2025
7bf2c5e
Merge from Master (#2155)
arjunsuresh Mar 13, 2025
2a73202
[Automated Commit] Format Codebase
github-actions[bot] Mar 13, 2025
7ad9af1
Auto-merge updates from master branch
invalid-email-address Apr 15, 2025
2fb1057
Inference docs - Update model and dataset download commands (#2153)
anandhu-eng Apr 22, 2025
d804837
add powershell command to get result folder structure (#2156)
anandhu-eng Apr 22, 2025
f09fb9d
Auto-merge updates from master branch
invalid-email-address Apr 22, 2025
046b57d
Auto-merge updates from master branch
invalid-email-address Apr 22, 2025
5b65f7b
[Automated Commit] Format Codebase
github-actions[bot] Apr 22, 2025
ee16c1a
Merge branch 'master' into dev
arjunsuresh Jun 10, 2025
70fcbe0
[Automated Commit] Format Codebase
github-actions[bot] Jun 10, 2025
a5f12c7
Auto-merge updates from master branch
invalid-email-address Jun 10, 2025
3868556
Auto-merge updates from master branch
invalid-email-address Jun 10, 2025
18d49b4
Merge branch 'master' into dev
arjunsuresh Jun 10, 2025
a91453b
Auto-merge updates from master branch
invalid-email-address Jun 11, 2025
d288662
Auto-merge updates from master branch
invalid-email-address Jun 11, 2025
88324f3
Auto-merge updates from master branch
invalid-email-address Jun 16, 2025
ac42a2f
[Automated Commit] Format Codebase
github-actions[bot] Jun 16, 2025
53ae823
Auto-merge updates from master branch
invalid-email-address Jun 16, 2025
2b371c9
[Automated Commit] Format Codebase
github-actions[bot] Jun 16, 2025
9d5abe7
Auto-merge updates from master branch
invalid-email-address Jun 16, 2025
b896d58
Auto-merge updates from master branch
invalid-email-address Jun 17, 2025
92d37ac
Auto-merge updates from master branch
invalid-email-address Jun 19, 2025
72d52bb
[Automated Commit] Format Codebase
github-actions[bot] Jun 19, 2025
3af0227
Auto-merge updates from master branch
invalid-email-address Jun 23, 2025
c8ed997
Auto-merge updates from master branch
invalid-email-address Jun 24, 2025
fa3987f
Auto-merge updates from master branch
invalid-email-address Jun 24, 2025
dbe9632
Auto-merge updates from master branch
invalid-email-address Jun 24, 2025
ace438c
Auto-merge updates from master branch
invalid-email-address Jun 24, 2025
f841519
Fix Typo in Interactive Latencies (#2147) (#2225)
arjunsuresh Jun 26, 2025
4e77177
Fix Typo in Interactive Latencies (#2147) (#2226)
arjunsuresh Jun 26, 2025
56f8acf
Merge pull request #2227 from mlcommons/master
arjunsuresh Jun 26, 2025
868dd2b
Auto-merge updates from master branch
invalid-email-address Jul 2, 2025
350032a
Update MLCFlow commands for v5.1 (#2237)
anandhu-eng Jul 5, 2025
3a8595a
[Automated Commit] Format Codebase
github-actions[bot] Jul 5, 2025
184f0e8
Auto-merge updates from master branch
invalid-email-address Jul 7, 2025
381d239
Auto-merge updates from master branch
invalid-email-address Jul 8, 2025
00bb55f
Auto-merge updates from master branch
invalid-email-address Jul 8, 2025
a79004e
Auto-merge updates from master branch
invalid-email-address Jul 8, 2025
ed94d7a
Auto-merge updates from master branch
invalid-email-address Jul 8, 2025
1f4d76a
Auto-merge updates from master branch
invalid-email-address Jul 8, 2025
4e178b6
Auto-merge updates from master branch
invalid-email-address Jul 8, 2025
7c4f919
Auto-merge updates from master branch
invalid-email-address Jul 8, 2025
f77d066
Auto-merge updates from master branch
invalid-email-address Jul 15, 2025
906c0fc
[Automated Commit] Format Codebase
github-actions[bot] Jul 15, 2025
111063c
Auto-merge updates from master branch
invalid-email-address Jul 15, 2025
1d252a9
Auto-merge updates from master branch
invalid-email-address Jul 16, 2025
8980b1a
Auto-merge updates from master branch
invalid-email-address Jul 16, 2025
024e4ad
Update main.py
anandhu-eng Jul 17, 2025
9e2c7a9
[Automated Commit] Format Codebase
github-actions[bot] Jul 17, 2025
0928e11
updating for 5.1-dev (inference doc)
anandhu-eng Jul 17, 2025
7069a9e
[Automated Commit] Format Codebase
github-actions[bot] Jul 17, 2025
253854f
fix typo
anandhu-eng Jul 17, 2025
2f64e8b
Auto-merge updates from master branch
invalid-email-address Jul 18, 2025
7710c4d
Auto-merge updates from master branch
invalid-email-address Jul 21, 2025
371d583
[Automated Commit] Format Codebase
github-actions[bot] Jul 21, 2025
49db182
Auto-merge updates from master branch
invalid-email-address Jul 22, 2025
c8eb259
Auto-merge updates from master branch
invalid-email-address Jul 22, 2025
8ded855
Auto-merge updates from master branch
invalid-email-address Jul 22, 2025
ea86fc0
Update main.py
anandhu-eng Jul 23, 2025
2a787a2
Auto-merge updates from master branch
invalid-email-address Jul 23, 2025
8a44674
Auto-merge updates from master branch
invalid-email-address Jul 23, 2025
448a17d
Auto-merge updates from master branch
invalid-email-address Jul 24, 2025
6210a1a
Auto-merge updates from master branch
invalid-email-address Jul 24, 2025
6907b2e
Auto-merge updates from master branch
invalid-email-address Jul 24, 2025
0828e9c
[Automated Commit] Format Codebase
github-actions[bot] Jul 24, 2025
ffdf918
Auto-merge updates from master branch
invalid-email-address Jul 24, 2025
2861d7a
Auto-merge updates from master branch
invalid-email-address Jul 25, 2025
545d349
Auto-merge updates from master branch
invalid-email-address Jul 29, 2025
f8c344f
[Automated Commit] Format Codebase
github-actions[bot] Jul 29, 2025
b15eeb2
Auto-merge updates from master branch
invalid-email-address Jul 29, 2025
7cde799
Auto-merge updates from master branch
invalid-email-address Jul 29, 2025
6c8a1f7
Auto-merge updates from master branch
invalid-email-address Jul 29, 2025
74be5d1
Auto-merge updates from master branch
invalid-email-address Jul 29, 2025
0b14b18
Auto-merge updates from master branch
invalid-email-address Jul 30, 2025
ffd8bf2
Auto-merge updates from master branch
invalid-email-address Jul 30, 2025
d0a2ed4
Doc updates (#2292)
anandhu-eng Jul 30, 2025
23bd062
Add quiet flags to MLC commands (#2309)
anandhu-eng Jul 31, 2025
6ec4929
Improve docs - submission generation (#2311)
anandhu-eng Jul 31, 2025
58345d3
Auto-merge updates from master branch
invalid-email-address Jul 31, 2025
ff856b8
[Automated Commit] Format Codebase
github-actions[bot] Jul 31, 2025
20fafca
Auto-merge updates from master branch
invalid-email-address Aug 1, 2025
2921a33
Auto-merge updates from master branch
invalid-email-address Aug 4, 2025
9320cd2
Auto-merge updates from master branch
invalid-email-address Aug 6, 2025
9e338a2
Auto-merge updates from master branch
invalid-email-address Aug 6, 2025
f747083
Auto-merge updates from master branch
invalid-email-address Aug 7, 2025
2099779
Auto-merge updates from master branch
invalid-email-address Aug 20, 2025
b5a7676
Auto-merge updates from master branch
invalid-email-address Aug 20, 2025
32d50ae
Auto-merge updates from master branch
invalid-email-address Sep 1, 2025
a3d9975
Auto-merge updates from master branch
invalid-email-address Sep 10, 2025
1370cdc
Auto-merge updates from master branch
invalid-email-address Sep 10, 2025
ab9fee2
Auto-merge updates from master branch
invalid-email-address Sep 11, 2025
d542e58
Auto-merge updates from master branch
invalid-email-address Sep 14, 2025
80dd511
Auto-merge updates from master branch
invalid-email-address Sep 16, 2025
aecee3d
Auto-merge updates from master branch
invalid-email-address Sep 18, 2025
db0d97b
Auto-merge updates from master branch
invalid-email-address Sep 18, 2025
7962e4e
Merge branch 'master' into dev
arjunsuresh Nov 4, 2025
bf1469e
[Automated Commit] Format Codebase
github-actions[bot] Nov 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/benchmarks/language/get-deepseek-r1-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ The benchmark implementation run command will automatically download the validat
mlcr get,preprocessed,dataset,deepseek-r1,_validation,_mlc,_r2-downloader --outdirname=<path to download> -j
```

### Get Calibration Dataset
```
mlcr get,preprocessed,dataset,deepseek-r1,_calibration,_mlc,_rclone --outdirname=<path to download> -j
```
### Get Calibration Dataset
```
mlcr get,preprocessed,dataset,deepseek-r1,_calibration,_mlc,_r2-downloader --outdirname=<path to download> -j
```

## Model
The benchmark implementation run command will automatically download the required model and do the necessary conversions. In case you want to only download the official model, you can use the below commands.
Expand Down
11 changes: 11 additions & 0 deletions docs/benchmarks/language/get-llama2-70b-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,17 @@ The benchmark implementation run command will automatically download the require
mlcr get,ml-model,llama2-70b,_pytorch,_r2-downloader,_70b,_mlc -j
```

### Get the Official MLPerf LLAMA2-70B model from MLCOMMONS Google Drive

=== "From MLCOMMONS Cloudfare R2"

> **Note:** One has to accept the [MLCommons Llama 2 License Confidentiality Notice](https://llama2.mlcommons.org/) to access the model files in MLCOMMONS Google Drive.

### Get the Official MLPerf LLAMA2-70B model from MLCOMMONS Cloudfare R2

```
mlcr get,ml-model,llama2-70b,_mlc,_r2-downloader,_70b -j
```
=== "From Hugging Face repo"

> **Note:** Access to the HuggingFace model could be requested [here](https://ai.meta.com/resources/models-and-libraries/llama-downloads/).
Expand Down
4 changes: 3 additions & 1 deletion docs/benchmarks/language/get-llama3_1-8b-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ The benchmark implementation run command will automatically download the validat

### Get Calibration Dataset
```

mlcr get,dataset,cnndm,_calibration,_llama3,_mlc,_r2-downloader --outdirname=<path to download> -j
```

Expand All @@ -40,12 +41,13 @@ The benchmark implementation run command will automatically download the require

=== "Pytorch"

=== "From MLCOMMONS Storage"
=== "From Cloudfare R2"

> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Storage.

### Get the Official MLPerf LLAMA3.1-8B model from MLCOMMONS Cloudfare R2
```

mlcr get,ml-model,llama3,_mlc,_8b,_r2-downloader --outdirname=<path to download> -j
```

Expand Down
4 changes: 2 additions & 2 deletions speech2text/accuracy_eval.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,12 @@
"x",
"y",
"z",
"'",
"'",
"0",
"1",
"2",
"3",
"4",
"4",
"5",
"6",
"7",
Expand Down
4 changes: 2 additions & 2 deletions speech2text/reference_SUT.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,12 +90,12 @@ def get_start_cores(start_cores="0"):
"x",
"y",
"z",
"'",
"'",
"0",
"1",
"2",
"3",
"4",
"4",
"5",
"6",
"7",
Expand Down