|
1 |
| -# Running Benchmarks on Windows 10 |
2 |
| - |
3 |
| -The LNT testing framework is tied to Unix. However, it can be run on Windows 10 |
4 |
| -using the beta |
5 |
| -[Windows Subsystem for Linux](https://blogs.msdn.microsoft.com/wsl/2016/04/22/windows-subsystem-for-linux-overview/). |
6 |
| -This provides a way to run LLVM test suite benchmarks on Windows hardware without a |
7 |
| -virtual machine affecting timing results. |
8 |
| - |
9 |
| -## Set up source code |
10 |
| -- Set up the Checked C LLVM and clang repos following these |
11 |
| -[directions](https://github.com/Microsoft/checkedc-clang/blob/master/docs/checkedc/Setup-and-Build.md). |
12 |
| -- Clone the Checked C LLVM test suite repo and the LLVM LNT repos: |
13 |
| -``` |
14 |
| -git clone -c core.autocrlf=false https://github.com/Microsoft/checkedc-llvm-test-suite.git |
15 |
| -git clone -c core.autocrlf=false https://github.com/llvm-mirror/lnt.git |
16 |
| -``` |
17 |
| - |
18 |
| -## Install the Windows Subsystem for Linux and the Bash shell |
19 |
| -- Add the Windows subsystem for Linux following these [directions](https://msdn.microsoft.com/en-us/commandline/wsl/install_guide). |
20 |
| -You will have to reboot your computer as the last step. |
21 |
| -- After rebooting, install Bash. Start a Windows command shell prompt |
22 |
| -and type `bash`. This will prompt you to install Ubuntu on Windows. |
23 |
| -You will need to create a UNIX user account and password. |
24 |
| -- Install additional packages needed by LNT: |
25 |
| - |
26 |
| -``` |
27 |
| -sudo apt-get install python-virtualenv |
28 |
| -sudo apt-get install python-dev -y |
29 |
| -sudo apt-get install tcl |
30 |
| -sudo apt-get install bison |
31 |
| -``` |
32 |
| - |
33 |
| -Note that you can find your Windows files under `/mnt/`_{drive letter}_`/`. |
34 |
| - |
35 |
| -## Setup LNT |
36 |
| - |
37 |
| -These directions are based on those at |
38 |
| -[http://llvm.org/docs/lnt/quickstart.html](http://llvm.org/docs/lnt/quickstart.html). |
39 |
| -In a bash shell: |
40 |
| - |
41 |
| -- Set up a Python sandbox using `virtualenv` |
42 |
| -``` |
43 |
| -virtualenv ~/mysandbox |
44 |
| -``` |
45 |
| - |
46 |
| -- Change directory to your LNT repo and run: |
47 |
| -``` |
48 |
| -~/mysandbox/bin/python setup.py install. |
49 |
| -``` |
50 |
| -There will be many error messages. |
51 |
| - |
52 |
| -## (Temporary) Build a Linux version of clang |
53 |
| -The Windows Subsystem for Linux added the ability to |
54 |
| -[launch Windows NT executables](https://msdn.microsoft.com/en-us/commandline/wsl/release_notes#build-14951) |
55 |
| -in [Windows build 14951](https://blogs.windows.com/windowsexperience/2016/10/19/announcing-windows-10-insider-preview-build-14951-for-mobile-and-pc), |
56 |
| -which was released in mid-October, 2016. This functionality |
57 |
| -has not yet made it into the release build of Windows 10. If you running Windows |
58 |
| -insider builds, you can skip this step. Otherwise, you need to build a Linux version of |
59 |
| -clang to run under the Windows Subsystem for Linux. You also need to install an updated |
60 |
| -version of cmake. |
61 |
| - |
62 |
| -To install cmake, you can go to the [cmake download page](https://cmake.org/download/). |
63 |
| -You will need to unpack cmake into a target directory and then invoke cmake |
64 |
| -from the target directory. |
65 |
| - |
66 |
| -LNT testing for LLVM is done using the `release+asserts` version of clang. To build it: |
67 |
| -- Create an LLVM build directory. |
68 |
| -- Change to the build directory and run |
69 |
| - |
70 |
| -``` |
71 |
| -cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On {path to your LLVM repo} |
72 |
| -make |
73 |
| -``` |
74 |
| -It may be possible to use a debug version of clang in LNT testing, although we have not tried it. |
75 |
| - |
76 |
| -## Run LNT |
77 |
| - |
78 |
| -In your Bash shell, run |
79 |
| - |
80 |
| -```~/mysandbox/bin/lnt runtest nt --sandbox {directory to build and store results in} |
81 |
| - --cc {path to clang} --test-suite {location of test suite} --cflags -fcheckedc-extension |
82 |
| -``` |
83 |
| - |
84 |
| -To narrow LNT to run on specific tests, use the `--only-test` option. You can use this to narrow |
85 |
| -the tests that are run to a specific directory: |
86 |
| -``` |
87 |
| ---only-test MultiSource |
88 |
| ---only-test MultiSource/Benchmarks/PtrDist |
89 |
| -``` |
90 |
| - |
91 |
| -If you just want to run tests quickly, use the `-j` option with the number of threads to use: |
92 |
| -``` |
93 |
| --j 8 |
94 |
| -``` |
95 |
| - |
96 |
| -The benchmark results will be output in CSV file and .json file. |
97 |
| - |
| 1 | +# Running Benchmarks on Windows 10 |
| 2 | + |
| 3 | +The LNT testing framework is tied to Unix. However, it can be run on Windows 10 |
| 4 | +using the beta |
| 5 | +[Windows Subsystem for Linux](https://blogs.msdn.microsoft.com/wsl/2016/04/22/windows-subsystem-for-linux-overview/). |
| 6 | +This provides a way to run LLVM test suite benchmarks on Windows hardware without a |
| 7 | +virtual machine affecting timing results. |
| 8 | + |
| 9 | +## Set up source code |
| 10 | +- Set up the Checked C LLVM and clang repos following these |
| 11 | +[directions](https://github.com/Microsoft/checkedc-clang/blob/master/docs/checkedc/Setup-and-Build.md). |
| 12 | +- Clone the Checked C LLVM test suite repo and the LLVM LNT repos: |
| 13 | +``` |
| 14 | +git clone -c core.autocrlf=false https://github.com/Microsoft/checkedc-llvm-test-suite.git |
| 15 | +git clone -c core.autocrlf=false https://github.com/llvm-mirror/lnt.git |
| 16 | +``` |
| 17 | + |
| 18 | +## Install the Windows Subsystem for Linux and the Bash shell |
| 19 | +- Add the Windows subsystem for Linux following these [directions](https://msdn.microsoft.com/en-us/commandline/wsl/install_guide). |
| 20 | +You will have to reboot your computer as the last step. |
| 21 | +- After rebooting, install Bash. Start a Windows command shell prompt |
| 22 | +and type `bash`. This will prompt you to install Ubuntu on Windows. |
| 23 | +You will need to create a UNIX user account and password. |
| 24 | +- Install additional packages needed by LNT: |
| 25 | + |
| 26 | +``` |
| 27 | +sudo apt-get install python-virtualenv |
| 28 | +sudo apt-get install python-dev -y |
| 29 | +sudo apt-get install tcl |
| 30 | +sudo apt-get install bison |
| 31 | +``` |
| 32 | + |
| 33 | +Note that you can find your Windows files under `/mnt/`_{drive letter}_`/`. |
| 34 | + |
| 35 | +## Setup LNT |
| 36 | + |
| 37 | +These directions are based on those at |
| 38 | +[http://llvm.org/docs/lnt/quickstart.html](http://llvm.org/docs/lnt/quickstart.html). |
| 39 | +In a bash shell: |
| 40 | + |
| 41 | +- Set up a Python sandbox using `virtualenv` |
| 42 | +``` |
| 43 | +virtualenv ~/mysandbox |
| 44 | +``` |
| 45 | + |
| 46 | +- Change directory to your LNT repo and run: |
| 47 | +``` |
| 48 | +~/mysandbox/bin/python setup.py install. |
| 49 | +``` |
| 50 | +There will be many error messages. |
| 51 | + |
| 52 | +## (Temporary) Build a Linux version of clang |
| 53 | +The Windows Subsystem for Linux added the ability to |
| 54 | +[launch Windows NT executables](https://msdn.microsoft.com/en-us/commandline/wsl/release_notes#build-14951) |
| 55 | +in [Windows build 14951](https://blogs.windows.com/windowsexperience/2016/10/19/announcing-windows-10-insider-preview-build-14951-for-mobile-and-pc), |
| 56 | +which was released in mid-October, 2016. This functionality |
| 57 | +has not yet made it into the release build of Windows 10. If you running Windows |
| 58 | +insider builds, you can skip this step. Otherwise, you need to build a Linux version of |
| 59 | +clang to run under the Windows Subsystem for Linux. You also need to install an updated |
| 60 | +version of cmake. |
| 61 | + |
| 62 | +To install cmake, you can go to the [cmake download page](https://cmake.org/download/). |
| 63 | +You will need to unpack cmake into a target directory. When you invoke cmake, you |
| 64 | +will need to use the full path name to the binary, including the path for the |
| 65 | +target directory. |
| 66 | + |
| 67 | +LNT testing for LLVM is done using the `release+asserts` version of clang. To build it: |
| 68 | +- Create an LLVM build directory. |
| 69 | +- Change to the build directory and run |
| 70 | + |
| 71 | +``` |
| 72 | +cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On {path to your LLVM repo} |
| 73 | +make |
| 74 | +``` |
| 75 | +It may be possible to use a debug version of clang in LNT testing, although we have not tried it. |
| 76 | + |
| 77 | +## Run LNT |
| 78 | + |
| 79 | +In your Bash shell, run |
| 80 | + |
| 81 | +```~/mysandbox/bin/lnt runtest nt --sandbox {directory to build and store results in} |
| 82 | + --cc {path to clang} --test-suite {location of test suite} --cflags -fcheckedc-extension |
| 83 | +``` |
| 84 | + |
| 85 | +To narrow LNT to run on specific tests, use the `--only-test` option. You can use this to narrow |
| 86 | +the tests that are run to a specific directory: |
| 87 | +``` |
| 88 | +--only-test MultiSource |
| 89 | +--only-test MultiSource/Benchmarks/PtrDist |
| 90 | +``` |
| 91 | + |
| 92 | +If you just want to run tests quickly, use the `-j` option with the number of threads to use: |
| 93 | +``` |
| 94 | +-j 8 |
| 95 | +``` |
| 96 | + |
| 97 | +The benchmark results will be output in CSV file and .json file. |
| 98 | + |
0 commit comments