@@ -8,31 +8,68 @@ Caliper: A Performance Analysis Toolbox in a Library
8
8
Caliper is a performance instrumentation and profiling library for HPC
9
9
(high-performance computing) programs. It provides source-code annotation
10
10
APIs for marking regions of interest in C, C++, Fortran, and Python codes,
11
- as well as performance measurement functionality for a wide range of use cases ,
12
- such as runtime profiling, event tracing, and performance monitoring.
11
+ as well as measurement functionality for a wide range of runtime profiling ,
12
+ event tracing, and performance monitoring use cases .
13
13
14
- Caliper can generate simple human-readable reports or machine-readable
15
- JSON or .cali files for automated data processing with user-provided scripts
16
- or analysis frameworks like [ Hatchet] ( https://github.com/LLNL/hatchet )
14
+ Caliper can generate simple human-readable reports or files for
15
+ performance data analysis frameworks like
16
+ [ Hatchet] ( https://github.com/LLNL/hatchet )
17
17
and [ Thicket] ( https://github.com/LLNL/thicket ) .
18
18
It can also generate detailed event traces for timeline visualizations with
19
19
[ Perfetto] ( https://perfetto.dev ) and the Google Chrome trace viewer.
20
20
21
21
Features include:
22
22
23
23
* Low-overhead source-code annotation API
24
- * Configuration API to control performance measurements from
25
- within an application
26
24
* Recording program metadata for analyzing collections of runs
27
- * Flexible key: value data model to capture application-specific
28
- features for performance analysis
29
- * Fully threadsafe implementation, support for parallel programming
30
- models like MPI, OpenMP, Kokkos, CUDA, and ROCm
25
+ * Fully threadsafe implementation
26
+ * Support for parallel programming models like MPI, OpenMP, Kokkos, CUDA, and ROCm
31
27
* Event-based and sample-based performance measurements
32
28
* Trace and profile recording
33
29
* Connection to third-party tools, e.g. NVidia's NSight tools, AMD
34
30
ROCProf, or Intel(R) VTune(tm)
35
31
32
+ Overview
33
+ ------------------------------------------
34
+
35
+ Caliper is primarily a source-code instrumentation library. To use it, insert
36
+ Caliper instrumentation markers around source-code regions of interest in the
37
+ target program:
38
+
39
+ ``` C++
40
+ #include < caliper/cali.h>
41
+
42
+ int get_answer () {
43
+ CALI_CXX_MARK_FUNCTION;
44
+
45
+ CALI_MARK_BEGIN ("computing");
46
+ int ret = 2 * 3 * 7;
47
+ CALI_MARK_END("computing");
48
+ return ret;
49
+ }
50
+ ```
51
+
52
+ There are annotation APIs for C, C++, Fortran, and Python codes.
53
+ To take performance measurements, Caliper provides built-in profiling recipes for
54
+ a wide range of performance engineering use cases. Available functionality includes
55
+ MPI function and message profiling, CUDA and HIP API as well as GPU activity
56
+ (kernel executions and memory copies) profiling, call-path sampling, and much more.
57
+ As a simple example, the ` runtime-report ` recipe prints the time spent in the
58
+ annotated regions on screen:
59
+
60
+ $ CALI_CONFIG=runtime-report ./answer
61
+ Path Time (E) Time (I) Time % (E) Time % (I)
62
+ main 0.000072 0.000083 17.469875 20.188570
63
+ get_answer 0.000008 0.000011 1.864844 2.718695
64
+ computing 0.000004 0.000004 0.853851 0.853851
65
+
66
+ Aside from simple text reports, Caliper can generate machine-readable output in JSON
67
+ or its own custom .cali file format, which can be analyzed with the Caliper-provided
68
+ ` cali-query ` tool and CalQL query language, or imported into Python analysis
69
+ scripts with the [ caliper-reader] ( python/caliper-reader/ ) Python module.
70
+ In addition, Caliper can collect data for [ Thicket] ( https://github.com/LLNL/thicket ) ,
71
+ a Python-based toolkit for Exploratory Data Analysis of parallel performance data.
72
+
36
73
Documentation
37
74
------------------------------------------
38
75
@@ -42,12 +79,13 @@ https://software.llnl.gov/Caliper/
42
79
Usage examples of the C++, C, and Fortran annotation and ConfigManager
43
80
APIs are provided in the [ examples] ( examples/apps ) directory.
44
81
45
- See the "Getting started" section below for a brief tutorial.
82
+ A basic tutorial is available here:
83
+ https://github.com/daboehme/caliper-tutorial
46
84
47
85
Building and installing
48
86
------------------------------------------
49
87
50
- You can install Caliper with the [ spack] ( https://github.com/spack/spack )
88
+ Caliper can be installed with the [ spack] ( https://github.com/spack/spack )
51
89
package manager:
52
90
53
91
$ spack install caliper
@@ -71,96 +109,6 @@ for MPI support.
71
109
See the "Build and install" section in the documentation for further
72
110
information.
73
111
74
- Getting started
75
- ------------------------------------------
76
-
77
- Typically, we integrate Caliper into a program by marking source-code
78
- sections of interest with descriptive annotations. Performance profiling can
79
- then be enabled through the Caliper ConfigManager API or environment
80
- variables. Alternatively, third-party tools can connect to Caliper and access
81
- information provided by the source-code annotations.
82
-
83
- ### Source-code annotations
84
-
85
- Caliper's source-code annotation API allows you to mark source-code regions
86
- of interest in your program. Much of Caliper's functionality depends on these
87
- region annotations.
88
-
89
- Caliper provides macros and functions for C, C++, and Fortran to mark
90
- functions, loops, or sections of source-code. For example, use
91
- ` CALI_CXX_MARK_FUNCTION ` to mark a function in C++:
92
-
93
- ``` C++
94
- #include < caliper/cali.h>
95
-
96
- void foo ()
97
- {
98
- CALI_CXX_MARK_FUNCTION;
99
- // ...
100
- }
101
- ```
102
-
103
- You can mark arbitrary code regions with the ` CALI_MARK_BEGIN ` and
104
- ` CALI_MARK_END ` macros or the corresponding ` cali_begin_region() `
105
- and ` cali_end_region() ` functions:
106
-
107
- ``` C++
108
- #include < caliper/cali.h>
109
-
110
- // ...
111
- CALI_MARK_BEGIN ("my region");
112
- // ...
113
- CALI_MARK_END("my region");
114
- ```
115
-
116
- The [cxx-example](examples/apps/cxx-example.cpp),
117
- [c-example](examples/apps/c-example.c), and
118
- [fortran-example](examples/apps/fortran-example.f) example apps show how to use
119
- Caliper in C++, C, and Fortran, respectively.
120
-
121
- ### Recording performance data
122
-
123
- With the source-code annotations in place, we can run performance measurements.
124
- By default, Caliper does not record data - we have to activate performance
125
- profiling at runtime.
126
- An easy way to do this is to use one of Caliper's built-in measurement
127
- recipes. For example, the `runtime-report` config prints out the time
128
- spent in the annotated regions. You can activate built-in measurement
129
- configurations with the ConfigManager API or with the `CALI_CONFIG`
130
- environment variable. Let's try this on Caliper's cxx-example program:
131
-
132
- $ cd Caliper/build
133
- $ make cxx-example
134
- $ CALI_CONFIG=runtime-report ./examples/apps/cxx-example
135
- Path Min time/rank Max time/rank Avg time/rank Time %
136
- main 0.000119 0.000119 0.000119 7.079120
137
- mainloop 0.000067 0.000067 0.000067 3.985723
138
- foo 0.000646 0.000646 0.000646 38.429506
139
- init 0.000017 0.000017 0.000017 1.011303
140
-
141
- The runtime-report config works for MPI and non-MPI programs. It reports the
142
- minimum, maximum, and average exclusive time (seconds) spent in each marked
143
- code region across MPI ranks (the values are identical in non-MPI programs).
144
-
145
- You can customize the report with additional options. Some options enable
146
- additional Caliper functionality, such as profiling MPI and CUDA functions in
147
- addition to the user-defined regions, or additional metrics like memory usage.
148
- Other measurement configurations besides runtime-report include:
149
-
150
- * loop-report: Print summary and time-series information for loops.
151
- * mpi-report: Print time spent in MPI functions.
152
- * sample-report: Print time spent in functions using sampling.
153
- * event-trace: Record a trace of region enter/exit events in .cali format.
154
- * hatchet-region-profile: Record a region time profile for processing with
155
- [Hatchet](https://github.com/LLNL/hatchet) or cali-query.
156
-
157
- See the "Builtin configurations" section in the documentation to learn more
158
- about different configurations and their options.
159
-
160
- You can also create entirely custom measurement configurations by selecting and
161
- configuring Caliper services manually. See the "Manual configuration" section
162
- in the documentation to learn more.
163
-
164
112
Authors
165
113
------------------------------------------
166
114
0 commit comments