The HPCToolkit Performance Tools


Version 2017.11

HPCToolkit is a collection of performance analysis tools for node-based performance analysis. It has been designed around the following principles:

HPCToolkit's website (hpctoolkit.org) contains papers that explain these design principles in more detail.

Table of Contents


Top-Down Analysis With Calling Context and Program Structure

A typical performance analysis session consists of:

  1. Measurement. Collect low-overhead, high-accuracy profiles using statistical sampling. hpcrun(1) collects call path profiles while hpcrun-flat(1) collects `flat' profiles (IP histograms, where IP is instruction pointer). hpclink(1) is used to collect call path profiles for statically linked applications.

  2. Recovering static source code structure. hpcstruct(1) recovers static program structure such as procedures and loop nests. It accounts for procedure and loop transformations such as inlining and software pipelining. Technically, this is an optional step, but critical information is lost without it.

  3. Correlating dynamic profiles with static source code structure. HPCToolkit's correlation tool combines dynamic profile information with hpcstruct(1) 's static program structure to correlate costs of the optimized object code to useful source code constructs such as loop nests. The result of correlation is called an Experiment database. Currently, hpcprof(1) is used for call stack profiles and hpcprof-flat(1) is used for flat profiles.

  4. Top-down visualization hpcviewer(1) is a Rich Client Platform-based tool for presenting the resulting Experiment databases. An important feature of the Experiment database is that it is relocatable, containing profile information and copies of application source files. This means that the first three steps can be performed remotely on a cluster and then the database can be viewed locally on a laptop or workstation.

Textual Analysis

  1. Measurement. Collect low-overhead, high-accuracy `flat' profiles using statistical sampling (hpcrun-flat(1) ).

  2. Recovering static source code structure. hpcstruct(1) is used for the same purpose described above.

  3. Correlating dynamic profiles with procedures source lines. hpcproftt(1) correlates `flat' profiles with source structure and produces textual output.


Interactively visualize call path or flat profiles correlated with program structure

Assume we have an application called zoo whose source code is located in in path-to-zoo.

  1. Compile. First compile and link the application normally with full optimization and as much debugging information as possible. Typically, this involves compiler options similar to -O3 -g. (See hpcstruct(1) for options for specific compilers.)

  2. Measure. Profile with hpcrun(1) or hpcrun-flat(1) . Assume we wish to use two different sets of events.
      hpcrun[-flat] <event-set-1> zoo
      hpcrun[-flat] <event-set-2> zoo
    hpcrun(1) will by default place the results in a measurement directory named hpctoolkit--measurements. hpcrun-flat(1) by default creates data files in the current directory; assume the results are placed in profile-file-1 and profile-file-2.

  3. Recover static source code structure. Use hpcstruct(1) to recover program structure and write it to the file zoo.hpcstruct
      hpcstruct zoo

  4. Correlate call path or flat metrics with static source code structure. Create an Experiment database using hpcprof(1) or hpcprof-flat(1) . (The version of hpcprof(1) must match the version of hpcrun(1) .) Assume the generated Experiment database is named hpctoolkit-database.
      hpcprof -I path-to-zoo/'*' -S zoo.hpcstruct hpctoolkit-zoo-measurements
      hpcprof-flat -I path-to-zoo/'*' -S zoo.hpcstruct profile-file-1 profile-file-2

  5. Visualize. Visualize the Experiment database using hpcviewer(1) :
      hpcviewer hpctoolkit-database
    Derived metrics may be computed on-the-fly with hpcviewer(1) . See The hpcviewer User Interface Guide for more information.

Visualize textual summaries of flat profiles

Begin with steps 1--3 above.

See Also

hpcrun(1) , hpclink(1) (hpcrun-flat(1) )
hpcprof(1) , hpcprof-mpi(1) (hpcprof-flat(1) )
hpcviewer(1) , hpctraceviewer(1)


Version: 2017.11 of 2011/02/22.

License and Copyright

© 2002-2018, Rice University.
See README.License.


Email: hpctoolkit-forum =at= rice.edu
WWW: http://hpctoolkit.org.