Building and Installing HPCToolkit

Contents

• Prerequisites
• Downloading Source Code
• Building and Installing
• Infrequently Used Build Options

Prerequisites

[Contents]

To reduce our maintenance efforts, we have opted to distribute HPCToolkit primarily in source form. The following prerequisites apply to our default source code distribution.

1. HPCToolkit is supported on the following platforms:

   Supported Platforms	Notes
   Linux + x86_64 (64-bit)
   Linux + KNL
   Linux + power7 (big endian)
   Linux + power8, 9 (little endian)
   Cray XT/XE/XK/XC (CNL + x86_64)	Compute Node Linux ≥ 2.1
   IBM Blue Gene/Q (BG/Q kernel + ppc64)	Use branch 'gcc-4.4' of externals
   ARM aarch64 (64-bit)

2. Compiler: HPCToolkit requires GNU GCC minimum version 4.8, but preferably 5.x or later. Some vendor compilers that support GNU extensions may also work, but this is not supported. For old systems with GCC 4.4 (eg, Blue Gene/Q and Red Hat 6.x), if a newer compiler is not available, then use branch 'gcc-4.4' for hpctoolkit-externals.

3. Other build tools: HPCToolkit also requires GNU make 3.81 or later and cmake 2.8.12 or later.

4. To use hpcviewer and hpctraceviewer (HPCToolkit's graphical interface) on MacOS, Windows, or Linux, a Java runtime environment (JRE) ≥ 1.7 is required. Use the Oracle or IBM version; we have had problems with OpenJDK.

   To use hpcviewer and hpctraceviewer on Linux, GTK+ is also required.

   Note: Since HPCToolkit's databases are portable, we often install hpcviewer and hpctraceviewer on local laptops/desktops to avoid sending display traffic over relatively slow network connections.

5. Highly recommended: To collect measurements based on hardware performance monitoring unit (PMU) information, PAPI ≥ 4.1 is required. Without PAPI, HPCToolkit can only collect measurements based on an operating system timer. (Exception: On IBM Blue Gene/P, PAPI is not required. On this system, HPCToolkit uses IBM's native Universal Performance Counters directly.)

   Use a kernel with perf_events (see also: this), which was introduced with kernel 2.6.32 (née PCL in kernel 2.6.31) and which is now becoming part of standard distributions. With perf_events, use PAPI ≥ 4.1.2.

6. Minor requirement: To use hpcsummary (a relatively minor tool), Python 2.x is required.

7. Minor requirement: To build HPCToolkit's man pages, a variant of Perl 5 is required. We have successfully used versions 5.6.x - 5.10.x.

Downloading Source Code

[Contents]

Download HPCToolkit components as follows:

1. Download hpctoolkit from GitHub.

   git clone https://github.com/HPCToolkit/hpctoolkit.git

2. Download hpctoolkit-externals from GitHub.

   git clone https://github.com/HPCToolkit/hpctoolkit-externals.git

   Think of hpctoolkit-externals as a "mini package manager" that configures and builds prerequisite libraries for hpctoolkit.

3. Download binary releases of hpcviewer and hpctraceviewer for the platform on which your are building HPCToolkit.

   http://hpctoolkit.org/download/hpcviewer

   Note: It is perfectly reasonable to install hpcviewer and hpctraceviewer in multiple locations. Since HPCToolkit's databases are portable, we often install hpcviewer and hpctraceviewer on local laptops/desktops to avoid sending display traffic over relatively slow network connections from systems where we collect measurement data.

Building and Installing

[Contents]

The build and install follows autotools conventions (./configure && make && make install). In each case, more options are available by executing ./configure --help. See the README file for more details.

The typical build is as follows. Please note that a list of examples below provides configure commands for selected platforms.

1. Configuring the perf_events sample source:

   HPCToolkit's configure command will automatically check the availability of the Linux perf_events interface on the build system. The test includes checking the existence of /usr/include/linux/perf_event.h and /proc/sys/kernel/perf_event_paranoid files. If the two files do not exist, it assumes the system does not support the perf_events interface.

   To profile on a remote machine that has different system or architecture configuration, one must download and build the libpfm4 library (http://perfmon2.sourceforge.net) and indicate its install directory by specifying the option --with-perfmon=<libpfm4_install_directory> to HPCToolkit's configure command.

2. Build and install HPCToolkit's Externals, which typically does not require any special options; we recommend using the GCC compilers:

   cd hpctoolkit-externals
mkdir BUILD && cd BUILD
../configure [CC=<c-compiler>] [CXX=<c++-compiler>] \
     [--prefix=<hpctoolkit-externals-install>]
   make install
make clean
   

   Externals is not persistent as HPCToolkit will copy everything it needs from the installation. Thus, the installation path <hpctoolkit-externals-install> should be a local path and not something like /usr. If --prefix is not supplied it will automatically be set to ./<Autoconf-host>.

   To specify a C and C++ compiler other than gcc and g++, use CC and CXX.

   Please be sure the Externals build completes before configuring HPCToolkit. (Note: After building Externals once, HPCToolkit can be configured and built several times using the same Externals installation.)

3. Build and install HPCToolkit using commands similar to the following; we recommend using the GCC compilers:

   cd hpctoolkit
mkdir BUILD && cd BUILD
../configure [CC=<c-compiler>] [CXX=<c++-compiler>] [MPICXX=<mpi-c++-compiler>] \
     --prefix=<hpctoolkit-install> \
     --with-externals=<hpctoolkit-externals-install> \
     [--with-papi=<papi-install>]
make install
   

   The above commands will build HPCToolkit and install it into <hpctoolkit-install>.

   To specify a C and C++ compiler other than gcc and g++, use CC and CXX. To enable MPI support, (1) ensure a working mpicxx or mpiCC is in your shell's path; or (2) use MPICXX. (Naturally, CXX and MPICXX should be compatible; typically MPICXX is a wrapper for CXX.) To enable PAPI-based measurements, use the --with-papi option; it takes as an argument the path of your PAPI installation, <papi-install>.

   Because the HPCToolkit installation is self-contained, it may be renamed or moved to another location. All HPCToolkit Externals source code, build files and installations may be removed; all HPCToolkit source code and build files may be removed.

4. Install hpcviewer and hpctraceviewer .

   • To install hpcviewer or hpctraceviewer as a stand-alone package, simply unpack and move the directory tree to the desired location. (This is the only option on Windows or MacOS.)
   • To install hpcviewer or hpctraceviewer in an HPCToolkit installation, unpack the tarball and execute the following commands:
     cd <hpcviewer> && ./install [ -j <path-to-java> ] <hpctoolkit-install>
cd <hpctraceviewer> && ./install [ -j <path-to-java> ] <hpctoolkit-install>
     Here, <hpcviewer> and <hpctraceviewer> are the directories created by unpacking the respective tarballs; and <hpctoolkit-install> is the above HPCToolkit installation directory.
     The optional flag -j <path-to-java> allows the specification of an appropriate java to be used with hpcviewer or hpctraceviewer.

Selected Examples

The list below gives example HPCToolkit Externals and HPCToolkit configure commands (respectively) for selected platforms.

N.B.: These commands are designed to replace the respective commands in the above build instructions.

• Blue Gene/Q:

  Load the '+mpiwrapper-gcc' softenv package (for the MPICXX compiler) and check that the back-end powerpc64-bgq-linux compilers are on your PATH. Use the '--enable-bgq' option for HPCToolkit externals to build for the Blue Gene/Q compute nodes.

  soft add +mpiwrapper-gcc
PATH="/bgsys/drivers/ppcfloor/gnu-linux/bin/:$PATH"

  HPCToolkit's Externals. The front end of Blue Gene/Q is normally Red Hat 6.x with gcc 4.4 which won't build the current externals. If a more recent compiler is not available, then use branch 'gcc-4.4' of externals.

  ../configure --prefix=`pwd`/../powerpc64-linux \
     --enable-bgq

  HPCToolkit

  ../configure \
     MPICXX=/bgsys/drivers/ppcfloor/comm/gcc/bin/mpicxx \
     HPCPROFMPI_LT_LDFLAGS="-all-static" \
     --prefix=<install-path> \
     --with-externals=<hpctoolkit-externals>/powerpc64-linux \
     --with-papi=/soft/perftools/papi

  Normally on Blue Gene, you want to build HPCToolkit to run on the compute nodes. Configure will auto-detect this and default to the back-end nodes (but issue a warning). The '--enable-bgq' flag adds options specific to the Blue Gene/Q compute nodes and suppresses the warning. If you want to build hpcrun to run on the front-end nodes, then use the option '--disable-bgq'.

  Note: The paths for MPICXX and PAPI may be different on your system.

• Cray XT/XE/XK/XC:

  N.B.: Use the GNU programming environment, e.g.:

  module swap PrgEnv-pgi PrgEnv-gnu

  HPCToolkit's Externals

  ../configure --prefix=`pwd`/../x86_64-linux


  HPCToolkit

  ../configure \
     MPICXX="CC" \
     HPCPROFMPI_LT_LDFLAGS="-all-static" \
     --prefix=<hpctoolkit-install> \
     --with-externals=<hpctoolkit-externals>/x86_64-linux \
     --with-papi=/opt/cray/papi/4.3.0.1/perf_events/no-cuda

   

  Note: On systems with AMD CPUs with the new XOP instructions, add the option '--enable-xop'. The AMD Interlagos CPUs (6200 series) use these instructions. Currently, Jaguar at ORNL uses these CPUs, but Hopper at NERSC does not.

• Old systems with GCC 4.4:

  The current versions of Dyninst and hpctoolkit-externals require GCC 4.8 or later (and preferably 5.x or later). As a transition for old systems with GCC 4.4 (eg, Blue Gene/Q and Red Hat 6.x), if a newer compiler is not available, then use branch 'gcc-4.4' of externals. This contains a snapshot of the last version of externals that will build with GCC 4.4.

  git clone https://github.com/HPCToolkit/hpctoolkit-externals.git
cd hpctoolkit-externals
git checkout gcc-4.4

   

  Then, configure and build as usual.

• Blue Gene/P: no longer supported.

• Intel MIC (k1om): no longer supported.

• Linux + i386 (32-bit): no longer supported.

Infrequently Used Build Options

[Contents]

The following options may be useful in exceptional circumstances.

• To enable/disable HPCToolkit features, the following options may be used.

  Option	Meaning
  --enable-mpi	enable/disable build of hpcprof-mpi [default: yes]
  --with-upc=PATH	path to IBM BlueGene/P UPC installation
  --with-upc-lib=LIBRARY	path to IBM BlueGene/P UPC library(s), absolute or relative to WITH_UPC/runtime/SPI
  --enable-develop	enable/disable build for development (sane debugging) [default: no]

• To request that HPCToolkit Externals use an alternate user-supplied installation for a given package, the following options may be used.

  Option	Meaning
  --with-binutils=PATH	path to binutils install directory
  --with-libdwarf=PATH	path to libdwarf install directory
  --with-libelf=PATH	path to libelf install directory
  --with-libmonitor=PATH	path to libmonitor install directory
  --with-libunwind=PATH	path to libunwind install directory
  --with-libxml2=PATH	path to libxml2 install directory
  --with-old-monitor=PATH	path to old monitor install directory
  --with-open-analysis=PATH	path to open analysis install directory
  --with-symtabAPI=PATH	path to symtabAPI install directory
  --with-xed2=PATH	path to xed2 install directory
  --with-xerces=PATH	path to xerces/c install directory

  In all cases, the default is for HPCToolkit externals build the package itself, depending on platform.

• To explicitly request that a package x not be built, use --without-x.

[Page last updated: 2018/10/22]

---

©2000-2018 Rice University • Rice Computer Science