Current Spack Issues for HPCToolkit

Spack is a moving target and receives multiple commits per day. Normally, HPCToolkit will build and run successfully with the latest version of all of its prerequisite packages, but sometimes not. This page covers the current known issues where HPCToolkit fails to build with the latest version of spack. The main build directions are at:

Report problems to hpctoolkit-forum at rice dot edu. But before reporting a problem, first try the versions recommended in the packages.yaml file in the spack subdirectory of the hpctoolkit repository. And always check the latest version of this file on the hpctoolkit web site.

http://hpctoolkit.org/spack-issues.html

2 Current Issues

2.1 (2019-08-28) Cray front-end compilers

Spack compiler find is currently broken for detecting the front-end compilers on Cray that HPCToolkit uses. Normally, you would load a module for gcc and run spack compiler find and spack would add that compiler to compilers.yaml, but this currently does not work.

Workaround: If you have a working compiler: entry for a front-end GNU compiler on Cray, then that will continue to work. If not, then you will have to add one manually. For example, this is an entry for the gcc/8.3.0 module on theta at ANL. Note that the front-end operating_system is something like sles12 (not cnl6), and the front-end target is x86_64 (not mic_knl). Your versions may differ.

- compiler:
    environment: {}
    flags: {}
    modules:
    - PrgEnv-gnu/6.0.5
    - gcc/8.3.0
    - craype/2.6.1
    - cray-mpich/7.7.10
    operating_system: sles12
    paths:
      cc:  /opt/gcc/8.3.0/bin/gcc
      cxx: /opt/gcc/8.3.0/bin/g++
      f77: /opt/gcc/8.3.0/bin/gfortran
      fc:  /opt/gcc/8.3.0/bin/gfortran
    spec: gcc@8.3.0
    target: x86_64

3 Recently Resolved Issues

3.1 (2020-03-27) Intel-TBB checksums don’t match

Intel recently moved the TBB project on GitHub to a new repository, oneAPI-SRC/oneTBB. When downloading the intel-tbb tar file, github makes the tar file on the fly, and this includes the new top-level directory name (oneTBB). The contents of the tar file are the same, but the new directory name is enough to invalidate the spack checksum.

Fixed: The new checksums were added to spack in commit 16f104aafe2b on 2020-03-27. So, if you update your spack repository after this commit, then the checksums will match.

Workaround: If you have an older spack repository, then you will need to disable verifying checksums.

spack install --no-checksum intel-tbb

4 General Problems

4.1 Unable to fetch

Sometimes spack fails to download the source file(s) for some package and dies with a message similar to this.

==> Fetching from https://ftpmirror.gnu.org/m4/m4-1.4.18.tar.gz failed.
==> Error: FetchError: All fetchers failed for m4-1.4.18-vorbvkcjfac43b7vuswsvnm6xe7w7or5

This problem is usually temporary and the solution is to either wait a few minutes or an hour and try again, or else download the file manually and put it into a spack mirror.

4.2 Connection timeout

Another way fetch can fail is with a connection timeout. Some sites, especially sourceforge are often slow to connect. If this happens, then increase the connection timeout in config.yaml to 30 or 60 seconds (default is 10 seconds).

4.3 New version breaks the build

Sometimes the latest version of some package breaks the build. This has happened a couple of times where a new version of Boost has broken the build for Dyninst. The solution is to revert the package to an earlier version until the rest of the code catches up.

4.4 Spack core breaks the build

Sometimes but rarely, something in the spack core will change or break the code in some package.py file. The solution is to look through the spack git log and revert the repository to a recent commit before the breakage.

5 Long Term Issues

5.1 Boost 1.68.0

Avoid boost version 1.68.0, it breaks the build for hpctoolkit. Version 1.70.0 works with the latest version of dyninst (10.1.0), or else 1.66.0 is good and works with all versions of dyninst.

5.2 Elfutils 0.176

Elfutils 0.176 requires glibc 2.16 or later (for aligned_alloc) and won’t work with an older glibc, including RedHat or CentOS 6.x and Blue Gene. On systems with an old glibc, use version 0.175.