INSTALL: Subprocess.CalledProcessError when installing the package

[handraqui]

Describe the error
I can't install Firedrake due to an error while trying to install the firedrake package after all the dependencies.

Steps to Reproduce
I have a brand new Ubuntu 20.04.6 LTS with a Python 3.8.10 pre-installed. I followed the site instructions, executing on terminal:

curl -O https://raw.githubusercontent.com/firedrakeproject/firedrake/master/scripts/firedrake-install
python3 firedrake-install

I had a problem with the libsupermesh dependency package, so after I tried for the first time, I executed the following, due to something that was in the install.log, after activating the virtual environment:

firedrake-update

After this, even if I would delete the whole firedrake folder and start everything all over again, I didn't have any problem with libsupermesh anymore, but I would already the same error message that I'm getting now (I'll show it below). In another attempt to fix everything, I also did

cmake --version

From which I realized I was running cmake version 3.16.3, which a log would say that was not the needed one. Thus, I did:

sudo apt remove cmake
pip install cmake --upgrade

From which now I have "Successfully installed cmake-3.28.3".

Neither running firedrake-update nor python3 firedrake-install is working in the end. I also updated the numpy version, which was already numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl, just as it is now.

Expected behavior
I expect the installation to be successful.

Error message
The whole thing:

Running firedrake-install
Checking for presence of package build-essential...
  installed.
Checking for presence of package autoconf...
  installed.
Checking for presence of package automake...
  installed.
Checking for presence of package bison...
  installed.
Checking for presence of package flex...
  installed.
Checking for presence of package cmake...
  installed.
Checking for presence of package gfortran...
  installed.
Checking for presence of package git...
  installed.
Checking for presence of package libcurl4-openssl-dev...
  installed.
Checking for presence of package pkgconf...
  installed.
Checking for presence of package libtool...
  installed.
Checking for presence of package libxml2-dev...
  installed.
Checking for presence of package python3-dev...
  installed.
Checking for presence of package python3-pip...
  installed.
Checking for presence of package python3-tk...
  installed.
Checking for presence of package python3-venv...
  installed.
Checking for presence of package zlib1g-dev...
  installed.
Checking for presence of package libboost-dev...
  installed.
Checking for presence of package libopenblas-dev...
  installed.
Creating firedrake venv in '/home/USER/firedrake'.
BLAS config is {}
Configuration saved to /home/USER/firedrake/.configuration.json
Cloning firedrake

Failed to clone firedrake using ssh, falling back to https.
Successfully cloned repository firedrake.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Creating firedrake-update script.
Installing numpy using setuptools==59.2.0 and wheel==0.37.0 and Cython==0.29.36
Installing numpy==1.24 due to performance regression
Updating setuptools and wheel to latest versions
Pip installing VTK to venv
Cloning the dependencies of firedrake
Cloning ufl

Failed to clone ufl using ssh, falling back to https.
Successfully cloned repository ufl.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning fiat

Failed to clone fiat using ssh, falling back to https.
Successfully cloned repository fiat.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning FInAT

Failed to clone FInAT using ssh, falling back to https.
Successfully cloned repository FInAT.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning tsfc

Failed to clone tsfc using ssh, falling back to https.
Successfully cloned repository tsfc.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning PyOP2

Failed to clone PyOP2 using ssh, falling back to https.
Successfully cloned repository PyOP2.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning pyadjoint

Failed to clone pyadjoint using ssh, falling back to https.
Successfully cloned repository pyadjoint.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning petsc

Failed to clone petsc using ssh, falling back to https.
Successfully cloned repository petsc.
Checking out branch firedrake
Successfully checked out branch firedrake
Updating submodules.
Successfully updated submodules.
Cloning pytest-mpi

Failed to clone pytest-mpi using ssh, falling back to https.
Successfully cloned repository pytest-mpi.
Checking out branch main
Successfully checked out branch main
Updating submodules.
Successfully updated submodules.
Cloning the dependencies of PyOP2
Cloning loopy

Failed to clone loopy using ssh, falling back to https.
Successfully cloned repository loopy.
Checking out branch main
Successfully checked out branch main
Updating submodules.
Successfully updated submodules.
Installing petsc/
Depending on your platform, PETSc may take an hour or more to build!
Installing pip dependencies for loopy
Installing pip dependencies for petsc4py
No dependencies found. Skipping.
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for pyadjoint
No dependencies found. Skipping.
Installing pip dependencies for pytest-mpi
No dependencies found. Skipping.
Installing pip dependencies for firedrake
Installing h5py
Cloning h5py

Failed to clone h5py using ssh, falling back to https.
Successfully cloned repository h5py.
Checking out branch firedrake
Successfully checked out branch firedrake
Updating submodules.
Successfully updated submodules.
Linking h5py against HDF5_DIR=/home/USER/firedrake/src/petsc/default

Removing existing h5py installations

Installing h5py/
Installing loopy/
Installing petsc4py/
Installing ufl/
Installing fiat/
Installing FInAT/
Installing tsfc/
Installing pyadjoint/
Installing pytest-mpi/
Installing libsupermesh
Cloning libsupermesh

Failed to clone libsupermesh using ssh, falling back to https.
Successfully cloned repository libsupermesh.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Traceback (most recent call last):
  File "firedrake-install", line 1923, in <module>
    build_and_install_libsupermesh(cc, cxx, f90, mpiexec)
  File "firedrake-install", line 1288, in build_and_install_libsupermesh
    check_call(cmd)
  File "firedrake-install", line 674, in check_call
    log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=os.environ).decode())
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cmake', '..', '-DBUILD_SHARED_LIBS=ON', '-DCMAKE_INSTALL_PREFIX=/home/USER/firedrake', '-DMPI_C_COMPILER=/home/USER/firedrake/src/petsc/default/bin/mpicc', '-DMPI_CXX_COMPILER=/home/USER/firedrake/src/petsc/default/bin/mpicxx', '-DMPI_Fortran_COMPILER=/home/USER/firedrake/src/petsc/default/bin/mpif90', '-DCMAKE_Fortran_COMPILER=/home/USER/firedrake/src/petsc/default/bin/mpif90', '-DMPIEXEC_EXECUTABLE=/home/USER/firedrake/src/petsc/default/bin/mpiexec']' returned non-zero exit status 1.

Install log saved in /home/USER/firedrake-install.log

Add error message with full backtrace.
Please add these as text using three backticks (`) for highlighting.
Please do not add screenshots.

Please also include either firedrake-install.log which is found in
the directory firedrake-install was run or firedrake-update.log
found in the virtualenv directory.

Additionally please include the PETSc configure log located in
$VIRTUAL_ENV/src/petsc/configure.log.
configure.log

firedrake-install.log

Environment:

• OS:Ubuntu 20.04.6 LTS
• Python version: 3.8.10
• numpy-1.24.0
• pip 24.0
• If any other relevant detail should be mentioned, please tell me.

[connorjward]

The log is is failing to build libsupermesh because of the cmake version:

2024-03-18 17:51:25,394 DEBUG  Running command 'cmake .. -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=/home/henriqueaquino/firedrake -DMPI_C_COMPILER=/home/henriqueaquino/firedrake/src/petsc/default/bin/mpicc -DMPI_CXX_COMPILER=/home/henriqueaquino/firedrake/src/petsc/default/bin/mpicxx -DMPI_Fortran_COMPILER=/home/henriqueaquino/firedrake/src/petsc/default/bin/mpif90 -DCMAKE_Fortran_COMPILER=/home/henriqueaquino/firedrake/src/petsc/default/bin/mpif90 -DMPIEXEC_EXECUTABLE=/home/henriqueaquino/firedrake/src/petsc/default/bin/mpiexec'
2024-03-18 17:51:25,406 DEBUG  CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
  CMake 3.18 or higher is required.  You are running version 3.16.3


-- Configuring incomplete, errors occurred!

Is this the most recent log file? Because if so your efforts to use a more recent version of cmake have not worked.

Having just checked we always install cmake for linux platforms by default. Could you try running firedrake-install with --no-package-manager after running sudo apt remove cmake? You should check that cmake --version gives you a more modern version before running firedrake-install.

[handraqui]

Contributor

I tried some other things, and ended up deleting and installing firedrake again, and also tried to upgrade it. I tried to upgrade cmake following

sudo remove cmake sudo apt install build-essential libssl-dev wget https://github.com/Kitware/CMake/releases/download/v3.20.2/cmake-3.20.2.tar.gz tar -zxvf cmake-3.20.2.tar.gz cd cmake-3.20.2 ./bootstrap make

Then, I once again ran firedrake-update in the virtual environment, and my most recent logs are the following:
firedrake-install.log

firedrake-update.log

Ironically, after doing all this, cmake --version gives me "3.16.3".

[connorjward]

Contributor

I tried some other things, and ended up deleting and installing firedrake again, and also tried to upgrade it. I tried to upgrade cmake following

sudo remove cmake sudo apt install build-essential libssl-dev wget https://github.com/Kitware/CMake/releases/download/v3.20.2/cmake-3.20.2.tar.gz tar -zxvf cmake-3.20.2.tar.gz cd cmake-3.20.2 ./bootstrap make

Then, I once again ran firedrake-update in the virtual environment, and my most recent logs are the following: firedrake-install.log

firedrake-update.log

Ironically, after doing all this, cmake --version gives me "3.16.3".

The install script automatically installs cmake from apt. You should pass --no-package-manager to disable this.

[dmitryshribak]

Why does the install script automatically install cmake 3.16.3 if the firedrake needs cmake>3.18 ???

[JDBetteridge]

This is a known issue that I have documented in this new issue.

I'm closing this issue in line with policy and as we no longer support Python 3.8