.. _Using Bee:

Using Bee
=========

.. _Bee login:

Login
-----

There is 1 login node:

============================  =======================================
Hostname                      Node type
============================  =======================================
``bee``                       Bee login node
============================  =======================================

Host key fingerprint:

=========  ========================================================
Algorithm  Fingerprint (SHA256)
=========  ========================================================
RSA        ``SHA256:1GlrJm7DIZ6yJTNTvivsXKBci4+IT31yW8pmpp35NDI``
ECDSA      ``SHA256:oHeyvgUhqF3lDaZLtdbjZBoOj8cQbgKNiojfLW9puIA``
ED25519    ``SHA256:eywXc/TEtuhxIhle6fvrAzvgba2O30BZkCeXMZO7pJg``
=========  ========================================================

.. _Bee building dependencies:

Building dependencies
---------------------

Boost
^^^^^

.. code-block:: bash

    # last update: August 2023
    module load gcc/8.4.0 openmpi/4.0.5
    mkdir boost-build
    cd boost-build
    BOOST_VERSION=1.82.0
    BOOST_DOMAIN="https://boostorg.jfrog.io/artifactory/main"
    BOOST_ROOT="${HOME}/bin/boost_mpi_${BOOST_VERSION//./_}"
    mkdir -p "${BOOST_ROOT}"
    curl -sL "${BOOST_DOMAIN}/release/${BOOST_VERSION}/source/boost_${BOOST_VERSION//./_}.tar.bz2" | tar xj
    cd "boost_${BOOST_VERSION//./_}"
    echo 'using mpi ;' > tools/build/src/user-config.jam
    ./bootstrap.sh --with-libraries=filesystem,system,mpi,serialization,test
    ./b2 -j 4 install --prefix="${BOOST_ROOT}"

FFTW
^^^^

.. code-block:: bash

    # last update: August 2023
    module load gcc/8.4.0 openmpi/4.0.5
    mkdir fftw-build
    cd fftw-build
    FFTW3_VERSION=3.3.10
    FFTW3_ROOT="${HOME}/bin/fftw_${FFTW3_VERSION//./_}"
    curl -sL "https://www.fftw.org/fftw-${FFTW3_VERSION}.tar.gz" | tar xz
    cd "fftw-${FFTW3_VERSION}"
    ./configure --enable-shared --enable-mpi --enable-threads --enable-openmp \
        --disable-fortran --enable-avx --prefix="${FFTW3_ROOT}"
    make -j 4
    make install
    make clean

.. _Bee building software:

Building software
-----------------

ESPResSo
^^^^^^^^

Release 4.2:

.. code-block:: bash

    # last update: August 2023
    module load gcc/8.4.0 openmpi/4.0.5 boost/1.75.0 python/3.8 cmake/3.26.3 \
        py-cython/0.29.21 fftw/3.3.9 cuda/10.2.89 hdf5/1.10.7 py-pip

    git clone --recursive --branch 4.2 --origin upstream \
        https://github.com/espressomd/espresso.git espresso-4.2
    cd espresso-4.2
    python3 -m pip install --user -c "requirements.txt" setuptools numpy scipy vtk h5py
    mkdir build
    cd build
    cp ../maintainer/configs/maxset.hpp myconfig.hpp
    sed -i "/ADDITIONAL_CHECKS/d" myconfig.hpp
    cmake .. -D CMAKE_BUILD_TYPE=Release -D WITH_CUDA=ON \
        -D WITH_CCACHE=OFF -D WITH_SCAFACOS=OFF -D WITH_HDF5=ON
    make -j 4

Release 4.3:

.. code-block:: bash

    # last update: August 2023
    module load gcc/8.4.0 openmpi/4.0.5 cmake/3.26.3 \
        py-cython/0.29.21 hdf5/1.10.7 py-pip
    CLUSTER_FFTW3_VERSION=3.3.10
    CLUSTER_BOOST_VERSION=1.82.0
    export BOOST_ROOT="${HOME}/bin/boost_mpi_${CLUSTER_BOOST_VERSION//./_}"
    export FFTW3_ROOT="${HOME}/bin/fftw_${CLUSTER_FFTW3_VERSION//./_}"
    export LD_LIBRARY_PATH="${BOOST_ROOT}/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
    export LD_LIBRARY_PATH="${FFTW3_ROOT}/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

    git clone --recursive --branch python --origin upstream \
        https://github.com/espressomd/espresso.git espresso-4.3
    cd espresso-4.3
    sed -ri 's/find_package\(Python 3\.[0-9] REQUIRED/find_package\(Python 3.6 REQUIRED/' CMakeLists.txt
    python3 -m pip install --user -c "requirements.txt" setuptools numpy scipy vtk h5py
    mkdir build
    cd build
    cp ../maintainer/configs/maxset.hpp myconfig.hpp
    sed -i "/ADDITIONAL_CHECKS/d" myconfig.hpp
    cmake .. -D CMAKE_BUILD_TYPE=Release -D ESPRESSO_BUILD_WITH_CUDA=OFF \
        -D ESPRESSO_BUILD_WITH_CCACHE=OFF -D ESPRESSO_BUILD_WITH_WALBERLA=ON \
        -D ESPRESSO_BUILD_WITH_SCAFACOS=OFF -D ESPRESSO_BUILD_WITH_HDF5=ON
    make -j 4

.. _Bee submitting jobs:

Submitting jobs
---------------

Batch command:

.. code-block:: bash

    sbatch --job-name="test" --nodes=1 --ntasks=4 --mem-per-cpu=2GB job.sh

Job script:

.. code-block:: bash

    #!/bin/bash
    #SBATCH --time=00:10:00
    #SBATCH --output %j.stdout
    #SBATCH --error  %j.stderr
    module load gcc/8.4.0 openmpi/4.0.5 boost/1.75.0 python/3.8 fftw/3.3.9 cuda/10.2.89
    srun --cpu-bind=cores ./pypresso testsuite/python/particle.py