Building oneAPI from Source

Build oneAPI completely from git

I'm back!! A few raw posts have been languishing and I decided the end of the year was the perfect time to put them out there. This will be one of three (hopefully).

I'm going to focus on how to build oneAPI from git. This is somewhat of a return to my earlier blog post where I talked about how to build the DPC++ compiler and it included the binary versions of the openCL and level zero run time.

That's all well and good but let's consider how we could build the run time from git completely. The ability to do reproducible builds is going to be important later when we dive into buildimg packaging that is up to date and available on any Linux distro.

The build only supports Intel hardware at this point since level zero doesn't support NVidia or AMD GPUs. If you are looking for such support, you might consider Codeplay's plugins that will allow you to use NVidia and AMD hardware.

These blog pages typically only focus on what we can do from an open source perspective and won't really focus on anything that has binary blobs if we can avoid it.

DISCLAIMER: Please don't use this set up for a production environment. It is not well tested. If you find any problems, please reach out in the comments so that I can help debug and update the blog post appropriately.

Setting up the build environment

I like to use containers which makes it easy to quickly set up and automate using distrobox.

You should be able to use whatever Linux distribution you want as long as you can install distrobox. You can, of course, use a virtual machine to accomplish this. I've used Vagrant successfully.

Assuming that you have distrobox installed - let's get to it.

Decide where you want to have the build for instance: ~/src/oneapi-build.

$ distrobox create --image docker.io/library/ubuntu 20.04 --name "oneAPIBuild"
$ distrobox enter oneAPIBuild

You should now be in a container running Ubuntu 20.04.

Install the appropriate packages

$ sudo apt-get install -y build-essential git libssl-dev flex bison libz-dev python3-mako python3-pip automake autoconf libtool pkg-config ruby

You will need a recent version of cmake for the builds. The one that comes with 20.04 is too old.

$  wget https://github.com/Kitware/CMake/releases/download/v3.28.1/cmake-3.28.1.tar.gz
$ tar xvfpz cmake-3.28.1.tar.gz
$ cd cmake-3.28.1
$ ./boostrap
$ ./configure
$ make 
$ sudo make install

Now you should have everything you need for the build.

Understanding the oneAPI Build

There are a number of prerequisites before you start the build. Here is a graphic of how the oneAPI build is put together.

The order of build is:

1) Intel Graphics Engine and its relevant prerequisites which consist of:

Intel Graphics Compiler (igc)
1. SPIRV Headers
2. SPIRV Tools
3. copy of the llvm project
4. vc-intrinsics
5. intel graphcis compiler 2) ocl-icd 3) GMMLib 4) NEO - Intel Compute Runtime - NEO is the primary GPU graphics driver and uses OpenCL to talk to the GPU.
Has the following pre-requisites:
1. Intel graphics compiler (IGC)
2. GMMLib 6) Level Zero 7) DPC++ SYCL Compiler 8) oneTBB Library

That's the progression to get the full build going.

Intel Graphics Engine

Let's start building the first prerequisites for the NEO which is the Intel Graphics Engine:

$ mkdir igc-workspace && cd igc-workspace
$ git clone https://github.com/KhronosGroup/SPIRV-Headers.git --depth 1

$ git clone https://github.com/KhronosGroup/SPIRV-Tools.git --depth 1

$ git clone -b llvmorg-14.0.5 https://github.com/llvm/llvm-project llvm-project --depth 1

$ git clone -b ocl-open-140 https://github.com/intel/opencl-clang llvm-project/llvm/projects/opencl-clang --depth 1

$ git clone -b llvm_release_140 https://github.com/KhronosGroup/SPIRV-LLVM-Translator llvm-project/llvm/projects/llvm-spirv --depth 1

$ git clone https://github.com/intel/vc-intrinsics --depth 1

$ git clone https://github.com/intel/intel-graphics-compiler igc --depth 1

$ mkdir build && cd build
$ cmake ../igc -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make -j `nproc`
$ sudo make install

It should build cleanly. If it doesn't - please check any errors and make sure you have all the prerequisites.

ocl-icd

ocl-icd is an OpenCL loader - and is used to link opencl software when compiling. Make sure you are back in your usual oneapi-build directory.

$ pwd
~/src/oneapi-build
$ git clone https://github.com/OCL-dev/ocl-icd --depth 1
$ cd ocl-icd
$ ./bootstrap
$ ./configure
$ make
$ sudo make install

Install GMMLib

NEO requires GMMLib as one of its prerequisites so we will build that now.

$ pwd
~/src/oneapi-build
$ git clone https://github.com/intel/gmmlib --depth 1
$ cd gmmlib
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ sudo make install

Install NEO

NEO is the Intel Compute Runtime and is necessary for the SYCL based applications to talk to the GPU. Go back to your ~/src/oneapi-build directory.

$ pwd 
~/src/oneapi-build # please note this output will be different for you
$ mkdir neo-workspace
$ cd neo-workspace
$ git clone https://github.com/intel/compute-runtime neo –depth 1
$ cd neo
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ sudo make install

Install level-zero

This is the main part of oneAPI and interfaces with NEO or other run times. Since NEO is the only one at the moment - it will only work with Intel devices.

$ pwd
~/src/oneapi-build
$ git clone https://github.com/oneapi-src/level-zero --depth 1
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ sudo make install

Install the DPC++ Compiler

Now to build the SYCL Compiler.

$ pwd
~/src/oneapi-build
$ mkdir sycl_workspace && cd sycl_workspace
$ export DPCPP_HOME=`pwd`
$ git clone https://github.com/intel/llvm.git -b sycl --depth 1
$ python3 $DPCPP_HOME/llvm/buildbot/configure.py --cmake-opt CMAKE_BUILD_PREFIX="/usr/local"
$ python3 $DPCPP_HOME/llvm/buildbot/compile.py

Install oneTBB

Finally, we need to install oneTBB

$ pwd
~/src/oneapi-build
$ git clone https://github.com/oneapi-src/oneTBB --depth 1
$ cd oneTBB
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ make install

Set the LD_LIBRARY_PATH

We need to make sure that the linker can find the proper libraries. The easiest way is to either set the LD_LIBRARY_PATH in your .bashrc or put it in /etc/environment.

$ export LD_LIBRARY_PATH="/usr/local/lib"

Test the environment

$ cd ~/src
$ mkdir simple-oneapi-app
$ cd simple-oneapi-app
$ cat > simple-oneapi-app.cpp
#include <sycl/sycl.hpp>

int main() {
  // Creating buffer of 4 ints to be used inside the kernel code
  sycl::buffer<sycl::cl_int, 1> Buffer(4);

  // Creating SYCL queue
  sycl::queue Queue;

  // Size of index space for kernel
  sycl::range<1> NumOfWorkItems{Buffer.size()};

  // Submitting command group(work) to queue
  Queue.submit([&](sycl::handler &cgh) {
    // Getting write only access to the buffer on a device
    auto Accessor = Buffer.get_access<sycl::access::mode::write>(cgh);
    // Executing kernel
    cgh.parallel_for<class FillBuffer>(
        NumOfWorkItems, [=](sycl::id<1> WIid) {
          // Fill buffer with indexes
          Accessor[WIid] = (sycl::cl_int)WIid.get(0);
        });
  });

  // Getting read only access to the buffer on the host.
  // Implicit barrier waiting for queue to complete the work.
  const auto HostAccessor = Buffer.get_access<sycl::access::mode::read>();

  // Check the results
  bool MismatchFound = false;
  for (size_t I = 0; I < Buffer.size(); ++I) {
    if (HostAccessor[I] != I) {
      std::cout << "The result is incorrect for element: " << I
                << " , expected: " << I << " , got: " << HostAccessor[I]
                << std::endl;
      MismatchFound = true;
    }
  }

  if (!MismatchFound) {
    std::cout << "The results are correct!" << std::endl;
  }

  return MismatchFound;
}

$ clang++ -fsycl simple-oneapi-app.cpp -o simple-oneapi-app

When you run the app you should get "Results are correct!".

$ ./simple-oneapi-app
Results are correct!

Now you've successfully built oneAPI from source!

Let me know if you have any issues with the instructions in the comments.

Photo by Dominik Lückmann on Unsplash

Blog