TensorFlow – GPU Support

Installing Cuda on Ubuntu

  • To install tensorflow with GPU support, and to run using CUDA, several amount of steps are required.
  • Verify the installed video card, and Ubuntu kernel, using the neoftech command
neofetch
  • Check for installed Nvidia drivers:
nvidia-smi

+—————————————————————————————+
| NVIDIA-SMI 545.23.08 Driver Version: 545.23.08 CUDA Version: 12.3     |
|—————————————–+———————-+———————+

Please take a look at section 3.10 – follow instructions over there.

Here we present part of the main installation process.


3.10.1. Prepare Ubuntu

  1. Perform the pre-installation actions.
  2. The kernel headers and development packages for the currently running kernel can be installed with:
    sudo apt-get install linux-headers-$(uname -r)
    
  3. Remove Outdated Signing Key:
    sudo apt-key del 7fa2af80
    
  4. Choose an installation method: local repo or network repo.

3.10.2. Local Repo Installation for Ubuntu

  1. Install local repository on file system:
    sudo dpkg -i cuda-repo-<distro>_<version>_<architecture>.deb
    
  2. Enroll ephemeral public GPG key:
    sudo cp /var/cuda-repo-<distro>-X-Y-local/cuda-*-keyring.gpg /usr/share/keyrings/
    
  3. Add pin file to prioritize CUDA repository:
    wget https://developer.download.nvidia.com/compute/cuda/repos/<distro>/x86_64/cuda-<distro>.pin
     sudo mv cuda-<distro>.pin /etc/apt/preferences.d/cuda-repository-pin-600
    

3.10.3. Network Repo Installation for Ubuntu

The new GPG public key for the CUDA repository is 3bf863cc. This must be enrolled on the system, either using the cuda-keyring package or manually; the apt-key command is deprecated and not recommended.

  1. Install the new cuda-keyring package:
    wget https://developer.download.nvidia.com/compute/cuda/repos/$distro/$arch/cuda-keyring_1.1-1_all.deb
    sudo dpkg -i cuda-keyring_1.1-1_all.deb
    

    where $distro/$arch should be replaced by one of the following:

    • ubuntu1604/x86_64
    • ubuntu1804/cross-linux-sbsa
    • ubuntu1804/ppc64el
    • ubuntu1804/sbsa
    • ubuntu1804/x86_64
    • ubuntu2004/cross-linux-aarch64
    • ubuntu2004/arm64
    • ubuntu2004/cross-linux-sbsa
    • ubuntu2004/sbsa
    • ubuntu2004/x86_64
    • ubuntu2204/sbsa
    • ubuntu2204/x86_64

    Note

    arm64-Jetson repos:

    • native: $distro/arm64
    • cross: $distro/cross-linux-aarch64
    sudo dpkg -i cuda-keyring_1.1-1_all.deb
    
  2. Or if you are unable to install the cuda-keyring package, you can optionally:
    1. Enroll the new signing key manually:
      wget https://developer.download.nvidia.com/compute/cuda/repos/<distro>/<arch>/cuda-archive-keyring.gpg
      sudo mv cuda-archive-keyring.gpg /usr/share/keyrings/cuda-archive-keyring.gpg
      
    2. Enable the network repository:
      echo "deb [signed-by=/usr/share/keyrings/cuda-archive-keyring.gpg] https://developer.download.nvidia.com/compute/cuda/repos/<distro>/<arch>/ /" | sudo tee /etc/apt/sources.list.d/cuda-<distro>-<arch>.list
      
    3. Add pin file to prioritize CUDA repository:
      wget https://developer.download.nvidia.com/compute/cuda/repos/<distro>/<arch>/cuda-<distro>.pin
      sudo mv cuda-<distro>.pin /etc/apt/preferences.d/cuda-repository-pin-600
      

3.10.4. Common Installation Instructions for Ubuntu

These instructions apply to both local and network installation for Ubuntu.

  1. Update the Apt repository cache:
    sudo apt-get update
    
  2. Install CUDA SDK:

    Note

    These two commands must be executed separately.

    sudo apt-get install cuda-toolkit
    

    To include all GDS packages:

    sudo apt-get install nvidia-gds
    
  3. Reboot the system
    sudo reboot
    

 

Some actions must be taken after the installation before the CUDA Toolkit and Driver can be used.

13.1.1. Environment Setup

The PATH variable needs to include export PATH=/usr/local/cuda-12./bin${PATH:+:${PATH}}. Nsight Compute has moved to /opt/nvidia/nsight-compute/ only in rpm/deb installation method. When using .run installer it is still located under /usr/local/cuda-12.2/.

To add this path to the PATH variable:

export PATH=/usr/local/cuda-12.2/bin${PATH:+:${PATH}}

In addition, when using the runfile installation method, the LD_LIBRARY_PATH variable needs to contain /usr/local/cuda-12.2/lib64 on a 64-bit system, or /usr/local/cuda-12.2/lib on a 32-bit system

  • To change the environment variables for 64-bit operating systems:
    export LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64\
                             ${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
    
  • To change the environment variables for 32-bit operating systems:
    export LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib\
                             ${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
    

Note that the above paths change when using a custom install path with the runfile installation method.


 

Installing cudnn

Download tar from https://developer.nvidia.com/rdp/cudnn-download

$ cd Downloads
$ tar -xzvf cudnn-linux-x86_64-8.4.1.50_cuda1<version>-archive.tar.xz
$ sudo cp cudnn/include/cudnn*.h /usr/local/cuda-<version>/include
$ sudo cp -P cudnn/lib/libcudnn* /usr/local/cuda-<version>/lib64
$ sudo chmod a+r /usr/local/cuda-11.7/include/cudnn*.h /usr/local/cuda-<version>/lib64/libcudnn*
$ sudo apt update

Conda Environment

Next, create a conda environment for the suitable python version

$ cd Downloads
conda create -p /path_to_conda/name_enviroment -c  python==
conda activate /path_to_conda/name_enviroment
pip install pip upgrade
pip install tensorflow