How to use NVIDIA graphic card in docker container

TechContainer
6 minutes read (About 913 words)

What is NVIDIA-Docker?

NVIDIA designed NVIDIA-Docker in 2016 to enable portability in Docker images that leverage NVIDIA GPUs. It allowed driver agnostic CUDA images and provided a Docker command line wrapper that mounted the user mode components of the driver and the GPU device files into the container at launch.

The NVIDIA-Docker overall architecture

The NVIDIA-Docker enables deployment of GPU-accelerated applications across any Linux server. Using Docker, we can develop and prototype GPU applications on a workstation, and then ship and run those applications anywhere that supports GPU containers.

The requirements

Just for your information, the current latest version of NVIDIA-Docker is 2.0. So in this tutorial we will just care about the requirements of this version. In order to run NVIDIA-Docker version 2.0, your Operation System must have:

  • GNU/Linux x86_64 with kernel version > 3.10
  • Docker >= 1.12
  • NVIDIA GPU with Architecture > Fermi (2.1)
  • NVIDIA driver >= 361.93

According to NVIDIA document, they support various Linux distribution.

  • Ubuntu 14.04/16.04/18.04
  • Debian Jessie/Stretch
  • CentOS 7
  • RHEL 7.4/7.5
  • Amazon Linux 1/2

Just for demonstration purpose, in this tutorial i will use Ubuntu 16.04 LTS . The same steps can be applied for other Debian family distribution.

Install NVIDIA-Docker on Ubuntu

First of all, you have to make sure your graphic card is working properly on the host operating system. It is required to have CUDA driver to run a NVIDIA graphic card. If you haven’t installed it yet, follow these 2 following steps.

Download the CUDA repository package and install it

1
2
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_10.0.130-1_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1604_10.0.130-1_amd64.deb

Then update our system apt repository and install CUDA driver

1
2
$ sudo apt-get update
$ sudo apt-get install cuda

Once you installed the CUDA driver, it is time to load the NVIDIA-Docker. Let’s add new repository for nvidia-docker2 package.

1
2
3
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey |   sudo apt-key add -
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

Then update our system apt repository and install nvidia-docker2. Please note that your /etc/docker/daemon.json file might be modified to load the new plugin.

1
2
$ sudo apt-get update
$ sudo apt-get install -y nvidia-docker2

Restart Docker daemon to load the nvidia-docker2 plugin

1
$ sudo systemctl restart docker

That’s all for the installation. It is simple right? Now we can verify whether our GPU can work inside the Docker container.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ sudo docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
Tue Oct 16 06:24:30 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.48                 Driver Version: 410.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1050    Off  | 00000000:02:00.0 Off |                  N/A |
|  0%   52C    P0    N/A /  70W |      0MiB /  1998MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

The command above will try to run a docker container from nvidia/cuda image and execute nvidia-smi command to show the basic GPU statistics.

GPU Benchmark in Docker container

Following is the simple code to do the test with Tensorflow using CPU/GPU acceleration. Let’s create a new file called benchmark.py with following content

benchmark.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import sys
import numpy as np
import tensorflow as tf
from datetime import datetime

device_name = sys.argv[1]  # Choose device from cmd line. Options: gpu or cpu
shape = (int(sys.argv[2]), int(sys.argv[2]))
if device_name == "gpu":
    device_name = "/gpu:0"
else:
    device_name = "/cpu:0"

with tf.device(device_name):
    random_matrix = tf.random_uniform(shape=shape, minval=0, maxval=1)
    dot_operation = tf.matmul(random_matrix, tf.transpose(random_matrix))
    sum_operation = tf.reduce_sum(dot_operation)

startTime = datetime.now()
with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as session:
        result = session.run(sum_operation)
        print(result)

# It can be hard to see the results on the terminal with lots of output -- add some newlines to improve readability.
print("\n" * 5)
print("Shape:", shape, "Device:", device_name)
print("Time taken:", str(datetime.now() - startTime))

Run Tensorflow test with CPU

1
2
3
$ docker run --runtime=nvidia --rm -ti -v "${PWD}:/app" tensorflow/tensorflow:latest-gpu python /app/benchmark.py cpu 10000
('Shape:', (10000, 10000), 'Device:', '/cpu:0')
('Time taken:', '0:00:19.239894')

Run Tensorflow test with GPU

1
2
3
$ docker run --runtime=nvidia --rm -ti -v "${PWD}:/app" tensorflow/tensorflow:latest-gpu python /app/benchmark.py gpu 10000
('Shape:', (10000, 10000), 'Device:', '/gpu:0')
('Time taken:', '0:00:06.165169')

You would see the difference in the performance when with Tensorflow test using CPU vs GPU.

What’s next?

If you want to build your own application which use the GPU acceleration, you can build a docker image for it and launch the application from the container. NVIDIA also offer their Docker images which your image can base on. You can get them from Docker Hub, and their Docker files are available on Github.

Since NVIDIA-Docker is an open source project. It is open to discuss. You can report issue on their Github repo. The Frequently Asked Questions also contains helpful information that you might want to take a look before playing further with NVIDIA-Docker.

Container, Docker
Share Comments