How to do it...

1. First, we will show how to install TensorFlow from your terminal (make sure that you adjust the link to the TensorFlow wheel for your platform and Python version accordingly):

pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.3.0-cp35-cp35m-linux_x86_64.whl

This will install the GPU-enabled version of TensorFlow and the correct dependencies.

1. You can now import the TensorFlow library into your Python environment:

import tensorflow as tf

1. To provide a dummy dataset, we will use numpy and the following code:

import numpy as np
x_input = np.array([[1,2,3,4,5]])
y_input = np.array([[10]])

1. When defining a TensorFlow model, you cannot feed the data directly to your model. You should create a placeholder that acts like an entry point for your data feed:

x = tf.placeholder(tf.float32, [None, 5])
y = tf.placeholder(tf.float32, [None, 1])

1. Afterwards, you apply some operations to the placeholder with some variables. For example:

W = tf.Variable(tf.zeros([5, 1]))
b = tf.Variable(tf.zeros([1]))
y_pred = tf.matmul(x, W)+b

1. Next, define a loss function as follows:

loss = tf.reduce_sum(tf.pow((y-y_pred), 2))

1. We need to specify the optimizer and the variable that we want to minimize:

train = tf.train.GradientDescentOptimizer(0.0001).minimize(loss)

1. In TensorFlow, it's important that you initialize all variables. Therefore, we create a variable called init:

init = tf.global_variables_initializer()

We should note that this command doesn't initialize the variables yet; this is done when we run a session.

1. Next, we create a session and run the training for 10 epochs:

sess = tf.Session()
sess.run(init)

for i in range(10):
    feed_dict = {x: x_input, y: y_input}
    sess.run(train, feed_dict=feed_dict)

1. If we also want to extract the costs, we can do so by adding it as follows:

sess = tf.Session()
sess.run(init)

for i in range(10):
    feed_dict = {x: x_input, y: y_input}
    _, loss_value = sess.run([train, loss], feed_dict=feed_dict)
    print(loss_value)

1. If we want to use multiple GPUs, we should specify this explicitly. For example, take this part of code from the TensorFlow documentation:

c = []
for d in ['/gpu:0', '/gpu:1']:
    with tf.device(d):
        a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3])
        b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2])
c.append(tf.matmul(a, b))
with tf.device('/cpu:0'):
    sum = tf.add_n(c)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print(sess.run(sum))

As you can see, this gives a lot of flexibility in how the computations are handled and by which device.