For Tensorflow GPU training, you'll need CUDA and CuDNN. I couldn't get the right version of CuDNN for Tensorflow on my linux desktop, because registration on the nvidia site is broken.
This gist explains how to install the correct versions of CUDA and CuDNN on Ubuntu. It worked for me on Ubuntu 16.04 and I wanted to share with the community.
There's nothing secret here or sketchy -- downloads are off the Nvidia servers.