Cracking with the desktop computer is not always a good idea. It can be quite noisy when you are trying to sleep. The room can become too hot for a hacker. And most important of all, all the hashes are already in the cloud! Let’s get started.
Create EC2 GPU Instance
Take a look the the current GPU instance details at amazon and choose the right version for your purse. Spin up the instance and login via ssh before proceeding. For starters I chose the x2.2xlarge instance type. The AMI image ID was ami-87564feb or simply choose a Ubuntu trusty amd64 image.
Setup EC2 GPU Instance
Before we begin, let’s verify that the system has got a NVIDIA graphics card.
$ lspci | grep -i "NVIDIA"
Looks good! In order to stick to the best practice, let’s follow the guide. Start by installing the required packages, such as gcc and header files.
$ sudo apt-get update $ sudo apt-get install -y gcc g++ build-essential "linux-headers-$(uname -r)" $ sudo apt-get install -y freeglut3 freeglut3-dev p7zip-full $ sudo apt-get install -y linux-image-extra-virtual
Then we simply choose the install method and download the respective driver from NVIDIA. For this time let’s stick to the installer file.
curl -O "http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run"
Next, the nouveau driver has to be removed.
$ echo <<EOF | sudo tee -a /etc/modprobe.d/blacklist-nouveau.conf lacklist nouveau blacklist lbm-nouveau options nouveau modeset=0 alias nouveau off alias lbm-nouveau off EOF $ echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf $ sudo update-initramfs -u
After this we have to restart the system for the changes to take effect.
After the reboot the next step is to unpack the runfile. The output will consist of three separate files. The first one to execute is the NVIDIA-Linux-x86_64…run file. This will install the kernel module. After that the module can be loaded.
$ sudo "./cuda_7.5.18_linux.run" -extract=./nvidia_installers $ sudo ./nvidia_installers/NVIDIA-Linux-x86_64-352.39.run --silent $ sudo modprobe nvidia
After the kernel module has been installed the system environment has to be adjusted. First the PATH should be adjusted. Also the LD_LIBRARY_PATH has to include the cuda libraries.
$ export PATH=/usr/local/cuda/bin:$PATH $ echo "$PATH" | sudo tee -a /etc/environment $ echo "/usr/local/cuda/lib64" | sudo tee -a /etc/ld.so.conf.d/cuda.conf $ echo "/usr/local/cuda/lib" | sudo tee -a /etc/ld.so.conf.d/cuda.conf $ sudo ldconfig
Once again, let’s reboot the system so the changes take effect.
After the final reboot quickly verify the kernel module is loaded.
$ lsmod | grep -q nvidia
If everything works, let’s proceed by downloading the cudaHashcat binary. Once it is downloaded we can unzip it and setup the access rights.
$ sudo curl https://hashcat.net/files/cudaHashcat-2.01.7z -O $ sudo 7za x -y -o /opt/ cudaHashcat-2.01.7z > /dev/null $ sudo chown -R ubuntu:ubuntu /opt/cudaHashcat-2.01/
So the next thing is to check that cudaHashcat is working as expected.
What better way to see the result, other than run a benchmark. This will give a true indicator of the gains over a traditional desktop GPU. The whole results are available on gist.
$ /opt/cudaHashcat-2.01/cudaHashcat64.bin --benchmark cudaHashcat v2.01 starting in benchmark-mode... Device #1: GRID K520, 4095MB, 797Mhz, 8MCU Hashtype: MD5 Workload: 1024 loops, 256 accel Speed.GPU.#1.: 2632.2 MH/s Hashtype: SHA1 Workload: 1024 loops, 256 accel Speed.GPU.#1.: 690.4 MH/s
My local machine with a AMD Radeon HD 7870 GHz Edition for a comparison. The graphics card encounters some problems with the temperature and aborts some tests. The full results are available on github.
$ sudo /opt/oclHashcat/oclHashcat64.bin --benchmark oclHashcat v2.01 starting in benchmark-mode... Device #1: Pitcairn, 1630MB, 1050Mhz, 20MCU Hashtype: MD5 Workload: 1024 loops, 256 accel Speed.GPU.#1.: 4477.7 MH/s Hashtype: SHA1 Workload: 1024 loops, 256 accel Speed.GPU.#1.: 1710.5 MH/s
The results are quite obvious. My old AMD is still better at cracking than a g2.2xlarge instance. Maybe scaling up to a g2.8xlarge give more promising results.
The speed is not really magnificent. Maybe running Hashcat in a spot instance for a cheap price is the right way to go. The problem here is that the instance might terminate because of price fluctuations. To solve this probelm S3 cloud storage would be a great place to store the hashes and results. During testing the connection to S3 via awscli did not work out. Maybe another tool can do the job. Or another storage mechanism is suitable to persist the relevant data.
In order to speed up the whole process a bash script is the obvious solution. The whole file can be found in this gist.
#basic setup, disable nouveau $ ./cuda-setup.sh -i #install cuda $ ./cuda-setup.sh -r
- CUDA 7.5 Downloads | NVIDIA Developer
- Getting Started Linux :: CUDA Toolkit Documentation
- CUDA 6.5 on AWS GPU Instance Running Ubuntu 14.04
- Amazon EC2: GPU Cluster