Deep Learning for the masses

Deep Learning for the masses

Andrew Ng’s group has just published a paper that shows how to build giant neural networks on low-cost GPUs. Excerpt:

In this paper, we present technical details and results from our own system based on Commodity Off -The-Shelf High Performance Computing (COTS HPC) technology: a cluster of GPU servers with In finiband interconnects and MPI. Our system is able to train 1 billion parameter networks on just 3 machines in a couple of days, and we show that it can scale to networks with over 11 billion parameters using just 16 machines. As this infrastructure is much more easily marshaled by others, the approach enables much wider-spread research with extremely large neural networks.

More information can be found in the full article on wired.

I am just wondering who is going to need this much computational power, except Google and some big companies. For me, even a single GTX 580 would be already overwhelmed… if I ever had one.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s