Where and why does deep learning shine?

The main purported benefits:

(1) Don't need to hand engineer features for non-linear learning problems (save time and scalable to the future, since hand engineering is seen by some as a short-term band-aid)

(2) The learnt features are sometimes better than the best hand-engineered features, and can be so complex (computer vision - e.g. face-like features) that it would take way too much human time to engineer.

(3) Can use unlabeled data to pre-train the network. Suppose we have 1000000 unlabeled images and 1000 labeled images. We can now drastically improve a supervised learning algorithm by pre-training on the 1000000 unlabeled images with deep learning. In addition, in some domains we have so much unlabeled data but labeled data is hard to find. An algorithm that can use this unlabeled data to improve classification is valuable.

(4) Empirically, smashed many benchmarks that were only seeing incremental improvements until the introduction of deep learning methods.

(5) Same algorithm works in multiple areas with raw (perhaps with minor pre-processing) inputs.

(6) Keeps improving as more data is fed to the network (assuming stationary distributions etc).