One major, major advantage that medical imaging has for deep learning is the similarity of each data point, especially the 'background data.' For instance, human brains typically look very similar across individuals (up to scanning parameter differences), except in the abnormalities - which are often precisely what you want to highlight.
As an example, I recently trained a neural neural network to perform a useful task for our lab using 3 (!) hand-labeled brains.
It's insane that you were able to get reasonable results with such a tiny dataset.
I am learning machine learning right now and I find working with datasets with fewer than 100 examples to be quite difficult.
It seems counter intuitive when you first think about it but having way more data actually makes the task of fitting the model much easier as there is granularity that can be used to get feedback on adjustments to the structure of the model.
It was an image segmentation task, and the features were similar across data sets. The other thing that made it work well was heavy use of data augmentation that captured ways in which different data points could reasonably differ.
There was a really cool medical imaging paper recently that literally just labeled several 2D slices in a 3D dataset consisting of 3 images and performed a reasonable segmentation:
As an example, I recently trained a neural neural network to perform a useful task for our lab using 3 (!) hand-labeled brains.