Combining UAV image data with deep learning technology can realize remote non-contact detection of cracks on the surface of hydraulic buildings, providing a guarantee for maintaining the normal operation of the structure. This paper introduces the popular deep learning methods in recent years, and compares the applications and characteristics of three types of algorithms based on convolutional neural networks for classification, detection and segmentation in crack detection. Combined with an embankment crack detection example, the process and challenges of using UAV images for crack detection are explained. Finally, the difficulties and outlook for crack detection research in hydraulic structures are given in the paper.