The hot spot effect can cause damage to PV modules and seriously affect the safe and stable operation of PV systems. Fast and accurate detection of hot spot faults is of great importance to extend the life of PV modules and reduce power generation costs. In this paper, we propose a conditionally generated adversarial network-based hot spot detection method for PV modules, which can achieve accurate hot spot detection with small samples. Specifically, for the problem of sparse infrared image data of PV module hot spot, the hot spot dataset is expanded by a conditional generation adversarial network. The new dataset is segmented by a semantic segmentation network, which improves the problem of insufficient training of model parameters caused by the small amount of data in the original model. Through experimental validation, the proposed method optimizes the image segmentation effect and achieves accurate detection of hot spots of PV modules compared with the original model.