In order to solve the problem that deep learning defect detection methods require many labeled samples for training and insulator defect samples are difficult to obtain, a few-shot insulator defect detection method based on transfer learning was proposed. Firstly, an Efficient Multi-scale Attention (EMA) mechanism was added to the backbone network to enhance the model’s ability to represent target features. Secondly, a hierarchical sampling-based Region Proposal Network (RPN) was constructed to select anchors in the feature pyramid uniformly, thereby improving the model’s ability to capture new class objects at different scales. Finally, the classification heads were decoupled, and the positive and negative samples were processed by the positive and negative heads, respectively, so that the model was able to adapt to the new class of objects more effectively. Experimental results show that compared with the baseline method TFA (Two-stage Fine-tuning Approach), on public dataset PASCAL VOC, the proposed method improves the mean Average Precision (mAP) (with IoU (Intersection over Union) of 0.5) of new class by 9.5 percentage points on average; on the insulator defect dataset, the proposed method has the mAP50 in detection tasks of 1-shot, 5-shot, 10-shot, 20-shot, and 30-shot increased by 15.8, 12.2, 17.4, 7.3 and 7.1 percentage points, respectively.