Surface defects of bearing have a significant impact on performance and stability of electromechanical equipment. Aiming at the issues of low recognition accuracy of small targets and low detection speed in current surface defect detection process for bearings, a tiny defect detection algorithm of bearing surface based on RT-DETR (Real-Time DEtection TRansformer) — FECS-DETR (Faster Expand and Cross hierarchical-scaled feature Screening DETR) algorithm was proposed. Firstly, a lightweight FasterNet-T1 was employed to reconstruct the backbone network of RT-DETR for reducing computational overhead. Secondly, an Attention-embedded Expand Residual Fusion (AERF) module was designed for deep feature extraction, thereby enhancing the description capability of small-scale abstract features. Thirdly, a Cascaded Group Attention (CGA) was applied to further reduce computational redundancy and improve operational efficiency of the model. Fourthly, a Cross hierarchical-scaled Information Screening Feature Pyramid Network (CIS-FPN) was proposed to address the issue of information loss during feature fusion and enhance feature integration capability. Finally, a joint regression loss optimization strategy combining Normalized Wasserstein Distance (NWD) and improved Inner-MPDIoU was employed to accelerate model’s convergence and improve model accuracy for small-scale targets. Experimental results show that on the bearing surface tiny defect dataset, compared with the original RT-DETR algorithm, FECS-DETR algorithm has the mean Average Precision (mAP) improved by 2.5 percentage points, the computation complexity reduced by 28.8%, and the detection speed increased by 20.8%. It can be seen that the proposed algorithm achieves a balance between accuracy and real-time performance, and satisfies the requirements for detection of bearing surface tiny defects in industrial environment.