Downsampling operation will make images lose high-frequency forensic traces and detail information, increasing the difficulty of image forensics. Existing deep learning-based image forensic networks cannot effectively detect the images tampered by downsampling operation, making the enhancement of robustness in downsampling image forensics methods becomes a bottleneck in image forensics. To solve these problems, a downsampling image forensic network named HirrNet (Hierarchical Ringed Residual U-Net) was proposed, which consists of an image recovery module and a tampering detection module. In the image recovery module, the idea of Hierarchical Conditional Flow (HCF) was used to reduce the loss of high-frequency information by recovering forensic traces and details in tampered images, so as to improve the performance of tampering detection. In the tampering detection module, an end-to-end image segmentation network RRU-Net (Ringed Residual U-Net) was employed for tampering detection. Besides, by combining the Spatial and Channel Squeeze & Excitation (SCSE) mechanism, the extraction of tampering-related features in the downsampled image was effectively enhanced. Experimental results show that HirrNet outperforms comparative networks in terms of Area Under the receiver operating characteristic Curve (AUC), F1-score and Intersection and Union (IoU) on DSO, Columbia, CASIA, and NIST16 datasets. Compared with the comparative methods, HirrNet improves the AUC by 25 and 30 percentage points on average for the tampered images scaled down to 1/2 and 1/4 of their original sizes on CASIA dataset. These findings indicate that HirrNet can effectively resolve the poor robustness of existing downsampled image forensic methods.