Existing infrared and visible image fusion models ignore illumination factors in fusion process and use conventional fusion strategies, leading to the fusion results with the loss of detail information and inconspicuous salient information. To solve these problems, a deep learning model for infrared and visible image fusion based on illumination weight allocation and attention was proposed. Firstly, an Illumination Weight Allocation Network (IWA-Net) was designed to estimate the illumination distribution and calculate illumination weights. Secondly, a CM-L1-norm fusion strategy was introduced to enhance the dependency between pixels and achieve smooth processing of salient features. Finally, a decoding network composed of fully convolutional layers was employed to reconstruct fused images. The results of the fusion experiments on publicly available datasets show that the proposed model outperforms the contrastive models, with improvements observed in all six selected evaluation metrics; specifically, the Spatial Frequency (SF) and Mutual Information (MI) metrics increase by 45% and 41% in average, respectively. The proposed model effectively reduces edge blurring and enhances clarity and contrast of the fused images. The fusion results of the proposed model exhibits superior performance in both subjective and objective aspects compared to other contrastive models.