In order to remove the effect of weather in degraded image, a fast haze removal algorithm for single image based on human visual characteristics was proposed. According to the luminance distribution of the hazy image and the human visual characteristics, the proposed method first applied luminance component to estimate coarse transmission map, then used a linear spatial filter to refine the transmission map and obtained the dehazed image by the atmospheric scattering model. Finally a new image enhancement fitting function was applied to enhance the luminance component of the dehazed image to make it more natural and clear. The experimental results show that the proposed algorithm effectively removes haze and is better than the existing algorithms in terms of contrast, information entropy and computing time.