Since the contradiction between edge-preserving in homogeneous regions and structure-preserving in the boundary region of an image, a new algorithm combined with cross-based framework and guided filter was proposed. The main idea of the algorithm was adding an adjust offset in guided filter to ensure remaining the edge structure. Usually the fixed size window was used as neighborhood filtering, while the new algorithm employed cross-based framework, which chose a threshold in grayscale similarity. Taking the advantage of stereo matching, the adaptive filtering blocks whose sizes and shapes can adjust automatically were generalized. The adjust offset was proportional to the threshold, which was more robust than a hard threshold. In the simulation experiments of processing international standard sequence, the blocks were generalized by cross-based framework efficiently and effectively, homogeneous regions were smoothed well. The added offset outperforms many other algorithms in terms of sharpness enhancement. Compared to the guided filter, the value of Peak Signal-to-Noise Ratio (PSNR) of the proposed method has been improved by about 2 dB. The test results of real natural picture show that the proposed algorithm has a good future in practical application.