It is crucial in solving Constrained Multi-objective Optimization Problems （CMOPs） to efficiently balance the relationship between diversity， convergence and feasibility. However， the emergence of complex constraints poses a greater challenge in solving CMOPs. Therefore， a Two-stage search-based constrained Evolutionary Multitasking optimization Algorithm （TEMA） was proposed to achieve the balance between diversity， convergence and feasibility by completing the two cooperatively evolutionary tasks together. At first， the whole evolutionary process was divided into two stages， exploration stage and utilization stage， which were dedicated to enhance the extensive exploration capability and efficient search capability of the algorithm in the target space， respectively. Second， a dynamic constraint handling strategy was designed to balance the proportions of the feasible solutions in the population to enhance the exploration capability of the algorithm in the feasible region. Then， a backward search strategy was proposed to utilize the information contained in the unconstrained Pareto front to guide the algorithm to converge quickly to the constrained Pareto front. Finally， comparative experiments were performed on 23 problems in two benchmark test suites to verify the performance of the proposed algorithm. Experimental results indicate that the proposed algorithm achieves optimal IGD （Inverted Generational Distance） and HV （HyperVolume） values on 14 and 13 test problems， respectively， which reflects its significant advantages.