Addressing the poor performance of population size improvement methods in the existing Differential Evolution (DE) algorithms when dealing with decreased population diversity and local optimum challenges, a dual-Archive Population Size adaptive Differential Evolution algorithm (APDE) was proposed on the basis of dual-Archive Population Size Adaptive method (APSA). Firstly, two archives were constructed to record individuals that had been discarded in previous evolutions and experimental individuals respectively. Then, diversity changes were measured according to the variations in the population distribution state. And when population diversity decreased, the individuals from the archives were selected and added to the population to enhance the population diversity and the ability to escape from the local optimum. Finally, an improved DE algorithm based on APSA method, APDE, was proposed.Results of extensive tests on CEC2017 test set and Lennard-Jones potential problem show that APDE algorithm outperforms five other DE algorithms in the average ranking based on Friedman test on 30 benchmark functions, and significant improvements are obtained on at least 20% of these functions. At the same time, APDE algorithm also achieves the best performance in solving the minimization of potential energy.