License plate recognition method is an important part of modern intelligent traffic management system and has been widely used in many fields. However, in practical application scenarios, there are multi-row license plates, and the traditional methods are not flexible enough to deal with multi-row license plates, and cannot achieve high-precision end-to-end recognition. Therefore, an end-to-end recognition method based on 2D Space Traversal Long Short-Term Memory (2DST-LSTM) network was proposed to recognize single-row and double-row license plates. The proposed method abandons the previous image segmentation step, and carries out license plate recognition in an end-to-end way, which makes the license plate recognition more efficient and accurate. 2DST-LSTM can improve the recognition effect of license plates, especially double-row license plates, in complex environment. Experimental results on multiple datasets show that the proposed method achieves the highest recognition rate up to 98.6% for double-row license plates, which verified its effectiveness.