Concerning the efficiency and security issues of automatic door, an safety and energy saving detection technology of automatic door based on Omni-Directional Vision Sensor (ODVS) was proposed. Firstly, on-site 360° panorama image around automatic door was collected timely by ODVS and preprocessed according to the detection requirements. Secondly, moving target was detected and tracked through Motion History or Energy Images (MHoEI) algorithm. Then, the behavior of pedestrian was analyzed according to the direction of motion and spatial position of the foreground object. Finally, in order to make automatic door secure, energy-saving and comfortable, the automatic door was controlled to open or close according to the behavior and state of pedestrian. At the same time, the number of people passing through the automatic door can also be worked out accurately. This technology can be directly used in intelligent monitoring and business survey. The experimental results indicate that the detection technology of automatic door can recognize the behavior of pedestrian around the automatic door, avoid the security risks of automatic door and improve the accuracy of people counting.