The current service resources in Internet of Services （IoS） show a trend of refinement and specialization. Services with single function cannot meet the complex and changeable requirements of users. Service integrating and scheduling methods have become hot spots in the field of service computing. However， most existing service integrating and scheduling methods only consider the satisfaction of user requirements and do not consider the sustainability of the IoS ecosystem. In response to the above problems， a service integration method based on adaptive multi?objective reinforcement learning was proposed. In this method， a multi?objective optimization strategy was introduced into the framework of Asynchronous Advantage Actor?Critic （A3C） algorithm， so as to ensure the healthy development of the IoS ecosystem while satisfying user needs. The integrated weight of the multi?objective value was able to adjusted dynamically according to the regret value， which improved the imbalance of sub?objective values in multi?objective reinforcement learning. The service integration verification was carried out in a real large?scale service environment. Experimental results show that the proposed method is faster than traditional machine learning methods in large?scale service environment， and has a more balanced solution quality of each objective compared with Reinforcement Learning （RL） with fixed weights.