Aiming at the problems of information distortion, poor objectivity of attribute weights, and high time complexity in hesitant fuzzy clustering analysis, an Agglomerative Hierarchical Clustering algorithm based on Hesitant Fuzzy set (AHCHF) was proposed. Firstly, the average value of hesitancy fuzzy elements was used to expand the data object with small hesitation. Secondly, the weights of data object before and after expansion were calculated by using the original information entropy and internal maximum difference, and the comprehensive attribute weight was determined according to the minimum discrimination information between the two weight vectors. Finally, with the goal of making the sum of weighted distances smaller, a center point construction method with constant hesitation was given. Experimental results on specific examples and synthetic datasets show that compared with the classic Hesitant Fuzzy Hierarchical Clustering algorithm (HFHC) and the recent Fuzzy Hierarchical Clustering Algorithm (FHCA), the proposed AHCHF increases the mean Silhouette Coefficient (SC) by 23.99% and 9.28% respectively, and shortens the running time by 27.18% and 6.40% averagely and respectively, proving that the proposed algorithm can effectively solve the problems of information distortion and poor objectivity of attribute weights, and improve the clustering effect and performance well.