基于注视兴趣区域聚类和转移的群体扫视路径生成

doi:10.11772/j.issn.1001-9081.2020061147

计算机应用 ›› 2021, Vol. 41 ›› Issue (1): 150-156.DOI: 10.11772/j.issn.1001-9081.2020061147

所属专题：第八届中国数据挖掘会议(CCDM 2020)

• 第八届中国数据挖掘会议(CCDM 2020) • 上一篇下一篇

基于注视兴趣区域聚类和转移的群体扫视路径生成

刘楠博, 肖芬, 张文雷, 李旺鑫, 翁尊

智能计算与信息处理教育部重点实验室(湘潭大学), 湖南湘潭 411105

收稿日期:2020-05-31 修回日期:2020-08-13 出版日期:2021-01-10 发布日期:2020-11-12
通讯作者: 肖芬
作者简介:刘楠博(1994-),男,河南洛阳人,硕士研究生,CCF会员,主要研究方向:眼动数据挖掘、显著性检测、深度神经网络;肖芬(1981-),女,湖南益阳人,教授,博士,CCF会员,主要研究方向:计算机视觉、人工智能、深度学习、智能计算;张文雷(1996-),男,湖南常德人,硕士研究生,CCF会员,主要研究方向:视频数据处理、显著性检测、深度神经网络;李旺鑫(1994-),男,山西太原人,硕士研究生,CCF会员,主要研究方向:图像去模糊、显著性检测、深度神经网络;翁尊(1996-),女,湖南岳阳人,硕士研究生,CCF会员,主要研究方向:图像分割、显著目标检测、深度神经网络。
基金资助:
湖南省自然科学基金资助项目（2018JJ2405）；湖南省教育厅创新平台开放基金资助项目（18K034）。

Group scanpath generation based on fixation regions of interest clustering and transferring

LIU Nanbo, XIAO Fen, ZHANG Wenlei, LI Wangxin, WENG Zun

Key Laboratory of Intelligent Computing and Information Processing, Ministry of Education(Xiangtan University), Xiangtan Hunan 411105, China

Received:2020-05-31 Revised:2020-08-13 Online:2021-01-10 Published:2020-11-12
Supported by:
This work is partially supported by the Natural Science Foundation of Hunan Province (2018JJ2405), the Innovation Platform Open Fund of Hunan Provincial Education Department (18K034).

摘要/Abstract

摘要： 为解决自然场景下群体观察者扫视路径数据冗余繁乱、缺乏表征的问题，通过挖掘个体路径的潜在特性，提出了一种基于注视兴趣区域（ROI）时空聚类和转移的群体扫视路径生成方法。首先，分析同一刺激样本下多名观察者的扫视路径，利用亲和力传播聚类算法来聚类注视点以生成多个注视兴趣区域；其次，统计分析兴趣区域的观察者数量、注视频率以及注视时长等与注视强度相关的信息并筛选兴趣区域；然后，通过定义兴趣区域中的注视行为提取不同类型的兴趣子区域；最后，提出了基于注视优先度的兴趣区域和兴趣子区域转移模式，从而生成自然场景下的群体扫视路径。在MIT1003和OSIE公共数据集上进行群体扫视路径生成实验，结果表明，与目前先进的eMine、扫视路径趋势分析（STA）、序列模式挖掘算法（SPAM）、基于候选约束的动态时间规整质心平均方法（CDBA）和Heuristic方法相比，所提方法生成的群体扫视路径获得了较高的整体相似度，ScanMatch （w/o duration）分别可达0.426和0.467，ScanMatch（w/duration）分别可达0.404和0.439。可见该所生成的扫视路径与真实扫视路径的整体相似度较高，具有一定表征作用。

关键词: 自然场景, 扫视路径, 群体扫视路径, 注视兴趣区域, 注视行为

Abstract: For redundancy chaos, and the lack of representation of group observers' scanpath data in natural scenes, by mining the potential characteristics of individual scanpaths, a method for group scanpath generation based on fixation Regions of Interest (ROI) spatial temporal clustering and transferring was proposed. Firstly, multiple observers' scanpaths under the same stimulus sample were analyzed, and multiple fixation regions of interest were generated by utilizing affinity propagation clustering algorithm to cluster the fixation points. Then, the statistics and analysis of the information related to fixation intensity such as the number of observers, fixation frequency and lasting time were carried out and the regions of interest were filtered. Afterwards, the subregions of interest with different types were extracted via defining fixation behaviors in the regions of interest. Finally, the transformation mode of regions and subregions of interest was proposed on the basis of fixation priority, so as to generate the group scanpath in natural scenes. The group scanpath generation experiments were conducted on two public datasets MIT1003 and OSIE. The results show that compared with the state-of-the-art methods, such as eMine, Scanpath Trend Analysis (STA), Sequential Pattern Mining Algorithm (SPAM), Candidate-constrained Dynamic time warping Barycenter Averaging method (CDBA) and Heuristic, the proposed method has the group scanpath generated of higher entirety similarity indexes with ScanMatch (w/o duration) reached 0.426 and 0.467 respectively, and ScanMatch (w/duration) reached 0.404 and 0.439 respectively. It can be seen that the scanpath generated by the proposed method has high overall similarity to the real scanpath, and has a certain function of representation.

Key words: natural scene, scanpath, group scanpath, fixation Region Of Interest (ROI), fixation behavior

中图分类号:

TP391

刘楠博, 肖芬, 张文雷, 李旺鑫, 翁尊. 基于注视兴趣区域聚类和转移的群体扫视路径生成[J]. 计算机应用, 2021, 41(1): 150-156.

LIU Nanbo, XIAO Fen, ZHANG Wenlei, LI Wangxin, WENG Zun. Group scanpath generation based on fixation regions of interest clustering and transferring[J]. Journal of Computer Applications, 2021, 41(1): 150-156.

参考文献

[1] GRILL-SPECTOR K,MALACH R. The human visual cortex[J]. Annual Review of Neuroscience,2004,27:649-677.
[2] 龚雪. 基于自适应视觉注意机制的图像描述算法研究[D]. 湘潭:湘潭大学,2019:12-16.(GONG X. Image captioning based on adaptive visual attention mechanism[D]. Xiangtan:Xiangtan University,2019:12-16.)
[3] 袁野, 和晓歌, 朱定坤, 等. 视觉图像显著性检测综述[J]. 计算机科学,2020,47(7):84-91.(YUAN Y,HE X G,ZHU D K,et al. Survey of visual image saliency detection[J]. Computer Science,2020,47(7):84-91.)
[4] ITTI L,KOCH C,NIEBUR E,et al. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[5] ZHANG J,SCLAROFF S. Saliency detection:a Boolean map approach[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2013:153-160.
[6] CORNIA M,BARALDI L,SERRA G,et al. Predicting human eye fixations via an LSTM-based saliency attentive model[J]. IEEE Transactions on Image Processing,2018,27(10):5142-5154.
[7] LEVY-SCHOEN A. Book review:eye movements and vision:A. L. Yarbus(translated by L. A. Riggs). Plenum Press,New York, 1967 XI+222 pp. $17, 50[J]. Neuropsychologia,1968,6(4):389-390.
[8] ZHOU W,WANG A,SHU H,et al. Word segmentation by alternating colors facilitates eye guidance in Chinese reading[J]. Memory and Cognition,2018,46(5):729-740.
[9] ERASLAN S,YESILADA Y,HARPER S. Identifying patterns in eyetracking scanpaths in terms of visual elements of web pages[C]//Proceedings of the 14th International Conference on Web Engineering,LNCS 8541. Cham:Springer,2014:163-180.
[10] ERASLAN S,YESILADA Y,HARPER S. Eye tracking scanpath analysis techniques on web pages:a survey, evaluation and comparison[J]. Journal of Eye Movement Research,2016,9(1):1-19.
[11] ERASLAN S, YESILADA Y, HARPER S. Scanpath trend analysis on web pages:clustering eye tracking scanpaths[J]. ACM Transactions on the Web,2016,10(4):No. 20.
[12] ERASLAN S,YESILADA Y,HARPER S. Trends in eye tracking scanpaths:segmentation effect?[C]//Proceedings of the 27th ACM Conference on Hypertext and Social Media. New York:ACM,2016:15-25.
[13] ERASLAN S, YESILADA Y, HARPER S. Less users more confidence:how AOIs don't affect scanpath trend analysis[J]. Journal of Eye Movement Research,2017,10(4):1-18.
[14] ERASLAN S,YESILADA Y,HARPER S. Engineering webbased interactive systems:trend analysis in eye tracking scanpaths with a tolerance[C]//Proceedings of the 2017 ACM SIGCHI Symposium on Engineering Interactive Computing Systems. New York:ACM,2017:3-8.
[15] LI A,ZHANG Y,CHEN Z. Scanpath mining of eye movement trajectories for visual attention analysis[C]//Proceedings of the 2017 IEEE International Conference on Multimedia and Expo. Piscataway:IEEE,2017:535-540.
[16] LI A,CHEN Z. Representative scanpath identification for group viewing pattern analysis[J]. Journal of Eye Movement Research, 2018,11(6):1-17.
[17] WEST J M,HAAKE A R,ROZANSKI E P,et al. eyePatterns:software for identifying patterns and similarities across fixation sequences[C]//Proceedings of the 2006 ACM Symposium on Eye Tracking Research and Applications. New York:ACM,2006:149-154.
[18] HEMBROOKE H, FEUSNER M, GAY G. Averaging scan patterns and what they can tell us[C]//Proceedings of the 2006 ACM Symposium on Eye Tracking Research and Applications. New York:ACM,2006:41-41.
[19] GOLDBERG J H, HELFMAN J I. Scanpath clustering and aggregation[C]//Proceedings of the 2010 ACM Symposium on Eye Tracking Research and Applications. New York:ACM,2010:227-234.
[20] HEJMADY P,NARAYANAN N H. Visual attention patterns during program debugging with an IDE[C]//Proceedings of the 2012 ACM Symposium on Eye Tracking Research and Applications. New York:ACM,2012:197-200.
[21] TSANG H Y,TORY M,SWINDELLS C. eSeeTrack-visualizing sequential fixation patterns[J]. IEEE Transactions on Visualization and Computer Graphics,2010,16(6):953-962.
[22] CHUK T, CHAN A B, HSIAO J H. Understanding eye movements in face recognition using hidden Markov models[J]. Journal of Vision,2014,14(11):No. 8.
[23] KANG Z,LANDRY S J. An eye movement analysis algorithm for a multielement target tracking task:maximum transition-based agglomerative hierarchical clustering[J]. IEEE Transactions on Human-Machine Systems,2015,45(1):13-24.
[24] FREY B J,DUECK D. Clustering by passing messages between data points[J]. Science,2007,315(5814):972-976.
[25] JUDD T,EHINGER K,DURAND F,et al. Learning to predict where humans look[C]//Proceedings of the IEEE 12th International Conference on Computer Vision. Piscataway:IEEE, 2009:2106-2113.
[26] XU J,JIANG M,WANG S,et al. Predicting human gaze beyond pixels[J]. Journal of Vision,2014,14(1):No. 28.
[27] ANDERSON N C,ANDERSON F,KINGSTONE A,et al. A comparison of scanpath comparison methods[J]. Behavior Research Methods,2015,47(4):1377-1392.
[28] JARODZKA H,HOLMQVIST K,NYSTRÖM M. A vector-based, multidimensional scanpath similarity measure[C]//Proceedings of the 2010 ACM Symposium on Eye-Tracking Research & Applications. New York:ACM,2010:211-218.
[29] CRISTINO F,MATHÔT S,THEEUWES J,et al. ScanMatch:a novel method for comparing fixation sequences[J]. Behavior Research Methods,2010,42(3):692-700.
[30] DIJKSTRA E W. A note on two problems in connexion with graphs[J]. Numerische Mathematik,1959,1(1),269-271.
[31] NEEDLEMAN S B,WUNSCH C D. A general method applicable to the search for similarities in the amino acid sequence of two proteins[J]. Journal of Molecular Biology, 1970, 48(3):443-453.

基于注视兴趣区域聚类和转移的群体扫视路径生成

Group scanpath generation based on fixation regions of interest clustering and transferring

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

[1]	邱津怡, 罗俊, 李秀, 贾伟, 倪福川, 冯慧. 基于卷积神经网络的多尺度葡萄图像识别方法[J]. 计算机应用, 2019, 39(10): 2930-2936.
[2]	马允, 王晓东, 章联军. 基于空域自然场景统计的无参考立体图像质量评价模型[J]. 计算机应用, 2016, 36(3): 783-788.
[3]	田金沙, 韩永国, 吴亚东, 赵小乐, 张红英. 基于尺度不变性的无参考图像质量评价[J]. 计算机应用, 2016, 36(3): 789-794.