Abstract: Visualization reconstruction technology aims to transform graphics into data forms that can be parsed and operated by machines, providing the necessary basic information for large-scale analysis, reuse and retrieval of visualization. However, existing research on reconstruction methods obviously focuses on the recovery of visual information, ignoring the key role of interaction information in data analysis and understanding. To address the above problems, a visualization interaction information reconstruction method for machine understanding was proposed. First, interactions were formally defined to divide the visual elements into different visual groups, and use automated tools to extract the interactive information of the visualization graphics. Then, they were decoupled from visual elements, and transformed into independent experimental variables to build an interaction entity library. Next, a standardized declarative language was formulated to enable the querying of interaction information. Finally, migration rules were designed to achieve migration adaptation of interactions between different visualizations based on visual elements matching and adaptive adjustment mechanisms.The experimental cases were focused on downstream tasks such as visual question answering, querying, and migration for machine understanding.The results show that adding interactive information can enable machines to understand the semantics of visual interaction, thereby expanding the application scope of the above tasks. The experimental results prove that the framework can achieve structural integrity of the reconstructed visual graphics by integrating dynamic interaction information.

Key words: machine understanding, interaction, reconstruction of interaction information, visualization reconstruction, data analysis

摘要： 可视化重构技术旨在将图形转换为机器能够解析和操作的数据形式，为可视化的大规模分析、重用及检索等提供了必备的基础信息。然而现有的重构方法明显侧重于视觉信息的恢复，忽视了交互信息在数据分析和理解中扮演的关键作用。针对上述问题，提出一种面向机器理解的可视化交互信息重构方法。首先对交互进行形式化定义，将可视元素划分为不同的视觉组集合，采用自动化工具提取可视化图形的交互信息；然后解耦交互与可视元素的关联，将其分离为独立的实验变量，构建交互实体库；接下来制定规范的声明式语言，实现交互信息的查询；最后，设计迁移规则，基于可视元素匹配与自适应调整机制，实现交互在不同可视化间的迁移适配。实验案例针对可视化问答、查询和迁移等面向机器理解的下游任务，结果显示增加交互信息能够使机器理解可视化交互语义，从而拓展上述任务的应用范围。实验结果证明该框架能够使重构后的可视化图形因为融合动态交互信息而达成结构完整性效果。

关键词: 机器理解, 交互, 交互信息重构, 可视化重构, 数据分析