Understanding How Humans Inject Knowledge into Machine Learning Workflows through Visual Analytics

A Comprehensive Survey of VIS4ML Workflows

Yiwen Xing; Philip Beaucamp; Joyraj Chakraborty; Afrah Farea; Yuanzhe Jin; Saiful Khan; Gennady Andrienko; Natalia Andrienko; Min Chen.

Visual Analytics Workflows & Pathways

Figure 1 illustrates the overall machine learning workflow considered in this survey, spanning development, deployment, and third-party evaluation stages. The development stage includes data preparation, learning configuration, model training, and evaluation, while the deployment stage focuses on model usage, followed by performance monitoring and external evaluation.

This framework highlights all possible pathways through which human knowledge can be injected into the ML workflow. In particular, visualization and visual analytics (VA) serve as the central interface through which humans interpret data, models, and results (e.g., what data, what plots, and what analytical tasks), enabling knowledge acquisition.

This acquired knowledge then leads to actions through interaction, such as modifying data, adjusting model configurations, or refining system behavior, which in turn influence different stages of the ML workflow. These interactions may occur during development, deployment, or third-party monitoring, forming a closed human-in-the-loop cycle.

Overall, this figure provides a unified view that connects visualization, human cognition, interaction, and actionable knowledge injection, forming the conceptual foundation for our coding scheme and subsequent analysis.

Main ML Workflow

Figure 1: Main ML Workflow. It shows all possible pathways of how human knowledge can be injected to different parts of a ML workflow.

Supplementary Materials

Detailed coded results, coding schemes, and additional data analysis results can be found in our repository:

Access the full data repository: ml-knowledge-inject-va/SpplementaryMaterials