Virginia Tech® home

Calibrating Trust in Human-Machine Interactions with Algorithm Transparency

Researchers seek a deeper understanding of trust in secure human-machine interactions by investigating the impact of algorithm transparency on human trust and final-task performance.

Funded by: CCI Northern Virginia Node

Project Investigators

Principal Investigator (PI): Ziyu Yao, assistant professor, George Mason University Department of Computer Science  

Co-PI: Tyler H. Shaw, associate professor, George Mason University Department of Psychology

Rationale and Background

Artificial intelligence (AI) systems assist people in daily tasks and boost productivity in the workplace. Most interactions between humans and machines are single-instance, as humans send a task request, and the machines complete the task. 

However, such one-shot tasks with no follow-up (humans validating the machine decisions), could expose individuals to insecure situations, because even state-of-the-art AI systems can make mistakes.

Properly calibrating human trust becomes crucial for secure human-machine interactions. 

Algorithm transparency is a promising solution to trust calibration. This enables humans to spot potential issues with the AI systems, dampening over-trust, which can lead to an inability to respond to errors or malfunctions. The algorithm explanation also preserves human trust if errors occur, mitigating under-trust, which causes unbalanced workloads.

Methodology

Researchers will:

  • Translate natural language questions to structured query language (SQL) queries, enabling users to seek information from a knowledge database without knowing SQL programming.
  • Investigate how to measure trust in interactive text-to-SQL semantic parsing.
  • Understand how different transparency levels of explanation affect human trust and task performance.
  • Showcase how calibrating human trust can make a state-of-the-art semantic parser more effective and secure in practice. 

Projected Outcomes

The project will:

  • Push state-of-the-art AI systems toward more secure, resilient, and harmonious interactions with humans. 
  • Allow people to query a database using natural language. 
  • Have the potential to benefit local industry and government and be transformed into commercial products.
  • Include submission of a paper reporting results to a top-tier AI conference.