Think First - Putting the user back into AI Evaluation

Microsoft Design on YouTubeWatch on YouTube

SaveMark complete

Sign in to save videos and track what you’ve finished.

IntermediateProduct6 min

About this video

Three Microsoft AI UX researchers discuss how to evaluate AI outputs such as summaries and recommendations, arguing that automated criteria must be complemented by what users find useful. The talk describes a response-quality evaluation program that compares Copilot's answers with other AI products.

What you’ll learn

  • Compare AI outputs against user-valued criteria rather than only automated checks
  • Trace differences in response quality back to what users actually value
  • Put the user's voice into the evaluation of AI-generated results