Home > Evaluation Methods

Usability Evaluation Methods

Usability evaluation methods were first described in formal presentations in 1990 at the CHI (Computer Human Interaction) '90 conference where papers were published on heuristic evaluation (Nielsen and Molich, 1990) and cognitive walkthroughs (Lewis et al., 1990). 

Since that time, several methods have been developed to evaluate product usability.  These methods, which were originally developed in the area of computer usability, fall into two basic categories – evaluation of the product’s usability by the product’s developers and usability professionals (referred to as expert reviews) and evaluation through the end user of the product (referred to as user testing).  In the case of testing usability of aviation technical documentation, the user’s or maintenance technician’s unique skill set would be considered  expert as well as a user.  This is one of the major differences in development of user evaluation tools for the technical writer versus the product developer for a consumer market. 

The Evaluation Methods selected for use with aviation technical documentation are:

For User and Task Analyses:

For Expert Reviews:

For User Testing:

The Evaluation Toolbox is designed to give you a comprehensive overview of the various types of user evaluation methods to aid your understanding and use of each method. By clicking on the evaluation methods above, each will be summarized using a comprehensive framework to allow for your use and understanding the various techniques.  Forms to use during testing are located in the  Evaluation Tools section.  Also, you may find more information on these methods by referring to the books, articles and websites cited on the References list and Useful Resources pages.  

 

Aviation Technical Manual Development Stage

Interviews

Questionnaires

Heuristic Evaluation

Cognitive Walkthrough

Co-Discovery

Think-Aloud Protocol

User Performance Testing

Project Initiation

X

X

 

 

 

 

 

Procedure Development

 

 

X

X

 

 

 

Validation Testing

 

X

 

 

X

X

X

Procedure Revision

X

 

X

X

X

X

X

Validation Retest

 

 

 

 

X

X

X

 (Adapted from Pieratti, 1995)


Human Factors at NIAR | Human Factors at FAA | Human Factors Psychology at WSU

Human Factors Laboratory, National Institute for Aviation Research at Wichita State University. Research funded by the Federal Aviation Administration.  All rights reserved.
Revised: 11/05/04