How to Select an Assessment Instrument


Establish your goals:
Do you want to use the assessment instrument for?

  • Needs assessment and analysis
  • Pre and post measurement of program impact
  • Individual development
  • Team development
  • Coaching
  • Selection
  • Re-entry
  • Career advising/academic advising

Measurement is the systematic application of pre-established rules or standards for assigning numbers or scores to the attributes or traits of something of interest.

Measurement supports intercultural work because we want to have data, which means we can do many sophisticated things not otherwise possible—

  • Clarify needs
  • Structure program design
  • Measure program impact       
  • Support selection decisions
  • Prepare cultural competence initiatives more precisely
  • Promote individual development

Questions about the quality of the assessment instrument:

  1. Is it reliable—free from random error?
  2. Is it valid—free from systematic, built in error?
    1. Content validity—Does it measure what it says it will measure?
    2. Predictive validity—Can it help you learn about things you are interested in?
    3. Convergent validity—Is it consistent with other similar measures?
    4. Face validity—Does it look to the participants as if it measures what it says it will measure?
    5. Differential validity—Is it free from cultural bias?
  3. Is it transparent? Can participants manipulate their scores through perceived social desirability?
  4. How were items developed? Are they based on a well-recognized theory or model?

Questions about the logistics of using the instrument:

  1. Do I need to be certified/qualified to use the instrument?
  2. What knowledge, skills, and attitudes do I need to master to use the inventory effectively?
  3. What is the amount of time for participants to take the inventory?
  4. What is the cost for the inventory per person, and the cost of materials and training for certification?
  5. Is it available in the appropriate language for my group?
  6. Is the translation professional and accurate?
  7. Is it available in the preferred language online as well as in paper and pencil formats?
  8. Is the instrument self-scoring or are there additional fees for processing the results?
  9. Are there 360 degree versions?
  10. What is the track record of this inventory with my audience?
  11. Are there research studies reporting results from using this instrument with my typical audience?
  12. Are there publications suggesting approaches for using this instrument?
  13. Does the inventory include sufficient information for participants to understand their profile, and to develop goals?
  14. Is the inventory accompanied by support materials for the educator that include data on the inventory development, strategies for communicating inventory results, and activities for enhancing learning based on results?
  15.  To use the inventory ethically:
    1. Must I conduct coaching interviews?
    2. Is it best used to gather aggregate data?
    3. Is it effective for pre and post testing?
    4. Is it effective for individual feedback without interviews?
  16. Is it possible to talk to other users about the effectiveness of the instrument?
  17. Is information available on the going consulting fees for conducting assessments using this instrument?

Janet M. Bennet, Ph.D. and Michael Stevens, Ph.D. 2011