Certification Evaluation Standards: Validity and Fairness

It is important to understand that all certification programs are not created equal. To ensure the value and reputation of the certification obtained, certification programs must incorporate assessment tools that were developed based on sound evaluation standards. These standards involve reliability, validity and fairness. Below, I provide an outline of 'validity' and 'fairness' as an evaluation standard.

Reliability, on its own, is insufficient as an evaluation standard. A tool may have high reliability, but low validity. For example, a laser instrument measuring a person’s height produces consistent results over numerous measurements. However, if this tool is used to determine a person’s ability to drive a boat, the tool will not be assessing the “right” things. This is why reliability needs to be informed by validity.

Validity refers to whether an assessment tool measures what it is intended to measure. In other words, it refers to the extent to which a decision based on the results of the assessment tool reflects the candidate’s true level of job performance or occupation-specific competence. Therefore, when building a quality certification program, we must also ensure validity of certification assessments.

Fairness is based on four principles: absence of bias; equitable treatment of candidates in the testing process; equality of testing outcomes for examinee subgroup (race, ethnicity, gender, disability, or other characteristics); and, equality of opportunity to learn the testing materials. It follows that fairness is applicable throughout the process, starting at the application through to the release of results.

In order to ensure fairness, it is important to review the questions on the assessment tool to ensure that they have no discriminatory language or content. The assessment must also provide sufficient opportunity for candidates to demonstrate their competence. The opportunity to do so is especially important when a certification program uses on-the-job observation or simulation. It is important to ensure that a lack of success on these assessments reflects a real deficit in competence and is not due to the lack of opportunity for a candidate to demonstrate his or her skills.

Equitable treatment does not necessarily imply equal treatment for all. In some cases, special accommodations for candidates with disabilities are needed. This can include extra time for examination completion for a person with dyslexia or providing a separate room for a person with ADHD. It must nevertheless be ensured that the accommodations provided are appropriate and reasonable given the nature of the occupation and that policies and procedures are in place to address such issues.

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.