Certification Evaluation Standards: Validity and Fairness

It is important to understand that all certification programs are not created equal. To ensure the value and reputation of the certification obtained, certification programs must incorporate assessment tools that were developed based on sound evaluation standards. These standards involve reliability, validity and fairness. Below, I provide an outline of 'validity' and 'fairness' as an evaluation standard.

Reliability, on its own, is insufficient as an evaluation standard. A tool may have high reliability, but low validity. For example, a laser instrument measuring a person’s height produces consistent results over numerous measurements. However, if this tool is used to determine a person’s ability to drive a boat, the tool will not be assessing the “right” things. This is why reliability needs to be informed by validity.

Validity refers to whether an assessment tool measures what it is intended to measure. In other words, it refers to the extent to which a decision based on the results of the assessment tool reflects the candidate’s true level of job performance or occupation-specific competence. Therefore, when building a quality certification program, we must also ensure validity of certification assessments.

Fairness is based on four principles: absence of bias; equitable treatment of candidates in the testing process; equality of testing outcomes for examinee subgroup (race, ethnicity, gender, disability, or other characteristics); and, equality of opportunity to learn the testing materials. It follows that fairness is applicable throughout the process, starting at the application through to the release of results.

In order to ensure fairness, it is important to review the questions on the assessment tool to ensure that they have no discriminatory language or content. The assessment must also provide sufficient opportunity for candidates to demonstrate their competence. The opportunity to do so is especially important when a certification program uses on-the-job observation or simulation. It is important to ensure that a lack of success on these assessments reflects a real deficit in competence and is not due to the lack of opportunity for a candidate to demonstrate his or her skills.

Equitable treatment does not necessarily imply equal treatment for all. In some cases, special accommodations for candidates with disabilities are needed. This can include extra time for examination completion for a person with dyslexia or providing a separate room for a person with ADHD. It must nevertheless be ensured that the accommodations provided are appropriate and reasonable given the nature of the occupation and that policies and procedures are in place to address such issues.

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.

What are Certification Evaluation Standards?

It is important to understand that all certification programs are not created equal. To ensure the value and reputation of the certification obtained, certification programs must incorporate assessment tools that were developed based on sound evaluation standards. These standards involve reliability, validity and fairness. Below, I provide an outline of 'reliability' as an evaluation standard.

Reliability refers to the consistency or repeatability of assessment results. Simply put, it is the ability of the tool to produce consistent results over time. In the context of certification, reliability refers to the likelihood of a candidate obtaining a similar result on the same or an equivalent assessment tool, such as a multiple-choice test, upon taking this test for the second time. When an assessment tool has low reliability, its scores have little meaning because there is no guarantee that the candidate will receive similar scores should the same test be taken again. Similarly, if a candidate takes two test versions that are said to be equivalent, he/she should either pass both tests or fail both tests. Furthermore, if two candidates have the same level of knowledge, skills and abilities, both of the candidates should either pass or fail. Hence, in order to ensure that a quality certification program is built, we must ensure that the assessments that are a part of the certification program are reliable.

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.

Setting the Stage for Certification and Assessment

To reap its full rewards, a certification program must employ the assessment tools best suited to meet the needs of the certification program and the targeted occupation. Assessment tools must adhere to a rigorous development process to ensure their validity, reliability and fairness. As well, they should be administered using proper administrative policies and procedures. Consideration of the advantages and suitability of the various assessment methods is required for certification programs to be constructed to deliver the promised organizational benefits.

Professional certification is a voluntary process that recognizes individuals who meet predetermined standards, as assessed by a third party. More specifically, professional certification programs provide an independent, impartial assessment of an individual’s occupational competence, where candidates are evaluated against predetermined standards for knowledge, skills and abilities.

Candidates who demonstrate that they meet established standards are granted a time-limited credential and are awarded an acronym to signify that they have obtained the credential. To retain the credential, candidates must maintain and demonstrate continued competence in their occupation, abide by a code of conduct and adhere to occupational standards.

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.

How Can You Plan For Competency-Based Management?

Stakeholder participation is important not only to create "buy-in" but also to ensure competencies truly reflect the behaviours that contribute to and sustain organizational success.
It takes effort and commitment to implement a fully-elaborated and integrated Competency-based human resource management (HRM) system. It is important, therefore, to take the time to evaluate the needs of the organization, and to create a strategy and plan that will meet these needs - in other words, getting it right the first time.

Developing the Strategy:
Having identified the business need, the champions for change and the organizational readiness, the organization is in a position to more precisely define a staged approach for developing and implementing competency models.

As the first major challenge the organization must decide to what level the competencies will be defined. For example, will it be sufficient to define the common / core competencies for everyone in the organization, or do specific competencies have to be developed for particular classifications and levels, functions, or jobs? The answer to this question hinges on how the competencies will be used. For example, to staff particular positions, competencies should be defined for the job. On the other hand, for appointments to level, for appointments to level, competencies need only be defined at the core or common level.

The organization must also determine the competency modeling method best suited to support the identified needs. In our experience, no one single method will effectively support all components of the human resource management system (i.e. training and development, selection, performance management, etc.), the full range of occupations and levels (executive, professional, skilled, semi-skilled, etc.), or the various types of organizational and business environments.

Finally, communication is imperative at all stages of the planning, development and implementation process. In addition to promoting the value, benefits and ways in which the Competency-based initiative will be implemented, stakeholder participation in the process is also important, not only to create "buy-in" for the initiative, but also to ensure that the competencies truly reflect the behaviours that will contribute to and sustain organizational success.

Common Pitfalls of Competency Initiatives
No sponsor, or sponsor with insufficient power, influence, credibility or strategic perspective.
No perceived need for change, among senior leaders or groups with power.
Resistance to change across the organization.
No clear identification of stakeholders – not involving them.
Losing momentum – priorities change.
Non-existent / inadequate training – managers, supervisors, employees, HR staff.
Support infrastructure and finding not in place.
Inadequate project management / project talent.
Not implementing right away.
Competencies / applications too complicated.

This post is based on content from 'Competency-based Human Resource Management: Planning for Success' by Suzanne Simpson, Ph.D. And Lorraine McKay, M.A.

Competency-Based Human Resource Management: Developing the Business Case

Stakeholder participation is important not only to create "buy-in" but also to ensure competencies truly reflect the behaviours that contribute to and sustain organizational success.
It takes effort and commitment to implement a fully-elaborated and integrated Competency-based human resource management (HRM) system. It is important, therefore, to take the time to evaluate the needs of the organization, and to create a strategy and plan that will meet these needs - in other words, getting it right the first time.

Developing the Business Case:
Our years of experience in implementing Competency-based human resource management programs have shown that, as with any other significant change initiative, there must be a compelling need and will to change. It is not sufficient for the organization’s human resource or training professionals to see the need; leaders of the organization must also see the benefits and be willing to champion the initiative. Likewise, employees must understand how the program will benefit them both in their current jobs, as well as in advancing their careers. For this reason, many organizations have chosen first to implement components of a Competency-based HRM system that address the expressed needs of employees, preferably in a non-threatening way - for example, a Competency-based self-directed learning program.

This post is based on content from 'Competency-based Human Resource Management: Planning for Success' by Suzanne Simpson, Ph.D. And Lorraine McKay, M.A.

What is Competency-Based Human Resource Management?

A competency is “any skill, knowledge, or other attribute that is observable and identifies successful performance.” Effectively, competencies translate the strategic vision and goals for the organization into measurable and observable behaviours or actions that employees must display.

A common framework of competencies provides the means for integrating all aspects of the HR system so that employees are selected, evaluated, developed, promoted and rewarded based on competencies that support organizational success. By communicating these competencies, organizations empower employees to take charge of their careers, direct their own personal development and continually self-evaluate and improve. At the same time, the framework allows the organization to pro-actively plan for its human resource needs both in the immediate and long term, and to establish programs that support employees in acquiring the competencies needed for organizational success.

This post is based on content from 'Competency-based Human Resource Management: Planning for Success' by Suzanne Simpson, Ph.D. And Lorraine McKay, M.A.

Competency-Based Management: Lessons Learned

There have been a number of lessons learned in designing and implementing Competency-based talent management systems over the past ten to fifteen years. First, the implementation of a Competency-based management system cannot be driven solely by the Human Resources Department. Senior and line management must have a compelling reason for implementing Competency-based management and see the value that it will be bring in supporting the organization's strategic goals. Further, they must be prepared to visibly support, fund and champion the initiative within the organization. The organization must commit the necessary resources to make it happen - it cannot be a secondary duty to be performed off the side of an already overworked HR professional's desk. There must be a well-articulated and staged plan for development and implementation, and employees have to be involved in the process and understand the benefits of CBM for them. This means designing and implementing a good communications plan. Finally, designing, implementing and gaining a full return on investment for Competency-based management takes time. However, quick gains can be made in many areas if managers and HR staff examine the organization's pain points and use Competency-based tools and processes to address these as quickly as possible. Having a well-designed talent management system will expedite this process.

The use of Competency-based management systems affords companies the opportunity to concentrate on their operations without sacrificing the need to have a well managed workforce. Recent research is showing that organizations can reap major financial gains through Competency-based management. Implementing a Competency-based system can make a major difference in the efficiency and profitability of the company, the productivity of the workforce and the amount of manager and HR staff time spent on HR issues, thereby providing a competitive advantage in the market place.

This post is based on content from 'Competency-Based Management That Works!' by Suzanne Simpson

The Competency-Based Management Advantage

Managers can compare and rank order the employee pool available within the organization against competency requirements for specific positions – for example, to support filling immediate vacancies or when planning for successors. Managers can view the whole resource pool that is available and not just those employees who are known to the manager. As a result, the organization benefits from the best employees being selected and promoted, and employees in turn are given more equitable opportunities for career development and growth within the organization.

From a career planning perspective, Competency-based talent management systems allow employees to compare the competencies they possess with those required in the various jobs in the organizations. Employees can then decide on career options they would like to pursue and develop plans to address gaps and progress in their careers.

Planning for longer-term strategic workforce requirements becomes a less complicated with the analysis and reporting capabilities that are built into the new talent management systems. These systems allow senior managers to easily compare the current workforce capacity and capabilities with the talent requirements to achieve the organization’s strategic goals. Based on this analysis, management can then put in place strategies and programs to address gaps and position the organization to achieve its goals.

This post is based on content from 'Competency-Based Management That Works!' by Suzanne Simpson