Types of Assessment Tools: Structured Interviews and Oral Assessments

A structured interview is a form of assessment where a candidate responds to questions about his or her skills and work accomplishments. An oral examination is similar to a written essay except that the questions are asked verbally and there is the possibility of asking further questions and clarifying answers. Both of these assessment methods are best suited for initial certification and for use in combination with other tools. The assumption underlying this approach is that a candidate’s past performance is the best predictor of future performance. The following table outlines the advantages and disadvantages of these tools.

Advantages
Can assess candidates’ soft skills (in case of structured interview) and knowledge (in case of oral examination)
Relatively easy to develop

Disadvantages
Must develop a thorough scoring key
Requires trained assessors (often 2 or more conducting the interview)
Can be time consuming and expensive to administer and score
Candidate is aware of evaluation and may respond in a dishonest fashion to meet assessor expectations
Candidates may misrepresent their achievements

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.

Why is Competency-based Management for Internationally Trained Individuals?

Competencies are only the starting point as, in organizations using CBM, the competencies are connected to each of the Human Resources functions. Thus, your competency approach can support ITIs in many different ways. The chart that follows on the next page illustrates some of the ways that CBM can help you bridge your gaps with ITIs in your organization.



This post is based on content from 'Integrating Internationally Trained Individuals without Reinventing the Wheel' by Shaun Vollick.

Types of Assessment Tools: Portfolio Assessment

A portfolio is an organized collection of material attesting a candidate’s learning and achievements related to standards or competencies. The candidate is required to prepare a personal portfolio that contains elements including: formal education, work history, documented informal learning experiences, and sample work. Candidates must provide supporting evidence of everything included. The following table outlines the respective advantages and disadvantages of this type of assessment.

Advantages:
Provides an assessment of knowledge and skills which are difficult to observe and evaluate for others

Disadvantages:
Costly to develop reliable scoring scheme (need very clear guidelines)
Candidates must spend considerable time assembling the portfolio
Time consuming and difficult to score
Variety of documents submitted therefore difficult to establish equivalencies

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.

Types of Assessment Tools: Work Simulation

In work simulation, candidates are presented with a task that is similar to work tasks that candidates typically perform on the job. They are required to perform interventions or actions based on the situation presented. This method is best suited for initial certification or for use in combination with other tools, depending on the scope and complexity of the simulation.
Examples of work simulation could include a 15-minute role-play for sales associates or the Objective Structured Clinical Examination (OSCE) for physiotherapist and physicians, a high-stakes exam that includes twenty work stations over the course of half a day. The following table outlines the advantages and disadvantages of work simulation assessments.

Advantages
Closely imitate the work environment
Tests candidates on tasks they typically perform on the job
High face validity and well accepted by candidates
Sound psychometric qualities

Disadvantages
Difficult and expensive to develop
Requires significant resources for administration
Candidates are aware of evaluation, so they can modify their behaviours to meet expectations of assessors

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.

Integrating Internationally Trained Individuals without Reinventing the Wheel

Due to the retirement of the Baby Boom generation and decrease in birth rates, labor markets in many Western countries are currently facing an impending knowledge and skill shortage as the lack of available domestic talent rapidly shrinks. One of the obvious sources for skilled workers to fill that gap has been through immigration policy and the recruitment of Internationally Trained Individuals (ITIs).

The hiring of Internationally Trained Individuals (ITIs) has not been as simple as many employers have expected. Many cite barriers to hiring these newcomers such as the inability to assess international training and education credentials, language proficiency and the ability for the individual to integrate into the culture of the company and the country.

As such governments and organizations have invested significant funds in the past decade resulting in millions of dollars being spent on thousands of projects to aid in the recruitment of ITIs. Many of these projects have sought to create new and innovative means of assessment and integration and many have been faced with an elimination of their funding due to the inability to produce results.

Filling the Gap without Starting from Scratch:

Especially during these difficult financial times and just as much so as we emerge from fiscal restraint, perspectives shift from looking for new and innovative solutions, to what we can do using what we have today. One such approach that has been used by organizations for decades in the recruitment and integration of its new employees is Competency-based Management or CBM.

Many organizations throughout the world have effectively built their human resource functions around CBM in order to operate more efficiently and productively. CBM is not only an effective method to organize talent but it is also associated with a critical collection of tools that can be used in bridging the skills gap for ITIs; an approach which may cost little or nothing for some organizations and is effective for all current and potential employees.

This post is based on content from 'Integrating Internationally Trained Individuals without Reinventing the Wheel' by Shaun Vollick.

Types of Assessment Tools: Performance-based or On-the-job?

On-the-job observation is a form of performance evaluation where a candidate’s on-the-job performance is observed and evaluated by an independent trained assessor. The candidate can be aware of being observed and evaluated by an assessor or observation can occur incognito (i.e., without the candidate’s awareness). This method of assessment is best suited for initial certification or used in combination with other tools, depending on extent of observation performed. According to Knapp’s Certification Industry Scan (2007) performance-based assessments, including work simulation, account for 17% of certification assessments. The following table outlines the advantages and disadvantages of the observation-based methods.

Advantages
Provides a direct assessment of individual’s skills, allowing to collect candidate performance information not otherwise available
High face validity and acceptance by candidates
Easy to develop

Disadvantages
Not suitable for assessing skills which are not directly observable
Need detailed scoring key
Need to recruit and train independent, third party assessors
Presence of an assessor typically changes the behaviour of a candidate
Can be time consuming
Can be expensive to administer (e.g., high volume of candidate, remote location)

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.

Types of Assessment Tools: Written Examinations

Written examination is the most commonly used type of assessment for certification purposes. This exam assesses candidates’ knowledge in specific areas, as defined in the blueprint. Written examination is best suited for initial certification as it enables a wide coverage of content and it can be used to assess both knowledge and its application. Other advantages that explain the popularity of this method include its low cost for candidates and its decreased cost to administrators as the number of candidates increase. It can also be delivered online and it has a high degree of candidate acceptance.

Written examination is flexible and can be used either on its own or paired with other tools. Although written examination can also be used for re-certification, in practice, this occurs infrequently. In Certification Industry Scan, Knapp & Associates (2007) found that 97% of certifying bodies administered written examination as one form of assessment and, on average, the written examinations included:

98% multiple-choice questions
9% essay questions
8% true-false questions
5% answer completion
3% short answer

The following provides information on the advantages and disadvantages of multiple-choice examinations and other forms of written examinations.

Multiple Choice Exams:
Advantages
Sound psychometric qualities
Wide coverage of domain content
Can assess both knowledge as well as application of knowledge
Can be scored quickly
Can be administered more than once
Low cost for candidate and decreases as the number of candidates increases
Possibility of online delivery
Well accepted by candidates

Disadvantages
Need large pool of questions to ensure their rotation. Measuring practical skills is not always possible
Initial development costs are high, therefore requiring significant investment capital
Requires creation of various versions of the exam to prevent question over-exposure and to ensure security of the test.

Essays//Short Answer/Answer Completion
Advantages
Can assess both knowledge as well as application of knowledge, can obtain more information in some areas
Can have reasonable coverage of domain content
Cost for candidate is still reasonable
Possibility of online delivery

Disadvantages
Need to revise content after each administration
Attention needs to be paid to development of scoring key
Measuring practical skills is not always possible
Scoring can be time-consuming and challenging

This post is based on content from 'Assessment Tools Certification' by Human Resource Systems Group, Ltd.