With so many assessment tools available to you, finding the right assessments that meet your business needs can be confusing. Here are six questions to ask about each assessment you are considering.
Creating a new assessment from scratch is an extremely lengthy process. Effective tests aren’t built in a garage over the weekend. Rather, the instrument’s concept must be vetted against solid, empirical research in a process that can take years to complete. Without data from real people, there’s little room to believe an assessment is predictive of employee performance. Take time to understand how the assessment was developed, how the assessment relates to predicting performance for your job types, and why the vendor developed the assessment in the manner they did.
Bottom line: Go beyond the software interface and look to the technical manual to understand the steps involved in building the assessment.
2. What is the theory behind the assessment?
A critical step in the development of any assessment is to define what characteristics or knowledge are to be measured, and to provide a rationale for why the instrument is expected to be valid. Assessment claims should always be backed up by substantial research evidence. To ensure the content of an assessment aligns with the behaviors and knowledge actually displayed on the job, test providers are expected to conduct thorough job analyses and observations.
Bottom line: Be wary of assessments that were developed based on pet theories or assumptions about how employees or organizations work. Take time to talk to the potential vendor about how they would defend their assessments.
3. What validity evidence does the assessment have?
Test reliability and validity are the most important features of a test in terms of quality and usefulness. Assessments need to not only be legally defensible, but should predict some type of performance. Criterion-related, content, and construct are the three main types of validity. By validating a test, you can determine how job-related and useful an assessment is for your business.
Bottom line: Make sure enough proof exists to support the interpretation of test scores. Ask how the vendor will develop and implement a validation strategy for your company.
4. Is the vendor willing to share technical manuals?
It’s wrong to judge a book by its cover. In the case of assessments, it’s even more of a mistake to not read the book at all. A technical manual is a testament to the vendor’s ability to rigorously design and evaluate its assessment. A high-quality manual includes information on the rationale behind and approach to developing the assessment; details of how the assessment was developed; research highlighting psychometric properties, fairness, and validation evidence; guidelines for assessment administration; and contact information for the test’s developers. A vendor’s inability or unwillingness to produce the technical documents for their assessments is a huge red flag.
Bottom line: Only in the hands of an expert does a technical manual make sense. Hire a third-party consultant - ideally an industrial-organizational psychologist - to evaluate the assessment’s supporting documents.
5. What kind of training and support does the vendor provide?
Bottom line: Make sure the vendor’s product is complemented by services and resources that help you thoroughly understand and implement an assessment process.
6. Will they adapt the assessment profile based on your evolving business needs?
There is no one-size-fits-all approach to assessing applicants for a job. What matters most to your company might not be what your competitors value. Plus, priorities change over time. If a vendor is unable to provide ongoing monitoring of how the assessments are performing and make recommendations to drive improvement, the value of the assessment process becomes less clear.
Bottom line: Avoid outpacing your vendor by partnering with a company that can adapt to the changing demands within your organization.
Topics: Talent Selection Ideas