An Integrated Approach to Assessment Validation and Refinement
Sanofi-Synthelabo, Inc., one of the fastest growing pharmaceutical companies in the world, needed to quickly double the size of its existing sales force to meet the demands of introducing a new line-up of products. Sanofi enlisted DDI’s help to rapidly hire 1,000 highly qualified sales representatives. The solution involved the development of an online screening system (qualifications and competency assessment), a DDI-conducted phone-based simulation, and behavior-based interviews by Sanofi managers.
DDI used a content validation strategy to initially establish the links between the selection system components and job activities. After a sufficient number of new hires had been on the job for a minimum of six months, DDI and Sanofi conducted a study to determine how performance on the screening tools related to various indicators of individual job effectiveness, and to identify areas for content improvement and refinement
The study was conducted in three parts: 1) a traditional criterion-related validation study that compared managers’ ratings of job performance and archival sales data with the employee’s online screening and phone assessment scores, 2) a policy- capturing investigation to determine the components of an applicant’s background that managers utilize when making decisions regarding candidate qualifications, and 3) focus groups with managers to better define online application content and scoring.
Results of the validation study showed several significant predictive relationships between the online application scores, phone assessment scores, performance criteria, and sales-related criteria. The policy capturing investigation indicated that managers tend to emphasize a person’s previous sales experience and attainment of sales awards when using background information to evaluate an applicant’s minimum qualifications and likelihood of success on the job. The focus groups confirmed the relative importance of the various online screening components (education, work experience, skills) in terms of screening for minimum qualifications and prioritizing applicants based on their backgrounds.
DDI and Sanofi integrated results from all three study components to refine the online application and phone assessment content, toward the goals of enhanced system efficiency and stronger prediction of performance We discuss our lessons learned and implications based on this case study employing multiple perspectives in the process of selection system redesign.
Speaker
Evan Sinar, Ph. D.
Consultant
DDI
As a consultant in DDI’s Assessment Technology Group, Evan develops content for and provides implementation guidance to numerous assessment solutions, including online qualifications screening, large-scale automated and traditional testing. Additionally, he conducts proactive research to ensure the psychometric integrity, legal defensibility, and productivity impact of DDI-developed selection systems. He has presented at numerous conferences and has published in professional journals such as the Journal of Applied Psychology, Personnel Psychology, and the International Journal of Selection and Assessment.
Speaker
Shiela Rioux, Ph. D.
Consultant
DDI
As a consultant with DDI, Sheila manages large engagements with key clients that involve web- based recruitment, selection, and assessment solutions. As part of the role, she designs and implements competency models, develops processes and procedures for improving web-enabled delivery of assessment technology, and coordinates information technology efforts in the delivery of customized solutions. In her previous role with DDI’s Center for Applied Behavioral Research, Sheila conducted and published numerous HR Benchmarking studies on topics such as Succession Management and Globalization of Human Resources.
