Assessment Centers as Tests
Our presentation focuses on the idea that while assessment centers are complex tests of performance, they are not treated like tests in development. That is, the rigor that is applied to test development in terms of item review and item analysis is not applied to exercise and rating form development in assessment centers.
The dimensional structure that is designed in exercises is not subjected to serious review and empirical evaluation. Just as dimensional structure for tests is assessed at the item level, the dimensional structure of assessment center exercises must start at the level of behaviors, the “test items” of assessment centers. Our review of the literature suggested that previous studies of assessment center construct validity have not been based on behavioral ratings as the initial unit of analysis. Behavioral ratings have been ignored in the search for construct validity in assessment centers.
We will present results from a current research project that demonstrate that by applying the same type of rigor that is used in traditional test development, to assessment center development, several benefits are realized. These include obtaining a more accurate view of the underlying structure of the assessment center behaviors, estimates of internal consistency reliability of dimensions and exercises, estimates of variance or lack of variance on individual behaviors, and insights into the causes of exercise effects. Conversely, we will also discuss the implications of not treating assessment center development with the same type of rigor that is applied in traditional test development. For example, we will discuss the implications of assigning dimension-level ratings and providing dimension-based feedback to candidates’ without first evaluating the “test items” in a more traditional manner. The proper integration of ratings will also be discussed from a test development perspective.
Speaker
Bob Lovler, Ph.D.
Selection and Assessment Technology Director
Wilson Learning Corporation
Dr. Bob Lovler is currently the Technology Director for assessment and selection services at Wilson Learning Corporation. He has over 25 years of experience as both an internal and external consultant to major corporations and private-sector organizations. He recently managed and played a key role in creating Wilson Learning’s electronic assessment center offering. Dr. Lovler received his Ph.D. in Industrial/Organizational Psychology from Hofstra University, and served on the faculty of The Graduate School at Hofstra and the State University of New York. In addition, he also served at the Vice-President level with a Fortune 500 communications company for 11 years.
Speaker
Mark Rose, Ph.D.
Wilson Learning Corporation
Dr. Mark Rose is responsible for the design, development, and implementation of selection, assessment, and development systems for Wilson Learning clients. He has eight years of consulting experience in both the private- and public sector. He recently played a key role in designing the content and format of Wilson Learning’s automated assessment centers. Dr. Rose received his Ph.D. in Industrial/Organizational Psychology from the University of South Florida, and has presented or published in the areas of assessment centers, leadership, statistical methodology, and burnout.
