The Effects of Re-Framing and Respondent Personality on Psychometric Outcomes: A Replication and Extension Study

Abstract

The present study investigated the effects of four item characteristics – context, direction, factor, and trait aspect switching – on inter-item correlations and participant misresponse rates for adjacent item pairs within a large set of personality scales. The responses to an online survey from 300 individuals recruited from Amazon’s Mechanical Turk allowed for an investigation of these effects. The results of this study suggested that, across 16 different personality scales, adjacent item pairs that: (1) required higher levels context switching, (2) represented different factors, or (3) differed in trait aspect predicted both lower inter-item correlations and a greater percentage of misresponders. These adjacent item characteristics create distinct interpretative problems for participants and make unique contributions to the impairment of inter-item correlation and a greater proportion of misresponders. In addition to these findings, the results of this study suggest that individuals who have difficulty shifting between tasks and who are more impulsive (both in terms of failing to pay attention or acting without thinking) are less consistent in their responding to personality scales. The results of the present study provide further evidence for the detrimental effects of the switching that is required when responding to adjacent items that vary in terms of their context specificity, content domain, or trait aspect, and also provide evidence that individuals who are less able to switch their attention and are more impulsive are less successful at responding consistently to personality scale items. A number of strengths, limitations, and recommendations for future research are discussed.