Applying Monte Carlo Simulation to Determine the Likelihood of Cheating on a Multiple-Choice Professional Exam

Abstract

This paper outlines statistical arguments used in an attempt to determine if cheating occurred on a multiple-choice exam. The arguments include the testimony in a court case involving accusations of cheating on a 100-question professional multiple-choice examination with four choices for each question. In response to the fact that the prosecution employed a witness who was an expert in statistical analysis, one of the authors was engaged by the defense to conduct an independent statistical analysis of the exam scores. The prosecution’s witness utilized a simulation to demonstrate, in his opinion, the relative certainty of cheating by the defendant in the case. The authors performed their own analysis, including simulations, to counter the testimony of the prosecution. The results presented in this paper highlight the fact that in the absence of definitive proof, in spite of a statistical analysis of data, there is still a need to make subjective interpretations when trying to decide if cheating has occurred on a multiple-choice test.