Critical-Thinking Assessment: A Case Applying Resampling to Analyze the Sensitivity of a Hypothesis Test to Confounding

Abstract

The Case for this paper illustrates how a statistical resampling technique can be used to provide a sensitivity analysis for the possible vulnerability of a hypothesis test to confounding. In spite of other methods for guarding against confounding influences, there will always be uncertainties associated with the model at the base of a hypothesis test. It is demonstrated that, by modeling those influences that might cause confounding, simulation can be used to determine the sensitivity of a hypothesis test to give a false positive, at various strengths of the confounding. To illustrate these methods, a case is introduced based on the longstanding efforts of nursing educators to improve the teaching of critical thinking (CT) in their programs, and to apply assessment tools to test whether their work has been successful. Given years of mixed results, some suggest that the measurement tools employed may be flawed. This raises the question of how sensitive the hypothesis tests they employed would have been to confounding effects, if these had been introduced by using unreliable measures to assess critical thinking.