Monday, March 29, 2010

Why it is important to record and report every auxiliary and decision made

The key ability of others to interpret and replicate experiments relies on the replication of the exact same decisions to result in the exact same auxiliary hypotheses. If these minute details are not reported, and they matter, then replication will fail and resources will be wasted searching for the proper design to replicate the previous experiment. Furthermore, as the research field builds, a standard set of decisions/auxiliaries builds and the decision-making process becomes easier and easier.
If we want to know the probability of our result replicating, we can use Prep. (Killeen, 2008; In Goode, 2008). Prep is P(d2|d1), however, it requires that study 2 is exactly identical to study 1. Thus, it is impossible for anyone to expect to replicate a study with probability= Prep if they do not know the exact details involved in the study. For example, “Many of the current conventions for sample size may be credited to Jacob Cohen’s (1988, 1992) tireless efforts to improve the quality of social science research. His voice is joined by a large group of methodologists deploring the failure of researchers to fully report the details needed to put the findings in context with the rest of the field. Therefore, an empirical study based on a limited sample is further weakened by superficial reporting that will not allow fellow scientists to replicate the complete study or even duplicate the calculations. This is a pointed cautionary note for novice researchers who have not yet grasped the nuances of the decisions to be made.” (137) (Peterson, 2008; in Goode, 2008).

No comments:

Post a Comment