1. Decide on a Topic

For this project you may either
a) replicate a published Monte Carlo simulation study or part of a Monte Carlo simulation study or
b) design and implement a Monte Carlo simulation study to investigate a question of interest

2. The Proposal

The proposal consists of a description of
a) the research question(s) that motivate the project; include citations when referring to other work
b) the statistical methods that will be evaluated in the study
c) the data-generating process.
d) the outcome measures that will be used to assess performance.
e) your expectations; what do you expect the results to reveal?

3. R Code Draft

The R code draft will be a .R file that contains code used to run the simulation study. Essential elements for the code:
a) it is well-commented so that the purpose of each statement is clear and seeds will be set for reproducible results
b) it is aesthetically pleasing and well-organized. This means that indentations are used when appropriate and the ordering flows from top to bottom.
c) it runs and is replicable so that I can simply run lines from top to bottom in order to reproduce your results.
d) it implements approaches that favor efficiency in terms of computational speed and/or ease of expression in R whenever possible.

4 Final Paper:

The final paper will be your finished product. The goal is to produce a paper that is on par with the academic standards in your area. For most of us this will mean using APA format. If you have another format in mind, it is fine, so long as it is an accepted format in some other field. The final paper will include the following.

a) Elements (a) through (d) from the proposal, though now perhaps in more detail.
b) A results section including relevant tables and figures that highlight important results.
c) A conclusion in which you discuss your results and, if relevant, how they compare with the results from the original study
d) A references section.
e) An appendix that contains your R code. The R code should be displayed in a monospaced (fixed-pitch) font such as courier.
f) Some examples of published papers that contain one or more Monte Carlo simulation studies are listed in the references. This list is in no way meant to be exhaustive; there are thousands and thousands of published examples.

References

• Lee, B. K., Lessler, J., & Stuart, E. A. (2011). Weight trimming and propensity score weighting. PLoS ONE, 6 , 1–6.

• Ali, M. S., Groenwold, R. H. H., Pestman, W. R., V., B. S., Roes, K. C. B., Hoes, A. W., . . . Klungel, O. H. (2014). Propensity score balance measures in pharmacoepidemiology: a simulation study. Pharmacoepidemiology and Drug Safety, 23 , 802–811.

• Austin, P. C. (2012). Using ensemble-based methods for directly estimating causal effects: An investigation of tree-based g-computation. Multivariate Behavioral Research, 47 , 115–135

• Austin, P. C., Grootendorst, P., & Anderseon, G. M. (2007). A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: A Monte Carlo study. Statistics in Medicine, 26 , 734–753

• Belitser, S. V., Martens, E. P., Pestman, W. R., Groenwold, R. H. H., de Boer, A., & Klungel, O. H. (2011). Measuring balance and model selection in propensity score methods. Pharmacoepidemiology and Drug Safety, 29 , 1115–1129.

• Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Stürmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163 , 1149–1156.

• Cepeda, M., Boston, R., T., F. J., & Strom, B. L. (2003). Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. American Journal of Epidemiology, 158 , 280–287.

• Hayes, A. F. (1996). Permutation test is not distribution-free: Testing H0 : ρ = 0. Psychological Methods, 1 , 184–198

• Keller, B. (2012). Detecting treatment effects with small samples: The power of some tests under the randomization model. Psychometrika, 77 , 324–338.

• Keller, B., Kim, J.-S., & Steiner, P. M. (2015). Neural networks for propensity score estimation: Simulation results and recommendations. In L. A. van der Ark, D. M. Bolt, S.-M. Chow, J. A. Douglas, & W.-C. Wang (Eds.), New methods and applications in psychometrics: The 79th annual meeting of the psychometric society. New York, NY: Springer

• Lee, B. K., Lessler, J., & Stuart, E. A. (2011). Weight trimming and propensity score weighting. PLoS ONE, 6 , 1–6

• Long, J. S., & Ervin, L. H. (2000). Using heteroscedasticity consistent standard errors in the linear regression model. The American Statistician, 54 , 217-224.

• Maas, C. J. M., & Hox, J. J. (2004). Robustness issues in multilevel regression analysis. Statistica Neerlandica, 58 , 127–137.

• Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1 , 86–92.

• Mewhort, D. J. K. (2005). A comparison of the randomization test with the F test when error is skewed. Behavior Research Methods, 37 , 426–435.

• Moineddin, R., Matheson, F. I., & Glazier, R. H. (2007). A simulation study of sample size for multilevel logistic regression models. BMC Medical Research Methodology, 7 , 1–10.
• Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49 , 1373–1379.

• Petscher, Y., & Schatschnelder, C. (2011). A simulation study on the performance of the simple difference and covariance-adjusted scores in randomized experimental designs. Journal of Educational Measurement, 48 , 31–43.

• Pustejovsky, J. E. (2014). Converting from d to r to z when the design uses extreme groups, dichotomization, or experimental control. Psychological Methods, 19 , 92–112.

• Tomarken, A. J., & Serlin, R. C. (1986). Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures. Psychological Bulletin, 99 , 90–99.

• Vittinghoff, E., & MuCulloch, C. E. (2007). Relaxing the rule of ten events per variable in logistic and Cox regression. American Journal of Epidemiology, 165 , 710-718.

• Williams, M. N., Grajalel, C. A. G., & Kurkiewicz, D. (2007). Assumptions of multiple regression: Correcting two misconceptions. BMC Medical Research Methodology, 18 , 1–14.

1. Decide on a Topic For this project you may either a) replica...