Data Mining Seminar Description
An intermediate 5-day course introducing several popular data mining approaches such as regression based methods (ridge and lasso regularized regression, regression splines), tree methods (random forests, boosted trees), and support vector machines, and their application to empirical data. The course combines lectures and hands-on practice using R.
Instructor: Gitta Lubke Ph.D.
Gitta Lubke is a Full Professor in the Department of Psychology/Quantitative Area at the University of Notre Dame. Her research interests are in data mining and general latent variable modeling. In addition to the challenges of analysing complex human behavior such as psychiatric disorders, she is interested in the analysis of genetic data. Related areas of expertise include mixture models, twin models, multi-group factor analysis and measurement invariance, longitudinal analyses, and the analysis of categorical data.
In the age of rapidly increasing data collection endeavors it has become more and more important to understand how to find structure in data, especially when substantive theory about structural relations between the collected variables is not yet fully developed. This short course starts with briefly outlining the key differences and similarities between standard parametric modeling (e.g., linear regression) and data mining approaches. The course provides basic insights into a number of popular methods such as regression methods (ridge regression and the lasso, regression splines), tree methods (CART, random forests, boosting), and support vector machines. The emphasis is on a conceptual understanding of these methods and their appropriate application to empirical data. Importantly, these methods are useful not only for large data collections, but also more generally for exploratory analyses when the substantive theory to design and fit suitable parametric models (e.g. SEM) is not available. Data mining (aka statistical learning) is used in a wide variety of fields including but not limited to public health, education, biology, and the different social sciences.
Participants are invited to discuss potential data mining applications to their particular field of interest during individual consultations with the instructor scheduled at the end of the second and third day.
Participants will receive an electronic copy of all course materials, including lecture slides, practice datasets, software scripts, relevant supporting documentation, and recommended readings. Participants will also have access to a video recording of the course.
- Review of linear regression and the least squares criterion
- Regularization methods (ridge regression, lasso, elastic net)
- Regression splines
- Prediction error and k-fold cross validation
- Tree methods to predict categorical or continuous outcomes (CART, random forest, boosting
- Support vector machines for classification
Course Learning Goals
After engaging in course lectures and discussions as well as completing the hands-on practice activities with empirical data, participants will be able to:
- Understand some of the key differences and similarities between parametric modeling and data mining methods
- Expand the acquired basic knowledge of several popular data mining methods and apply these methods to empirical data
- Assess and interpret the results of empirical analyses through k-fold cross validation and computation of prediction errors
- Utilize R packages for data mining
- Understand and evaluate scientific papers covering data mining applications to empirical data
- advanced proficiency in linear regression, including the estimation of regression coefficients using least squares
- intermediate familiarity with iterative optimization (e.g. how to use the Newton-Raphson algorithm to find a maximum)
- Intermediate proficiency with R
- Intermediate knowledge of exploratory data analysis
Not required but advantageous:
- At least limited experience (e.g., graduate-level course) in calculus
- Understanding the relation between multiple testing and Type I error, and, more generally, the challenges of finding relevant predictors in large data sets
No level of proficiency beyond basic awareness is assumed for skills related to:
- Data mining methods
- More advanced mathematical or statistical topics such constrained estimation (e.g., using Laplace multipliers)
Course Software and Computer Support
Participants need to bring a laptop computer with Wi-Fi capabilities.
All statistical software used at Stats Camp will be available, free to participants, on our SMORS (statistical modeling on remote servers) system for the duration of camp.
All instruction for this course will be based on the freely available software program R. Please make sure to have a recent version installed.
Seminar files will be provided by the instructor on the first day of the seminar. You do not need to download anything prior to the event date. All materials will be provided during or after the class.