January 2024 Online Short Course:
Web Scraping: Data Collection and Analysis
Web Scraping: Data Collection and Analysis
Dr. Richard Landers
Wednesday, January 3 – Friday, January 5
Taught daily from 10:00 AM ET – 4:00 PM ET
Instructor Biography
Dr. Richard Landers, Ph.D., is an Associate Professor of Psychology, and holds the John P. Campbell Distinguished Professorship of Industrial-Organizational
Psychology at the University of Minnesota. His research concerns the use of innovative technologies in assessment, employee selection, adult learning, and
research methods. Recent topics have included big data, game-based learning, game-based assessment, gamification, unproctored Internet-based testing, mobile
devices, virtual reality, and online social media. His work has been published in Journal of Applied Psychology, Industrial and Organizational Psychology Perspectives, Computers in Human Behavior, Simulation & Gaming, Social Science Computer Review, and Psychological Methods, among others, and his research and writing have been featured in Forbes, Business Insider, Science News, Popular Science, Maclean’s, and the Chronicle of Higher Education, among others. He is also author of a statistics textbook, A Step-by-Step Introduction to Statistics for Business, editor of Social Media in Employee Selection, and editor of the upcoming Cambridge Handbook of Technology and Employee Behavior. He currently serves as Associate Editor of Simulation & Gaming and the International Journal of Gaming and Computer-Mediated Simulations, and he is also part of the steering committee of the Coalition for Technology in Behavioral Science.
Course Description
In this course, you will learn how to create novel datasets from information found for free on the internet using only R and your own computer. First, after a brief introduction to data source theory, web architecture, and web design, we will explore the collection of unstructured data by scraping web pages directly through several small hands-on projects. Second, we will explore the collection of structured data by learning how to send queries directly to service providers like Google, Facebook and Twitter via their APIs. Third, we will briefly explore analytic techniques for analyzing scraped data. Finally, we will walk through the various ethical and legal issues to be navigated whenever launching a web scraping project.