Applied Scaling & Classification Techniques in Political Science using text data
(academic year 2023/24)

Syllabus

Course aims and objectives
Students will learn how to employ some widely discussed methods advanced in the literature to analyze political texts and to extract from them useful information for texting their own theories.

First Lecture 
1/12/23  Theory: An introduction to text analytics 
Reference texts:  (1; 2, 3)
  1. ​Grimmer, Justin, and Stewart, Brandon M. 2013. Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis, 21(3): 267-297
  2. Benoit, Kenneth (2020). Text as data: An overview. In Luigi Curini and Robert Franzese (eds.), SAGE Handbook of Research Methods is Political Science & International Relations, London, Sage, chapter 26 ​
  3. Grossman, Jonathan, and Pedahzur Ami (2020). Political Science and Big Data: Structured Data, Unstructured Data, and How to Use Them, Political Science Quarterly, 135(2): 225-257

​1/12/23 Lab class: An introduction to the Quanteda package​ (a) packages to install for Lab 1; b) scripts for Lab 1 (Part 1; Part 2); datasets: a) Boston tweets sample; b) Inaugural US Presidential speeches sample; c) sample of Japanese legislatives speeches (to open this file, please use the data compression tool WinRAR)

Second Lecture
8/12/23  Theory:  (Part 1): From words to positions: supervised and unsupervised scaling models ;  (Part 2): CMP dataset
Reference texts (1; ​2; 3; 4; 5; 6)
  • ​Laver, Michael, Kenneth Benoit, John Garry. 2003. Extracting Policy Positions from political texts using words as data. American Political Science Review, 97(02), 311-331
  • Egerod, Benjamin C.K., and Robert Klemmensen (2020). Scaling Political Positions from text. Assumptions, Methods and Pitfalls. In Luigi Curini and Robert Franzese (eds.), SAGE Handbook of Research Methods is Political Science & International Relations, London, Sage, chapter 27 
  • Martin, Lanny W., and Georg Vanberg. 2008. A robust transformation procedure for interpreting political text. Political Analysis, 16: 93-100
  • Proksch, Sven-Oliver, and Slapin, Jonathan B. 2008. A Scaling Model for Estimating Time-Series Party Positions from Texts. American Journal of Political Science, 52(3): 705-722.
  • Proksch, Sven-Oliver, and Slapin, Jonathan B. 2009. How to Avoid Pitfalls in Statistical Analysis of Political Texts: The Case of Germany. German Politics, 18(3): 323-344.
  • Curini, Luigi, Hino, Airo, and Atsushi Osaki. 2020. Intensity of government–opposition divide as measured through legislative speeches and what we can learn from it. Analyses of Japanese parliamentary debates, 1953–2013, Government and Opposition, 55(2), 184-201

8/12/23 Lab class: How to implement the Wordscores & Wordfish algorithms (a) packages to install for Lab 2; b) Lab 2 scripts (part I: Wordscores & Wordfish; part II: CMP); c) dataset for the first part of the Lab (party manifestos dataset); d) dataset for the second part of the Lab (music dataset))

First assignment (due: 15 December 2023)  (dataset for Assignment 1. To open this file, please use the data compression tool WinRAR)

Third Lecture
15/12/23  Theory: From words to issues: unsupervised classification models 
Reference text (1; 2):
  • ​Grimmer, Justin, and Stewart, Brandon M. 2013. Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis, 21(3): 267-297
  • Chan, Chung-hong, and Marius Sältzer. oolong: An R package for validating automated content analysis tools. The Journal of Open Source Software: JOSS 5.55 (2020): 2461

15/12/23 Lab class: How to implement a Topic Model (a) packages to install for Lab 3; b) Lab 3 script (part I: Topic Model; part II: CMP); c) dataset for Lab 3: Guardian 2016; EXTRA: a) Clustering Models; b) how to extimate a cluster model)

Second assignment (due: 22 December 2023) (dataset for Guardian 2013)

Fourth Lecture
22/12/23  Theory: Some Advancements in Topic Modeling: the Structural Topic Model and the Semi-Supervised (Structural) Topic Model
Reference text Reference text (1, 2, 3):
  • ​Robert, Margaret E., Brandon M. Stewart, Dustin Tingley, Christopher Luca, Jetson Leder-Luis, Shana Kushner Gadarian, Bethany Albertson, David G. Rand. 2014. Structural Topic Models for Open-Ended Survey Response. American Journal of Political Science, 58(4), 1064-1082
  • Roberts, Margaret E., Brandon M. Stewart, Dustin Tingley. 2014. STM: R Package for Structural Topic Models. Journal of Statistical Software
  • Shusei Eshima, Kosuke Imai, and Tomoya Sasaki (2023). Keyword-Assisted Topic Models, American Journal of Political Science, DOI: 10.1111/ajps.12779

22/12/23 Lab class: How to implement a Structural Topic Model and a Semi-supervised (structural) topic model (a) packages to install for Lab 4; b) Lab 4 scripts (part I: STM; part II: keyATM); datasets: a) NyT; b)  ​data for topical content analysis

Third Assignment (due: 12 January 2024) (dataset for Assignment 3. To open this file, use the command readRDS("Trump2018.rds"))

Fifth Lecture
12/1/24  Theory: (Part 1): Dictionary models; (Part 2): An introduction to supervised classification models
Reference texts (1; 2; 3)
  • Grimmer, Justin, and Stewart, Brandon M. 2013. Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis, 21(3): 267-297
  • Curini, Luigi, and Robert Fahey. 2020. Sentiment Analysis. In: Luigi Curini and Robert Franzese (eds.), Sage Handbook of Research Methods in Political Science and International Relations, London: Sage, chapter 29
  • Barberá, Pablo et al. (2020). Automated Text Classification of News Articles: A Practical Guide. Political Analysis, DOI: 10.1017/pan.2020
12/1/24 Lab class: How to implement a dictionary model and a naive-bayes one ( a) packages to install for Lab 5; b) Lab 5 scripts (part I: dictionary models; part II: Naive Bayes model); datasets: 1) sample of tweets discussing Donal Trump; 2) disaster training-set; 3) disaster test-set; 4) airlines training-set. EXTRA: a) converting an external dictionary to a Quanteda dictionary; b)  split-half reliability test; c) The meaning of a compressed sparse matrix)

Fourth Assignment (due: 19 January 2024)  (datasets for Assignment 4: a) UK training set; b) UK test set)

Sixth Lecture
19/1/24  Theory: Supervised classification methods with human tagging: some further ML algorithms
Reference text (1):
  • Olivella, Santiago, and Shoub Kelsey (2020). Machine Learning in Political Science: Supervised Learning Models. In Luigi Curini and Robert Franzese (eds.), SAGE Handbook of Research Methods is Political Science & International Relations, London, Sage, chapter 56  
19/1/24 Lab class: How to implement Random Forest and Support Vector Machine ML algorithms (a) packages to install for Lab 6; b) Lab 6 script)

Fifth Assignment (due: 26 January 2024)

Seventh Lecture
26/1/24  Theory:  (Part 1): How to validate your ML results; (Part 2): The importance of a good training set
Reference text (1, 2, 3, 4; 5; 6):
  • Grimmer, Justin, and Stewart, Brandon M. 2013. Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis, 21(3): 267-297
  • Cranmer, Skyler J. and Desmarais, Bruce A. (2017) What Can We Learn from Predictive Modeling?, Political Analysis, 25: 145-166
  • Soren Jordan, Hannah L. Paul, Andrew Q. Philips, How to Cautiously Uncover the “Black Box” of Machine Learning Models for Legislative Scholars”, Legislative Studies Quarterly, 2022, https://onlinelibrary.wiley.com/doi/abs/10.1111/lsq.12378
  • Arnold, Christian, Biedebach Luka, Küpfer Andreas, and Neunhoeffer Marcel. (2023). The Role of Hyperparameters in Machine Learning, Political Science Research and Methods, forthcoming
  • Barberá, Pablo et al. (2020). Automated Text Classification of News Articles: A Practical Guide. Political Analysis, DOI: 10.1017/pan.2020
  • Curini, Luigi, and Robert Fahey. 2020. Sentiment Analysis. In: Luigi Curini and Robert Franzese (eds.), Sage Handbook of Research Methods in Political Science and International Relations, London: Sage, chapter 29

26/1/24 Lab class: How to compute internal and external validity of a ML algorithm & inter-coder reliability (a) packages to install for Lab 7; b) Lab 7 scripts (part I: external validity; part II: internal validity; part III: inter-coder reliability);  functions to compute external and internal validity: for 2 class labels; for more than 2 class labels (to open the two files, please use the data compression tool WinRAR)

Sixth Assignment (due: 2 February 2024)

Seminar
29/1/23 - Third Period, room 902 Theory: A practical introduction to word embedding techniques for social science - with an extension to LLM 
Reference texts (1, 2; 3):
  • Rodriguez Pedro L. and Spirling Arthur (2022). Word Embeddings: What works, what doesn’t, and how to tell the difference for applied research, Journal of Politics, 84(1), 101-115​
  • Wankmüller, S. (2022). Introduction to Neural Transfer Learning With Transformers for Social Science Text Analysis. Sociological Methods & Research https://doi.org/10.1177/00491241221134527
  • Kjell, O., Giorgi, S., & Schwartz, H. A. (2023, May 1). The Text-Package: An R-Package for Analyzing and Visualizing Human Language Using Natural Language Processing and Transformers. Psychological Methods. Advance online publication. https://dx.doi.org/10.1037/met0000542
29/1/23 - Fourth Period, room 902  Lab class: How to implement GloVe and BERT. (scripts: a) packages to install; b) Script about Glove; c) Script about BERT; dataset & function for the lab: a) movie reviews dataset; b) social-disaster dataset; c) pre-trained WE on Google news; d) pre-trained WE on Facebook posts; e) function to compute cross-validation for a Random Forest model; f) BERT results via text package (rds file);  g) BERT results via grafzahl package (rds file); h) fine-tuning files (to open the rar file, please use the data compression tool WinRAR). EXTRA script: fine-tuning BERT for classification purpose)
​​