Syllabus

Course details

  • Any day
  • August 24–December 15, 2020
  • Any time
  • Slack

Contacting me

E-mail and Slack are the best ways to get in contact with me. I will try to respond to all course-related e-mails and Slack messages within 24 hours (really), but also remember that life can be busy and chaotic for everyone (including me!), so if I don’t respond right away, don’t worry!

Course objectives

By the end of this course, you (1) will be literate in the language of causal inference, (2) will communicate evaluation outcomes clearly, and (3) will understand the ethics and limits of data analysis by designing, critiquing, coding, and running rigorous, valid, and feasible evaluations of public sector programs focused on society’s most pressing problems.

Specifically, you’ll be able to:

  • Explain the philosophy of causation
  • Identify and diagram program logic models
  • Outline theories of change with directed acyclic graphs (DAGs)
  • Summarize key threats to causal inference, identify these threats in evaluations, and mitigate these threats with research design
  • Develop rigorous and valid statistical measures
  • Run statistical models
  • Explain the theory, research design, methods, and results of evaluations to all types of stakeholders, from highly trained econometricians to the general public
  • Share your analyses and data with the public
  • Identify ethical issues and limits in data science and program evaluation
  • Become curious and confident in consuming and producing evaluations

Course philosophy

Classical statistics classes spend substantial time covering probability theory, null hypothesis testing, and other statistical tests first developed hundreds of years ago. Some classes don’t use software or actual real data and instead live in the world of mathematical proofs. They can be math-heavy and full of often unintuitive concepts and equations.

In this class, we will take the opposite approach. We begin with data and learn how to tidy, wrangle, manipulate, and visualize it with code. Later in the semester we’ll turn to the powerful toolbox of causal inference approaches, but continue to keep the focus on data as we do so.

In other words, there’s way less of this:

\[ f(x) = \dfrac{1}{\sqrt{2\pi}} e^{-\frac12 x^2} \]

And way more of this:

summary_monthly_temp <- weather %>% 
  group_by(month) %>% 
  summarize(mean = mean(temp), 
            std_dev = sd(temp))

Over the last decade there has been a revolution in statistical and scientific computing. Open source languages like R and Python have overtaken older (and expensive!) corporate software packages like SAS and SPSS, and there are now thousands of books and blog posts and other online resources with excellent tutorials about how to analyze pretty much any kind of data.

This class will expose you to R—one of the most popular, sought-after, and in-demand statistical programming languages. Armed with the foundation of R skills you’ll learn in this class, you’ll know enough to be able to find how to analyze any sort of data-based question in the future.

Important pep talk!

I promise you can succeed in this class.

Learning R can be difficult at first—it’s like learning a new language, just like Spanish, French, or Chinese. Hadley Wickham—the chief data scientist at RStudio and the author of some amazing R packages you’ll be using like ggplot2made this wise observation:

It’s easy when you start out programming to get really frustrated and think, “Oh it’s me, I’m really stupid,” or, “I’m not made out to program.” But, that is absolutely not the case. Everyone gets frustrated. I still get frustrated occasionally when writing R code. It’s just a natural part of programming. So, it happens to everyone and gets less and less over time. Don’t blame yourself. Just take a break, do something fun, and then come back and try again later.

Even experienced programmers and evaluators find themselves bashing their heads against seemingly intractable errors. If you’re finding yourself taking way too long hitting your head against a wall and not understanding, take a break, talk to classmates, e-mail me, etc.

Alison Horst: Gator error

Course materials

Most of the readings in this class are free.

Books

We will only use one physical textbook. There are three official textbooks for the class:

The World Bank’s Impact Evaluation in Practice will be our main textbook. It’s written at a general, easy-to-understand level with relatively minimal math. Mastering ’Metrics goes into more depth about the mechanics of different causal inference approaches and has a bit more math, but it’s still fairly accessible. Causal Inference: The Mixtape has even more math, but hopefully not an excessively terrifying amount.

You do not need to understand all the equations and notation. If your eyes start to gloss over the Greek letters and subscripts, it’s okay. Try to learn them, but don’t stress out about it too much.

Articles, book chapters, and other materials

There will also occasionally be additional articles and videos to read and watch. When this happens, links to these other resources will be included on the reading page for that week.

R and RStudio

You will do all of your analysis with the open source (and free!) programming language R. You will use RStudio as the main program to access R. Think of R as an engine and RStudio as a car dashboard—R handles all the calculations and the actual statistics, while RStudio provides a nice interface for running R code.

R is free, but it can sometimes be a pain to install and configure. To make life easier, you can (and should!) use the free RStudio.cloud service, which lets you run a full instance of RStudio in your web browser. This means you won’t have to install anything on your computer to get started with R! We will have a shared class workspace in RStudio.cloud that will let you quickly copy templates for labs and problem sets.

RStudio.cloud is convenient, but it can be slow and it is not designed to be able to handle larger datasets or more complicated analysis. Over the course of the semester, you’ll probably want to get around to installing R, RStudio, and other R packages on your computer and wean yourself off of RStudio.cloud. This isn’t necessary, but it’s helpful.

You can find instructions for installing R, RStudio, and all the tidyverse packages here.

Online help

Data science and statistical programming can be difficult. Computers are stupid and little errors in your code can cause hours of headache (even if you’ve been doing this stuff for years!).

Fortunately there are tons of online resources to help you with this. Two of the most important are StackOverflow (a Q&A site with hundreds of thousands of answers to all sorts of programming questions) and RStudio Community (a forum specifically designed for people using RStudio and the tidyverse (i.e. you)).

If you use Twitter, post R-related questions and content with #rstats. The community there is exceptionally generous and helpful.

Searching for help with R on Google can sometimes be tricky because the program name is, um, a single letter. Google is generally smart enough to figure out what you mean when you search for “r scatterplot”, but if it does struggle, try searching for “rstats” instead (e.g. “rstats scatterplot”).

Additionally, we have a class chatroom at Slack where anyone in the class can ask questions and anyone can answer. I will monitor Slack regularly and will respond quickly. (It’s one of the rare Slack workspaces where I actually have notifications enabled!) Ask questions about the readings, assignments, and project. You’ll likely have similar questions as your peers, and you’ll likely be able to answer other peoples’ questions too.

Course policies

Be nice. Be honest. Don’t cheat.

We will also follow Georgia State’s Code of Conduct.

This syllabus reflects a plan for the semester. Deviations may become necessary as the semester progresses.

Student hours

Please watch this video:

Student hours are set times dedicated to all of you (most professors call these “office hours”; I don’t1). This means that I will be in my office at home (wistfully) waiting for you to come by talk to me remotely with whatever questions you have. This is the best and easiest way to find me and the best chance for discussing class material and concerns.

Because of the pandemic, we cannot meet in person. I can meet you online via Webex. Make an appointment with me here, and then use this link to talk to me during student hours: https://gsumeetings.webex.com/meet/aheiss. You can also find me through e-mail and Slack.

Learning during a pandemic

Life absolutely sucks right now. None of us is really okay. We’re all just pretending.

You most likely know people who have lost their jobs, have tested positive for COVID-19, have been hospitalized, or perhaps have even died. You all have increased (or possibly decreased) work responsibilities and increased family care responsibilities—you might be caring for extra people (young and/or old!) right now, and you are likely facing uncertain job prospects (or have been laid off!).

I’m fully committed to making sure that you learn everything you were hoping to learn from this class! I will make whatever accommodations I can to help you finish your exercises, do well on your projects, and learn and understand the class material. Under ordinary conditions, I am flexible and lenient with grading and course expectations when students face difficult challenges. Under pandemic conditions, that flexibility and leniency is intensified.

If you tell me you’re having trouble, I will not judge you or think less of you. I hope you’ll extend me the same grace.

You never owe me personal information about your health (mental or physical). You are always welcome to talk to me about things that you’re going through, though. If I can’t help you, I usually know somebody who can.

If you need extra help, or if you need more time with something, or if you feel like you’re behind or not understanding everything, do not suffer in silence! Talk to me! I will work with you. I promise.

Please sign up for a time to meet with me during student hours at https://calendly.com/andrewheiss/. I’m also available through e-mail and Slack. I’ve enabled notifications on my Slack account, so I’ll see your messages quickly!

I want you to learn lots of things from this class, but I primarily want you to stay healthy, balanced, and grounded during this crisis.

Late work

You will lose 1 point per day for each day a problem set is late. This is designed to not be a huge penalty (3 days late = 27/30 points on a problem set that gets a ✓), but instead is a commitment device to help you stay on schedule.

After 1 week, I will send an automated reminder e-mail. After 2 weeks, you will receive no points.

Counseling and Psychological Services (CPS)

Life at GSU can be complicated and challenging (especially during a pandemic!). You might feel overwhelmed, experience anxiety or depression, or struggle with relationships or family responsibilities. Counseling and Psychological Services (CPS) provides free, confidential support for students who are struggling with mental health and emotional challenges. The CPS office is staffed by professional psychologists who are attuned to the needs of all types of college and professional students. Please do not hesitate to contact CPS for assistance—getting help is a smart and courageous thing to do.

Basic needs security

If you have difficulty affording groceries or accessing sufficient food to eat every day, or if you lack a safe and stable place to live, and you believe this may affect your performance in this course, please contact the Dean of Students for support. They can provide a host of services including free groceries from the Panther Pantry and assisting with homelessness with the Embark Network. Additionally, please talk to me if you are comfortable in doing so. This will enable me to provide any resources that I might possess.

Lauren’s Promise

I will listen and believe you if someone is threatening you.

Lauren McCluskey, a 21-year-old honors student athlete, was murdered on October 22, 2018 by a man she briefly dated on the University of Utah campus. We must all take action to ensure that this never happens again.

If you are in immediate danger, call 911 or GSU police (404-413-3333).

If you are experiencing sexual assault, domestic violence, or stalking, please report it to me and I will connect you to resources or call GSU’s Counseling and Psychological Services (404-413-1640).

Any form of sexual harassment or violence will not be excused or tolerated at Georgia State. GSU has instituted procedures to respond to violations of these laws and standards, programs aimed at the prevention of such conduct, and intervention on behalf of the victims. Georgia State University Police officers will treat victims of sexual assault, domestic violence, and stalking with respect and dignity. Advocates on campus and in the community can help with victims’ physical and emotional health, reporting options, and academic concerns.

Academic honesty

Violation of GSU’s Policy on Academic Honesty will result in an F in the course and possible disciplinary action.2 All violations will be formally reported to the Dean of Students.

Special needs

Students who wish to request accommodation for a disability may do so by registering with the Office of Disability Services. Students may only be accommodated upon issuance by the Office of Disability Services of a signed Accommodation Plan and are responsible for providing a copy of that plan to instructors of all classes in which accommodations are sought.

Students with special needs should then make an appointment with me during the first week of class to discuss any accommodations that need to be made.

Assignments and grades

You can find descriptions for all the assignments on the assignments page.

Assignment Points Percent
Weekly check-ins (14 × 10) 140 15.6%
Problem sets (8 × 30) 240 26.7%
Evaluation assignments (4 × 30) 120 13.3%
Exam 1 100 11.1%
Exam 2 100 11.1%
Final project 200 22.2%
Total 900
Grade Range Grade Range
A 93–100% C 73–76%
A− 90–92% C− 70–72%
B+ 87–89% D+ 67–69%
B 83–86% D 63–66%
B− 80–82% D− 60–62%
C+ 77–79% F < 60%

Star Wars

Once you have read this entire syllabus and the assignments page, please click here and e-mail me a picture of a cute Star Wars character.3 Brownie points if it’s animated.

Baby Yoda with IG-11

  1. There’s fairly widespread misunderstanding about what office hours actually are! Many students often think that they are the times I shouldn’t be disturbed, which is the exact opposite of what they’re for!↩︎

  2. So seriously, just don’t cheat or plagiarize!↩︎

  3. Baby Yoda, Babu Frik, porgs, etc. are all super fair game.↩︎