A Thought Experiment Regarding Experimental Ethics

The current controversy about a large scale experiment conducted in Montana by Stanford and Dartmouth political scientists raises several issues about research ethics in political science. To see a scan of the mailer that was sent to a random subset of Montana registered voters, follow this link.

Many of the objections I have read specifically refer to the form of the mailer. I am not going to engage with those objections here. I am interested in whether any experiment conveying the same information about candidates was necessarily objectionable, which is relevant for assessing whether the broad class of electoral field experiments are ethically problematic.

In an effort to help identify exactly what people find troubling about the study, I propose the following thought experiment. Imagine that the researchers had sent out the mailer without the Montana state seal. Imagine further that the graphical presentation of the scale positions was identical to that actually used. However, that the title and description of the information was as follows:

Public Information About Donors to Nonpartisan Candidates

Donations to candidates for public office are public information. However they are collected in large databases that are difficult for most people to access. We have been working for several years to develop simple summaries of whether candidates are supported by more liberal or more conservative donors. The upcoming 2014 nonpartisan campaign for the Montana Supreme Court has candidates who receive donations from individuals around the US. When we look at the other candidates that those donors are giving money to, we can calculate whether the donors tend to be liberal or conservative. This gives a general indication of whether the donors for a given candidate are more liberal or more conservative on average, even when the candidates themselves are nonpartisan. The average positions of donors for each of the candidates in the upcoming Supreme Court election are shown above, compared to the average positions of donors to Barack Obama and Mitt Romney in the 2012 US Presidential Election.

For more information on how these calculations were constructed, and why we think they provide reasonable assessments of the extent to which a candidate’s donors are generally more liberal or conservative, please see http://data.stanford.edu/dime. This guide was created…

My version of the mailer is a mechanical description of what the CFScores are and how they are calculated. It is relatively difficult to argue that this is misleading, although not impossible, as one could quibble about the “liberal” vs “conservative” labels. The final paragraph implies that one might plausibly reject the provided interpretation of the scores in that light. Assuming you find the actual experiment objectionable, do you find this hypothetical version objectionable?

For some other comments on this, see:

Chris Blattman

Thomas Leeper