Ethical Questions in (Economic) Experiments

Something I have written about previously on this blog is the challenges researchers face in identifying causation. When we try to test how and why a policy works we want to be confident in attributing any changes to the policy and not some other factor. Say we found a policy intervention that we think would reduce the amount of sexual harassment women face in their workplace. However, we only know how this policy works in theory. To find out if it actually has the intended effect we must test it in practice. Often, the golden standard for testing a theory in the real world (from the researchers' point of view) is a random experiment. In our example, we would take a bunch of firms that look about the same: a similar number of employees, the same industry, a similar number of sexual harassment cases, and so on. This step is really important because it means that we are comparing apples with apples. If we compared very large firms with very small firms, it might be the size of the firm, not our policy, that influences outcomes. Now that we have our sample of very similar firms we split them into two groups, a treatment group, and a control group. Importantly, how we assign the groups is random. For example, for each firm, we flip a coin and if the coin shows heads the firm is in the treatment group and if the coin says tails the firm is in the control group. Firms in the treatment group will implement the policy and firms in the control group will just go about business as usual. After about six months we look at the firms and see if anything changed in the number of sexual harassment cases, in other words, we want to see if our policy worked in the firms that implemented the policy. If the sexual harassment cases in the treatment group are significantly different from the number of cases in the control group then we can claim with reasonable confidence that our policy works.

All fine right? Well, if you have watched the Good Place and now have a little Chidi in your ear screaming "BUT IS IT ETHICAL???" whenever you try to make a decision, you probably see that there might be a problem here. If we know (= if our theory predicts it and we are pretty sure this is the case) we can reduce sexual harassment at a workplace then is it right to only give this "cure" to the treatment group? Isn't it wrong to withhold it from all the other women in the control firms who still have to deal with sexual harassment? Well, the problem is, that sadly we don't actually know that this policy is the cure. We only know in theory. And that's a crucial difference. Often, economists or other researchers think they know a policy or an intervention works in theory and when they try it out in the real work they actually find that it does not work as they thought (an excellent example here). This is why it's important to even test theories that might seem "obvious" in conventional wisdom with real-world data. Like the idea that female politicians spend more on child-care-related issues. It seems obvious, right? But to know for sure you must test it with empirical evidence. And depending on what you find you can ask more questions such as how does it work or why does it work/not work? 

Back to our ethical dilemma. From the terminology "treatment" and "control" you can already tell that this closely relates to problems in the medical field when it comes to testing drugs that could potentially cure cancer or Alzheimers*. While economists often don't work on such literally life-saving interventions, policies that tackle poverty alleviation by giving people cash can make a significant difference in someone's life. And while the trial in our sexual harassment example ended after only six months, these interventions can often go on for years! In reality, economists usually don't get to put people (or firms) in treatment and control groups themselves, but they try and find scenarios that closely mirror a random approach or they use their fancy econometric methods to control for the factors that could influence outcomes. 

As much as economics tries to be a "real natural" science, economists ought to remember that ultimately much of their work deals with actual, real people. Thus, researchers must walk a fine line between sticking to ethical principals and creating scenarios that allow them to identify causality. 

*Quick detour: Since learning about randomized control trials, how they work, and how crucial random assignment is I always have to think about that one Grey's Anatomy episode where Derek thought he found a cure for Alzheimer's and ran a medical trial and then when Meredith found out that Richard's wife was in the control and not the treatment group she meddled with the trial so that Richard's wife would also get the treatment. Back then when I watched the episode I was "100%" on Meredith's side, she only wanted to help and it was soooo unfair that she got fired. But today I understand just how bad it was what she did, in terms of experimental design but also ethically. Sorry, this episode still gets me like 6 years later. 

No comments:

Post a Comment