Are business schools intellectually bankrupt?

From the New York Times today, a report from business school professors concerning a study which claims to show that professors are prejudiced, too. I remember reading the original paper on this study, which made it painfully clear that the authors were pursuing an agenda and that they arrived at their conclusions by scouring their data for correlations which supported their case, a classic hallmark of poor science. But perhaps sound methodology is too much to expect from business school professors?

The reason I paid particular attention to this study was that I was one of the participants. Here was the original email I received.

Dear Professor XXX,

I am writing you because I am a prospective doctoral student with considerable interest in your research. My plan is to apply to doctoral programs this coming fall, and I am eager to learn as much as I can about research opportunities in the meantime.

I will be on campus next Monday, and although I know it is short notice, I was wondering if you might have 10 minutes when you would be willing to meet with me to briefly talk about your work and any possible opportunities for me to get involved in your research. Any time that would be convenient for you would be fine with me, as meeting with you is my first priority during this campus visit.

Thank you in advance for your consideration.

I remember receiving this email. What immediately struck me was the repeated vague references to “my research.” Now in order to have any appreciation of my research, you would, at the very least, have to know that that Langlands program exists, or that my research is related to Wiles’ proof of Fermat’s Last Theorem and algebraic number theory. The fact that there is no mention of number theory nor any indication of the background of the student immediately links the email (in my mind) to academic spam. Surely it’s the case that my reaction would be shared by many academics? Who is so desperate for attention that they would imagine this email reflects a genuine personal interest in their work? As you would expect, I completely ignored the email and promptly forgot about it. Then, a week later, I received the following email:

Dear Professor XXX,

Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph.D. program (the body of the email appears below). We are emailing you today to debrief you on the actual purpose of that email, as it was part of a research study. We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. We want to thank you for your time and for reading further if you are interested in understanding why you received this message. We hope you will see the value of the knowledge we anticipate producing with this large academic study.

We are decision-making researchers interested in how choices differ when they are made for “now” versus for “later”. Previous research has shown that people tend to favor doing things they viscerally want to do over what they believe they should do when making decisions for now, while they are more likely to do what they believe they should when making decisions for later (for a review, see Milkman, Rogers and Bazerman, 2008). The email you received from a student asked for a meeting with you either today (if you were randomly assigned to the “now” condition) or in a week (if you were randomly assigned to the “later” condition). This email was actually from a fictional student. It was designed for a study of the responsiveness of University faculty to meeting requests from prospective students of various backgrounds made on short notice versus well in advance. Faculty members at the top 260 U.S. Universities (as ranked by U.S. News and World Report) and affiliated with Ph.D. programs were identified as potential participants in this study, and a random sample (6,300 faculty in total – one per Ph.D. program) were selected to receive emails. In addition to examining the responsiveness of faculty to meeting requests for “now” versus “later”, we are also interested in how the identity of the applicant affects, or does not affect, response rates, and as such, the name of the student sending a meeting request was varied (by race and by gender). We expected that students from underrepresented groups would receive fewer meeting acceptances than other students, though we have competing hypotheses about whether this would effect would be stronger in the “now” or the “later” condition.

I love the line concerning the fact they the have “competing hypotheses about whether this would effect would be stronger in the “now” or the “later” condition” — see, they prove their case whatever the data!

Given the ridiculousness of the initial email, I was appalled that my response might contribute to some published data implying that professors were dismissive of minorities and/or women. Let me be clear at this point that it may well be the case that academics are more dismissive towards women (I fear that this may indeed be true in our field), but I am convinced that this study would have little of academic value to say on the matter. That said, having received this second email, I did nervously look back to my initial email to see what fake name I had failed to respond to.

The answer: Steven Smith. Yes! I ignored the name that could easily be a white male.

Update: For a different take on this study by Andrew Gelman (who, apparently, is more willing to spend time answering random emails than I am) see here, here, and here.