On This American Life this week, is a podcast devoted to the subject of “So Crazy it might just work.” Now if you are interested in scientific discovery, its culture and openness, go and listen to the first half of that podcast right now. I’ll wait.
OK, all done? I’m not going to discuss the prime number story although that was also interesting. Instead, I’m interested in quite gripping tale of cancer researcher Jonathan Brody who was approached by his old music teacher Anthony Holland who had an idea about killing cancer cells with sound waves. It was an out of the box idea but as you could see from the podcast, it turned out to be less crazy than one might have supposed.
Of course, the idea doesn’t quite pan out. Well, to be sure, the experiments conducted do not lead to any conclusion one way or the other. Holland becomes frustrated by the standards Brody communicates to him and eventually appears to stop trying.
However, it turns out he didn’t do that. Holland believed in his idea and was not bound by academic standards and norms. So he chose to post his approach and results on the net. The exchange that followed with Brody is instructive.
He starts showing Jon his new website for his new nonprofit company, called–
Anthony Holland Novobiotronics Incorporated, “the future of biotechnology research,” and a lot of dramatic statements.
Gabriel Rhodes Apparently, for the past year, Anthony’s been forming this nonprofit company and building the website, which has tabs for all the different research he hopes to do with the device. Not just cancer, but also Lyme disease, malaria. He doesn’t get into the malaria or any part of that with Jon. But he does show him the pages and pages where he explains, in detail, the cancer experiments they’ve done, without mentioning Jon’s name or the name of his lab, since he doesn’t have permission.
Anthony says his plan is to use the website to raise money so they can continue the experiments. I thought Jon might be angry. Why hadn’t Anthony told him that this is what he’d been doing all year? But Jon’s reaction is–
Jonathan Brody This is amazing. I wish that all the students or the graduate students or people or the scientists that believed in what they were doing took this sort of initiative.
Anthony Holland Wow. That’s nice to hear.
Jonathan Brody I think it’s a beautiful website, Anthony. I think it’s great. I think my only concern is, again, which is my overall concern of where things are at, is you have to be really careful here, that, really, a trained eye or a trained scientist is going to immediately question the overabundance of data without the controls and without the reproducibility.
Anthony Holland So let me respond to this idea that intelligent and expert trained scientists, if shown this data, might dismiss it for lack of more information on the controls. And here’s where I’ve been saving the best information for last, for a sort of dramatic and controversial revelation. I’ve had three important scientists in the field, in the field of cancer research, contact me about my work. They have seen this data. They have seen all of our data.
Gabriel Rhodes This changes the conversation entirely. Suddenly, Jon’s reputation is on the line.
Jonathan Brody I would never give that data out to another scientist. It wasn’t controlled. That’s highly inappropriate behavior that you did, that you actually went behind my back and you sent this to other researchers, where I allowed you to use my credibility and my lab to do this work.
Anthony Holland OK, so here’s my response. First of all, if there was some breach in protocol I made by sharing some preliminary data–
Jonathan Brody Why didn’t you ask me? Why didn’t you say, “Look, someone’s contacting me. Do you mind if I share some of our data together?”
Anthony Holland Could I finish without being interrupted?
Jonathan Brody Yeah.
Anthony Holland If I broke some protocol, I apologize. You never told me, “Keep this data secret and don’t show it to anybody.” It just was given freely–
Gabriel Rhodes Even though Anthony promises he didn’t mention Jon’s name to these other scientists, if one of those scientists were to do a quick Google search, in a few clicks, he could get to Jon, who would then be associated with data he would never want public.
Anthony Holland I’m sorry. I mean, to me as a composer, it’s like if some violinist says, “Hey, do you have a piece for a violin?” I say, “Well, I have a sketch. It’s nothing really great, but I have a little start.” “Well, let me see what it is.” “Sure. I’ll send it along.” So it’s the same kind of thing. And when people contacted me–
Jonathan Brody You didn’t send it as a sketch, Anthony. You sent them data that is as if we had done the proper controls. That’s false advertising.
Anthony Holland Jonathan, in one case, I met for hours with the director of a medical research hospital in the cancer division. And I explained to this doctor exactly, exactly what we did. She knew it was preliminary. She knew there were holes and things that had to be nailed down. She knew all the details of everything, looked at the data, and said, “Yeah, let’s go. Here’s a seven-page contract.” So I’m just saying–
Jonathan Brody I just wanted to be part of the conversation.
Anthony Holland You asked me to respond. No scientist would take my preliminary data seriously. I’m sorry, that’s wrong. And I think you need to make up your mind. Either this preliminary data I have is not worthy of another scientist’s consideration, or you’re mad at me for showing it to another scientist who considered it worthy.
Jonathan Brody It’s not. Anthony, we’ve had discussions. And you were mostly concerned about–
Now this incident is instructive because it gives us some insight into the issue of openness in science. As regular readers know, this is something I am actively concerned about and also I have pointed previously to the open science movement — the case for which is currently most powerfully put forward by Michael Nielsen (see his TED talk just posted here). It has also been put forward in a TED talk by Jay Bradner that specifically deals with the case of cancer research.
What Holland did was precisely what the open science advocates recommend. He set up a website and then actively engaged in communicating the idea to other scientists. The podcast does not really illuminate the outcome of that and neither did the website. But the point is that he did not see why the idea should be kept hidden and acted to do the opposite.
Interestingly, he does not get push-back from Brody on that general concept. He is quite supportive of the website. But talking with other scientists is another matter. Despite assurances, Brody becomes very concerned regarding his reputation. Put simply, Brody had assessed that the results were not ready for academic prime time. Not enough investigation had been done and so there was nothing to communicate. From Holland’s perspective, that was immaterial because without funding and external interest the project seemed dead in the water. Why not disclose whatever it was they knew? Again, this latter belief is entirely consistent with the open science movement and especially the idea of open notebooks.
Now the podcast does expose Brody’s issue. He is concerned for his reputation. If he is seen as over-selling these results or selling them at all, it will poorly signal his adherence to academic standards. Not to mention the relatively unconventional risk he took in collaborating with an amateur. Of course, knowledge of that has now been widely communicated as has the whole idea here — something Brody had clearly assented to.
But what of Brody’s reputation was really at risk? At least the podcast makes clear that he stuck by standards and enforced them rigorously. So there is no risk there. What I suspect is that there is a broader issue scientist’s face: their reputation depends on their ability to garner attention at the right time. Take up other scientist’s time with stuff you could have investigated yourself and you lose an opportunity to take up their attention when you really have something. For Brody, the communication of results to other scientists was therefore worse than the public story itself. It took up their attention and perhaps he was concerned it would reduce his ability to gather interest for other current or future projects.
Nonetheless, all of this just highlights the subtle interactions that occur in science and that the open science movement has its work cut out for it.