The Necessity Of Failure, and how science advances because of it...

Wired ran a great article this month on the science of science, namely how scientists bring their own biases into their research, and often fail to recognize how unexpected outcomes in their experiments can offer the real solutions.

Full article

It's hella long, so I'm not gonna post the whole thing here, but it's very good. Here's some of the money quotes...

Please read the full article, this was only a tease. It cites a great example of not seeing the forest for the trees in science, and how it blocked a great discovery. I'm betting that this article was prepared before the full implications of Climategate, but the relevance should be obvious.
Science has a grand history of not seeing the real implications of scientific outcomes, which isn't a failure of science, but a failure of the humans behind it. After all if you're professional success depends of proving certain outcomes, you're going disregard data that repeatedly tells you otherwise.

Plus if you follow the link there's an awesome article on how Duke Nukem Forever failed due to it's own success, but you gotta follow to get the link.

This is general stuff about science. But I'm betting you won't use this rationale to try criticise any other scientific theories.

Kuhn is controversial too. Popper is more mainstream.

While I wouldn't deny much in the article I am surprised that this is being treated as 'new' information in some way. The history of science is littered with examples of missed opportunities or fortuitous discoveries, the big bang one is just one of the best examples.

However, I'm not sure it is catastrophic for science in anyway. People just need to be aware of these dangers. Having said that it is perfectly sensible to assume that if the experiment didn't work then there is probably a problem – because often there would be. After all your aim is to test whatever you set out to test so you should always prioritise checking that you are actually testing what you think you are testing. I read a similar story to his ‘two balls falling’ thing in a book on probability. Can’t remember all the details but in short a guy asked a set of doctors (having already done it with members of the public) about the probability of getting certain example symptoms. One example was common, one was uncommon and one was both at once. The vast majority of the doctors indicated that they believed a patient was more likely to get both symptoms at once.
Of course that cannot be true (unless the conditions were related in some way) but they thought the probability of getting a rare symptom and a common symptom together was higher than the probability of getting a rare symptom alone. Because it FELT more correct.

You ya'll admit that the CRU scientist who commented "what a tragedy it is that we can't show warming" was in fact misguided by his own bias and desire to see warming trends?

That was easier than I expected.

Huh?

Way to miss the point.

I am implying that you have no serious interest in this subject except as a stick to beat AGW with.

So what?

That's like saying "you have no serious interest in tools except as a lugwrench to replace your tire with".

Or "asking about the virgin birth is nothing but trolling"?

Surely this is common knowledge. Very few important or revolutionary theories (etc) have been discovered by people in their forties or older.
Newton - 23, Einstein - 26, James Watson - 25, Darwin - 29, their ages when their most revolutionary discoveries were made.

The problem nowadays is the sheer quantity and years of learning required to come to terms with a modern research field (as well as the too-structured nature of science in western culture)

Yeah, well, we know what the taboo topics are around here.

Yes, exactly: None.

Oh, just another global warming bash is it

True. But unless society works out an unbiased way of funding science, this has been and will always be the case. And then there's public perception, fed by media frenzy and bolstered by an essential ignorance of scientific method. Whattaya gonna do?

This is the most interesting part of the article to me:

Certainly that applies to each side in the AGW debate. Of course, there's also the metaquestion - why would someone who is skeptical of the short term scientific process parse that information itself as accurate. The answer is easy - it matches my own observational data, and I've got nothing which contradicts the hypothesis at this point. I don't know its correct, but like so much of science I don't have the time to research it in anywhere near depth.

As far as AGW goes, I'm a lukewarmer - I don't think the premise that 'the CRU group cooked the books and CO2 absorbs outbound radiative forcing' are in any way mutually exclusive.

However, within that context, I'm a HELL of a lot more skeptical about the science, especially considering the remarkable vested interests pushing this thing.

No, it's a good article regardless, it's only extremely timely in that some of the failures listed fall right in line with the apparent behavior of CRU scientists. Excuse me for being topical.

Well sticking with the global warming meme, this would be extremely easy to solve. If the government is going to fund pro AGW scientists, it could fund some nonAGW people as well. As politicized as AGW has become, governments have a vested interest in proving AGW, as it has all sorts of policy implications if they can prove it. Tossing some money to the other side, of which there are reputable scientists, would go some short way in correcting bias.

Or cut public funding altogether.

The beauty of science is that we get the unexpected, even when we're looking for something else. Hell, that's half the pharmaceutical industry right there.

J.

So the article confirms that scientists as a whole are not doing science any differently to the previous 500 years, and that, as has always been the case, the most radical and important breakthroughs are made by individuals outside the mainstream. (Unfortunately) now that mainstream is larger and individuals and radicalism is less encouraged, but that doesn't just apply to studies of global warming. Surely we should have faith that - as we have to assume is the case - as part of the natural course of man's learning of a field, the "truth" will be made clear over time, just as it was with natural selection, the quantum, etc.

The massive amount of public funding is probably part of the reason for the reason for the large-scale nature of modern science nature, and the apparent difficulties for individuals to revolutionise (although this has often not been easy in the past), but the questionable funding discrepancies in competing reasearch areas is definitely not just restricted to GW, and powerful individuals in science at reasearch centres are as much to blame as the government.

Not bad points Fox. If breakthroughs come from outside the mainstream, that's what makes the politics of getting published in journals, and the conspiring of CRU personal so disgusting.

Einstein spent a great deal of time starring at data that should have led him to The Big Bang Theory years before it was first proposed... but he rejected the entire idea because it implied a specific "moment of creation" and how it seemed too... theistic.... and it was actually a catholic preist, Ftr. George LeMetra that actually put the theory forth. The Pope at the time loved the Big Bang Theory.

You get flipper babies.

Yeah, I always bring that up in the 'religion is innately anti-science debates.'

It often has been, but there's been some great scientists that were religious, even clergy.

These people employ doublethink to agreeable effect.

No, they understand that the role of science is limited. It's not a substitute for spiritualism, it has nothing to say on the matter. Hence the two concepts are perfectly agreeable when utilized together.

I used to not understand why such a concept was hard to understand, then I read Evenflow's article, and now I understand the biochemical process in the brain that keeps people from accepting such obvious truths.

Depends what 'innately anti-science' means. Anyway, one person isn't evidence of anything. And neither are all the scientists from 200 years ago. After all religion was pretty much a given then, you weren't really choosing to be religious.

The biochemical process that keeps people from agreeing with you is the fact that NOMA is a crock of shit. The domain of science is everything in existence. And religion makes many scientific claims.

LOL, yes, I know you regularly masturbate to Dawkins.

Stephen Gould, the National Academy of Sciences, and the vast majority of humanity disagree with you.

For those that aren't aware of the concept and what Rick is explicitly calling a crock of shit:

NOMA stands for Non-Overlapping Magesteria. From wiki:

I'll let folks draw their own conclusions on that one. Obviously Rick and I disagree stringently over that statement.

Absolutely, completely and utterly false. Pathetically, embarrasingly false, and clearly the reason why you fail to understand these points. And hypocritically false, because it makes claims on science that it doesn't take for itself and elevates science itself to the ultimate authority.

The domain of science is everything that humankind can observe, measure and recreate. It's why two of the key tenants of science are falsifiablility and prediction.

Science is in no way, shape or form the laws of the universe.

Science is mankind's attempt to understand those laws. And because the laws are seen through all to fallible humans eyes, often our understanding is imprecise and in some cases distorted by our own desires.

Hell, we've seen less than a fraction of a yoctabyte of the universe. Claims that we've definitely ruled out the concept of divinity and spirituality are absurd, and one that the vast majority of scientists would rightfully decry.

Science can never state definitively X is impossible. It can only say 'we have no data at this time that supports X.' That's a huge differentiation.

Some, and in the realms where it makes scientific claims that extend beyond it's magesteria then clearly it's overstated its bounds.

However, the scientific theory that the Earth is 4.5 billion years old contradicting young earth Creationism has very little to say on the matter of spirituality or the existence of the divine. Hell, you could make a good argument that the Bible itself doesn't even support YEC.

I'd also certainly argue that science can overstep its magisteria as well.

And I think the written traditions of religion often have something to say about supposed historical accuracy (in this case, a science). Troy was considered a myth after all.

And of course religion and spirituality are hardly the same thing, only aspects. I'd put religion and philosophy in the same magesteria, and intelligent inquiry into one often leads into the other.