07 November 2012

Observer-expectancy effect, confirmation bias and the 2012 election

Quite obviously, political predictions and projections are not made in a vacuum. We use results from previous elections, polls, and whatever else we can get our hands on to make a best guess about the next time. We make inferences on that data, and we assign more value to some things and less value to others. In many ways, it is indeed subjective. And we often see more data as somehow being better. But sometimes this can be problematic.

As I put together my projection, I tried to be conscious of confirmation bias. In a close election, partisans on either side are likely going to tilt the projection to their side. I'm not necessarily going to discount this data, but I want to look beyond it, too. So while I acknowledged the potential for confirmation bias, going beyond that for more data tended to mentally insulate me from it. The problem was that additional data, gathered from very smart people, wasn't necessarily any better; just more confirmation bias. And as a result, my results were ultimately biased because of a sort of observer-expectancy effect. Acknowledging the initial confirmation bias gave me the assurance that I was on the right track, yet it was that assurance itself that covered up the secondary bias associated with the additional data.

It's an Inception-like bias inside another bias. Weird. At least that's how I see it.

Next time, I'll have to figure out a way to acknowledge confirmation bias without also simultaneously and inadvertently insulating myself from it at the same time.
Post a Comment