Forgetting about the scalability issues, the brand issues and the marketing challenges in general of human powered search, I'm wondering if these companies will be able to live up to their bottom line value proposition: that humans can provide better results than machines.
Right after we sold Topix, Yahoo came a calling and tried to get us to switch from Google Adsense to their contextual product. Given the success we had with Adsense, this was going to be a tough sell, so they had us come down to Los Angeles, meet with their team and pitch us on their value proposition. At the time (not sure if this is still the case), their claim was that they actually manually build ad libraries for the pages on our site and, as a result, delivered better results.
We decided to take the Yahoo solution for a test drive and ran a test on the product. Actually, many tests. We used the Yahoo ads with various form factors, placements, fonts, colors, etc. Unfortunately for Yahoo, it turned out that their product never even came close to the performance of AdSense on our pages. Turns out the machines did a better job. To their credit, we never had any brand issues with the Yahoo ads (like Sampsonite ads on pages talking about body parts in a suit case washing ashore, or Thailand vacation ads on pages containing Tsunami stories). But from a performance side (CTR, CPC, effectivce CPM- however you want to measure) Google's contextual approach (with the keywords and hints we provided) was clearly superior.
Now you could call this a scaling issue. With Topix having in excess of 400,000 pages on the site, perhaps Yahoo was unable to unleash the full power of its human powered libraries across the site - and therefore was doomed to fail. Perhaps. But even on more generic pages (like sports, entertainment, etc. - as opposed to more obscure pages like Plano, TX news), Google still performed better.
Another explanation is that Yahoo didn't have a large enough pool of advertisers to compete. Or perhaps this was a branding issue - I've always theorized that the "Ads By Google" labeling of the Adsense box made it more likely to be clicked on. The Google brand being there instantly turns those ad links into trusted links, something just labeling "Sponsored Links" can never do.
It's hard to pin-point an answer. But maybe though the machines, without going into solving a problem with the baggage of a particular editorial slant or any pre-conceived notion of a "right" answer, did a better job. Who knows? We'll see how this plays out in search world.