Part of the reason is, as I've mentioned before, a sort of singlemindedness endemic to the industry that assumes anything that has worked in the past will continue to do so in the future. If Esteban Fancypants' novel The Art Dealer's Wife becomes a huge hit, he may well end up with a seven-figure offer for his second (or second and third) books. True, it's not entirely fair to blame the industry for this; time and again, consumers have returned to a brand (read: author) they love, and if enough people hear great things about (and later buy) Mr. Fancypants' first book, he's got an established audience for his future titles. Publishers throw money at "sure bets" because publishing has been and always will be something of a gamble, and if there's any indication that leveraging an author with a proven track record can produce more profit, they'll be all over that deal like a monkey on a cupcake.
HOWEVER. Simply because Mr. Fancypants' first novel sold 5 million copies doesn't mean his second and third novels will perform similarly, and for every story of the out-of-the-blue début novelist whose first book sold millions of copies, there's a story of an out-of-the-blue début novelist whose second book tanked. Maybe the second book received bad reviews; maybe it was so different from the first book that it alienated his or her established audience; maybe it comes to light that the author is a terrible person and nobody wants to be caught dead reading his or her book; the number of reasons is potentially infinite. Thing is (and it is fair to blame the industry for this), publishers 1.) perpetuate the blockbuster cycle by driving prices up at auction to patently absurd levels, and 2.) tend to chase losing brands by throwing more money at them long after the authors' audiences have either stopped caring or died.
What, then, can be done about this, mes auteurs?
Well, publishers could consistently include and enforce clauses in contracts stipulating that authors whose books fail to earn out their advances must return the balance of the advance to the publisher, but that's not going to happen unless the industry adopts this policy across the board (not to mention it's not really fair to authors who receive little in the way of marketing and co-op dollars).
I'm interested by this NPR article from last year, which mentions HarperStudio's experimenting with publishing two books a month, neither with an advance of over $100,000, and attracting authors by promising a 50/50 split on the book's profit. While I'm not sure this exact model is a solution, I think programs like it may offer a way out of the boom-and-bust, all-or-nothing blockbuster model that prevails in the industry today.
What say you, gentle readers?