The economics have changed: thanks to digital typeset and paperback printers, "out-of-print" could now just mean "special order." And by "special order," I mean "wait 15 minutes for the machine out back to finish whipping up your book."
Or, put them in the supply chain. Eight of these machines, costing about a million dollars, replace a lot of warehouse, especially because it means your entire back catalog is never out of print.
But, a lot of publishers don't do this. My conspiracy theory runs like this:
Publishers put up a lot of money to bring a book to print: editing, initial print run, marketing, maybe an advance. (Yeah...) They then add in a percentage for their profit, call it a loan, and take the royalties until that loan is paid off.
That means books go through three stages: 1) they earn back the publisher's investment, 2) profit goes to the publisher, 3) the publisher splits royalties with the author. Finally, if the book is very old, it's in a fourth stage: anyone can print it or download it from Gutenberg/Google Books/etc., the market is competitive and the publisher thus only makes normal profits.
A lot of books peter out in stage 1. This is a risk of doing business, and is (hopefully) made up by profits on successful books. Stage 2 is pure gravy. But, them damn classics hanging out in stage 3 are just clogging up the market.
Do you think the publishers are going to be wild for something that lets books live forever? That's why we still have "out of print."