Then we get our software on these heavy handed structures and run happily for about a year and in most cases six months as new better hardware comes around and our software begin to lag. So we rinse and repeat.
It seems Google is doing this in a bit of different way. The difference is that Google build its system from ground up, on low cost hardware. It is their software infrastructure that scales and manage to run on every possible hardware solution thrown at them.
Google work with the premise that even the best and most expensive hardware have a limit of not being 100% reliable. So they make their software the resilient one.
There is one good story that I read at Google Enterprise blog which explains very well the Google Enterprise architecture. I learned about it of course at Google official blog. You will find the links to both below.
Official Google Enterprise Blog: What we talk about when we talk about cloud computing
Business in the cloud