Asking the Stupid Questions Since 1971
After discussion of how to make aggregators' fetches more efficient, Dave Winer supported conditional GETs using ETags in Radio Userland, and Brent Simmons is working on it for NetNewsWire. This, and some of the changes made for redirection, simply make aggregators better HTTP user agents. How the user agent should react under certain conditions is specified, while in other cases a facility is provided that enables other behavior.
For all the talk of avoiding wheel invention, this industry does it constantly. Mark Pilgrim notes the similiarity between aggregator behavior and that of the push clients, such as PointCast. Streaming media sites have problems with excessive connections overwhelming both their bandwidth and their server farm: one solution is multicast; another is Akamai. Security patches and other updates generate a lot of traffic. Behind Microsoft's Windows Updates is the Background Intelligent Transfer Service (BITS). Software distribution sites have mirrors, and now BitTorrent.
There is more than one way to do pretty much anything. But sometimes all you need is to learn what has been done before.
And what's the worst thing that happens? Huge corporations that make owners and shareholders rich will cease to be a viable economic model and lightweight coops of engineers and product specialists will become the norm. [Chuck Shotton's Logic Faults]
c.f. Red Mars