> Funny how everything is happening at the same time: yesterday
> we were having a San Francisco P2P get-together, and Travis Kalanick
> reminded me of the similar conversation that we had on Pho list about
> his RedSwoosh enterprise more than two years ago, where people were
> trying to make a very similar case for P2P being "unnecessary" - just
> as what Brian and Clay are saying today.

As the resident Behlendorfian, I'll reply that neither Brian nor I
are saying it is "unnecessary.' For my part, I'm saying a) that much
of the predicted spread of decentralized systems didn't happen, and
b) that many of the disadvantages of centralized systems turned out
to be relatively shallow issues of components and cost, rather than
deep ones of topology or architecture.

Category B is a result of Moore's Law and its variants for storage,
redundancy, etc, but Category A happened in part because the
maximalist case for decentralization was too maximal -- the utility
of decentralization is relative, not absolute.

It happened in part because "embarrassingly parallel" problems are
rarer than we thought, and because outside E.P. problems, the twin
use cases of hiding from the law and lowering bandwidth costs ended
up covering a higher percentage of the remaining use cases than we
imagined.

It also happened in part because the decentralized search problem
turned out to be harder than we imagined, given the early work on
distributed search systems like Chord.

And finally (and perhaps most significantly) end users turn out not
to care as much about issues of local control as we imagined. At time
T+1 hour after the launch of any database-backed service, Marc Canter
sends out mail claiming that the service really wants to be hosted as
distributed but well-marked up data sets, but the advantages of LJ,
YT, and MS have nevertheless persisted in offering features users
prefer.

Now handling E.P. problems is a big win for people who have those
problems (genetics would be grinding to a halt right about now
without decentralized sequencing.) Hiding from the long arm of the
RIAA is a behavior best served by this architecture. Internalizing
bandwidth away from the backbone when you are paying, or
externalizing it to the backbone when the user is paying, is likely
to be a good move for a long time.

These are great things (RedSwoosh falls in category 3) but the kind
of decentralization we were talking about 5 years ago didn't in fact
come to cover a huge area -- it is more like an archipelago of
special cases. P2P is far from unnecessary, but it is also far from
ubiquitous.

-c

>
> Since then Travis got funding, developed the product, and then
> recently sold his RedSwoosh company to Akamai, thus sort of validating
> at least one point out of several that I was making in that
> discussuion.
> But even the remaining points are still something that I would repeat
> today. Amazingly enough, they still seem as valid to me as they
> were on
> November 15-16, 2004, and there's not a single word that I would
> change
> today, YouTube success notwithstanding. (Maybe I'd change some numbers
> a bit, but their precise values are not really important anyway and
> are
> quoted below mainly to provide the conversation context, so I'm
> leaving
> them unchanged, too.)
>
> Though some of these words do contradict what Travis was - and
> still is - doing, I guess that is to be expected. He's a businessman,
> after all. If he'd agree, he'd never sell his business to Akamai :-)
>
> Sorry for the extensive self-quoting, but I figured that many
> people won't remember what was said back then; I sure didn't remember
> much of it myself until yesterday's conversation prompted me to dive
> into the archives. So here we go:
>
> ----------------------------------
> [On non-monetary benefits of P2P:]
>
> "...saving bandwidth is just one of the P2P system features - and in
> the online music space, probably not even the most important one.
>
> The bandwidth cost of a single track is below one cent, so it is
> not really an issue for any setup where you'd have *any* money paid by
> the users at all. Of course, there is also the hosting cost, the
> server
> farm maintenance cost and all that, but these are not very high
> either.
> For me, the most important feature of the P2P system is the easy
> access
> to any data available on the network regardless of its publisher. In a
> sense, I see it as a solution to the publishing problem, not the
> distribution one.
>
> As an analogy, the Web did not create a new way to deliver
> the digital data - Compuserve already had that. What the Web did,
> it created an environment that allowed millions of people to publish
> things, leading to all these eight billion documents indexed by Google
> today, facilitating the Google itself and adding the verb "to google"
> to the dictionary along the way. Managed P2P is about as exciting as
> Compuserve switch to the new data delivery protocol - it is nice, but
> hardly a revolutionary event. That's not why people are excited about
> P2P."
>
> ------------------------------------------------------------------
> [On Grokster touting 'Legal, Licensed' P2P Music Share System with
> a centralized filtering database:]
>
> "Yep. That's a very good illustration. This is Marc Andreessen
> saying: "we have a wonderful new application called Mosaic - it will
> allow you to access the full spectrum of content offered by Compuserve
> to its subscribers".
>
> Or Sergey Brin saying: "we have this wonderful search engine
> called Google that will allow you to search everything on AOL site".
>
> Both would be nice applications, not much doubt about that.
> But we would not have The Web and The Google."
>
> --------------------------------------
> [On Travis' business plan circa 2004:]
>
> "You have a broadband ISP account, you pay forty bucks a month for it,
> and you get your 128 kilobits per second upstream, which happens to be
> 1/12 of the full T1, and so it comes to $40*12=$480 per month for T1,
> which is not a bad deal for your ISP - since it buys bandwidth in
> bulk,
> it would probably make some money even if you'd use your uplink full-
> time.
>
> Which you probably don't.
>
> (Note that downlink is irrelevant - the data transfer rate on the
> network as a whole is limited by the summary capacity of its slowest
> connections, which are these lousy 128 kbit/s uplinks.)
>
> But now Travis enters the picture and says to the ISP: "look, half
> of this traffic can be passed from one node on your internal
> network to
> another without involving the backbone at all - it will be free for
> you,
> since your internal network has higher capacity than your backbone
> connection anyway, and you won't have to pay for passing this to the
> backbone". So ISP saves money, users get faster transfers, and Travis
> puts some butter on his bread. What's wrong with this picture?"
>
> Apparently nothing. Travis, congratulations again! :-)
>
> Best wishes -
> S.Osokine.
> 1 May 2007.
>
>
> -----Original Message-----
> From: decentralization@yahoogroups.com
> [mailto:decentralization@yahoogroups.com]On Behalf Of Clay Shirky
> Sent: Monday, April 30, 2007 6:26 AM
> To: decentralization@yahoogroups.com
> Subject: Re: [decentralization] The decline of P2P and
> Decentralisation
>
>
>> Further, the premise that "certain kinds of apps are simply so big/so
>> bandwidth-intensive/so latency-sensitive that only decentralized
>> architectures
>> can work" has become quaint. Google has made distribution of
>> centrally-managed
>> data look effortless;
>
> I was thinking about Google, in relation to this thread: they're
> interesting, because they have three separate topologies at work --
> in front of the webserver, there is the familiar broadcast topology
> -- Google to users -- and at the server, there is the usual load-
> balancing for the index.
>
> Behind the webservers, however, Google File System is a decentralized
> service that dwarfs what most of us were talking about in 01-03. For
> Google, decentralization is a feature, not a goal, so while
> predictions of utility of decentralization at scale were right,
> predictions that the end users machines would always be part of the
> mesh were wrong.
>
>> Finally, there are classes of functionality that people have come
>> to expect
>> from modern web sites - let's take tag clouds as an example - that
>> would be
>> very difficult to do if the data were truly decentralized. At the
>> very least
>> an index needs to be kept; and if that's as large as the body of
>> content
>> anyways, it's not that much harder to host the data/apps directly.
>
> I got beat up a few years ago for suggesting that Napster was a
> better architecture than Gnutella on every axis except legal risk; at
> the time, decentralization was assumed to be like chocolate - more is
> always better, and there is no such thing as too much. Similarly,
> LiveJournal, and later MySpace, are able to do things with their
> respective indices that the Canter-inflected vision of 'everyone
> hosts their own data' can't reach.
>
> The decentralization pattern is so obvious for both processing and
> storage that no one notices it as a special thing anymore, analogous
> to what happened to OO programming.
>
> It still makes sense for IM, which is, by number of users, the
> largest decentralized use case, probably by a couple orders of
> magnitude.
>
> It still makes sense for risk mitigation, as with Gnutella and legal
> risk, or Groove being used in war zones so there will be no servers
> to attack.
>
> But I'm a Behlendorfian on the question of overall utility -- the
> general case of 'aggregate cycles/storage' has been commodified by
> Moore's Law-style economics so quickly that supply outstripped demand
> long ago. Most of the decentralization is going on inside the
> corporate world, as with GFS or grids, as opposed to involving end
> users directly.
>
> -clay
>
>
>
>
> Announce or discover P2P conferences on the P2P Conference Wiki at
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
> Yahoo! Groups Links
>
>
>
>
>
>
> Announce or discover P2P conferences on the P2P Conference Wiki at
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
> Yahoo! Groups Links
>
>
>