The open source movement can cast a wide net…
It’s easy for us to fall into the trap of thinking about open sources initiatives as simply displacing traditional computing categories like operating systems, browsers, and application suites. After all, there’s a certain ‘David vs. Goliath’ drama in defining the open source movement as just a of a bunch of ‘rag-tag’ programmers working to bring down the Microsoft empire. But to think of it in those terms sells the transformational nature of the movement short.
At it’s heart, Open Source is about creating software that is technically superior to commercial offerings by leveraging the knowledge, passion, and creativity of an open development community. It embraces a competitive and, to a certain extent, Darwinian ethos. Only the best ideas, innovations, and code to emerge from the community survive. The rest are rapidly pruned, creating a purity of focus not typically found in the more politically based decision making of larger commercial organizations.
And that is the basis for it’s success…
However, open source development initiatives seem to be missing one critical component that will eventually need to be addressed.
An open source alternative to hosting…
I started to think about this when looking in to open source alternatives to Twitter. There are several initiatives underway in this area, but none of them address the fundamental limitation of Twitter – namely that it is incredibly expensive to scale it using a centrally hosted model. But if a new approach were taken – if every member of the network could share the load of message storing, routing, subscription, authentication, and even front end delivery – these scale issues could certainly be minimized. What would be needed is a framework to efficiently federate these tasks.
There have been a few successful examples of this already…
From a distributed computing perspective, there was an initiative by SETI (the Search for ExtraTerrestrial Intelligence) that created a “screen saver” that would let them leverage spare CPU cycles on people desktop computers. SETI works by ‘listening’ to radio frequencies from space to try and find non-random patterns that might represent the actions of an intelligent race. (at least one that developed TV and Radio) Since SETI never had much in the way of funding and an enormous computational task to do, farming work out to the general population was a brilliant solution.
From a routing and connectivity perspective, BitTorrent is a great example. Unlike files on traditional P2P services, a BitTorrent file is broken up into small chunks. If you want to download a file, you are able to simultaneously grab whatever chunks are available from others on the network in whatever order they become available. The more people that participate, the more efficient the distribution platform becomes.
So what’s wrong with using the current open source approach?…
While open source produces fantastic code, many of the most interesting applications and services people are working on today demand an operational footprint that is beyond the reach of anyone but the largest corporations. The code might be completely open and free, but it’s still shackled by the bandwidth and computing costs required to deploy it in a meaningful way.
This is a discussion that goes way beyond replacing Twitter…
The entire participatory/social framework of “Web 2.0″ depends on achieving scale. And if we want to create open source alternatives in domains like web search – especially looking forward to the Semantic Web – we need to be sharing more than just code.
SETI’s approach was fine for it’s time, but building application specific distributed frameworks isn’t the way to go. I believe next frontier for open source should be the creation of an “Open Host”. It will need to include a broad range of standardized services: identity management and security, encryption, sandboxed code hosting, message routing, data hosting and management, etc. Developing this is clearly a massive task, and would need to happen incrementally over time.
But it is a task that I believe will eventually need to happen…
A necessary predicate here is that people in the community could still choose what services or initiatives they would let their systems be used for, so the competitive nature of open source will still be preserved. The best, most promising projects will get the resources and the rest will be niched or simply starve. This could also change the nature of venture funding, requiring fewer dollars and focusing them more on valuable IP and less on infrastructure.
I have a lot more thinking to do to flesh out this concept into something more concrete. With that in mind, I would love to hear from anyone about initiatives similar to this that might already be underway, or insights you may have into approaches that could be taken here.
There are certainly a lot of spare computing cycles connected to the web.
This could be a great way to leverage them…