Tuesday, December 20, 2011

Do the Math

While I’ve been having lots of fascinating conversations about how terms like “community,” “network,” and “network-weaver” are actively experienced, to get to the bottom of some precise definitions, inevitably you have to do the math.

Fortunately, I can help you do that, with thanks to Albert-Laszlo Barabasi’s Linked for enlightening me (including with some quantum mechanics and molecular biology which I actually hope not to get into here).

Imagine a group of nodes which are connected by links. Now, here are the definitions:
  • A network is simply a map of these nodes and links.
  • A community exists amongst a group of nodes if there are links between all of them – resulting in there being more links between those nodes than to outlying nodes. 
  • A network weaver is someone who works to raise the number of links within this map of nodes and links (i.e. network).
There is an easy way to measure how tightly-knit a group of nodes is. You simply divide the number of links that currently exist between the nodes by the total possible number if they all were connected – this is called the clustering coefficient. In a community, the clustering coefficient is 1. In increasing the number of links between nodes, network-weaving should raise the clustering coefficient over time.

There, that was easy. Now, what are some of the mathematical properties of how a network connects links between nodes?

Among the first mathematicians who set out to tackle graph theory, Erdos and Renyi, created models where they connected nodes with the assumption that there’s an equal probability that any node will get a link – resulting in a rather democratic and geometrically patterned network. Subsequently, their assumption was proven untrue in real-life networks, as Barabasi documents in networks as diverse as actors in Hollywood and links between Web pages. Here’s why – nodes and links are governed by certain properties:
  • Growth: The number of nodes in a network is observed to change over time, as new ones appear on the scene (and old ones exit). This gives an advantage to those who showed up first – there is a higher probability that they will get the links.
  • Preferential attachment: Nodes actually prefer to link to nodes that already have lots of links (we will call these the hubs). In fact, it was found that the probability a node will link to a second node is proportional to the number of links the second node already has.
  • Fitness: Even given the last two properties, there are nodes that show up late in the game and yet at the end of the day lots of nodes end up having a preferential attachment to link to them (think Google). Why? Each node has a fitness: an ability to attract links compared to the ability of everyone else. Bring on the competition.
Of course, there are many intriguing lessons that can be retrieved from this math. I’d like to focus on one basic one. In their ground-breaking work, Erdos and Renyi seem to have left a legacy of a popular regard for networks as creating the effect of “bringing in the masses” – when you open up a system so that it self-organizes with equal probability that any given node will link to any other given node, who knows what will connect where (and, by extension, who will get the power inherent in the links)? However, as I’ve been saying (and I now have the math to back me up here), this is simply not the case. A network’s links develop according to probabilities that are not equal for each node – they are based on such properties as fitness, growth, and preferential attachment that enable hubs to emerge. That's what makes things interesting.

How does all of this play out in real life, in real time, say, in the Jewish world? How can it be leveraged by hubs and those who wish to become hubs? More on the applications to come – but at least we did the math!

1 comment:

  1. Great post. Of the posts that don't mention me or Ayelet, this is definitely my favorite. I would suggest you linkify it a bit, link to http://www.amazon.com/Linked-Everything-Connected-Else-Means/dp/0452284392/ and http://en.wikipedia.org/wiki/Clustering_coefficient and http://en.wikipedia.org/wiki/Paul_Erd%C5%91s and http://en.wikipedia.org/wiki/Alfr%C3%A9d_R%C3%A9nyi etc.

    Also, I'd define a community as having a clustering coefficient close to one, but not necessarily equal to one.

    I look forward to reading more about this and what you think are the applications.

    Finally, I recommend you refer to the "hubs" instead as the "community shapers" (see http://www.forumblog.org/blog/2011/10/who-are-the-global-shapers.html)