Swarms of Data

We are sometimes asked “why did you call yourself DataSwarm?”

The answer is that Memes in their original sense, are unique units of cultural ideas (we covered Memes in some detail in this post). However in practice most memes do not travel alone, they travel in groups (meme-complexes or memeplexes in the jargon). Individual memes continually leave or join memeplexes, and even when part of a memeplex, ther relationships with other memes in it constantly shifts.

Circumcision memeplex

Example of a Memeplex – the Circumcision Memeplex, from the paper by Hugh Young


Over time if you map this on a chart where distance between 2 memes = closeness of association (how often they are found travelling together in the media milieu over time) then they seem to do a “dance”, continually coming together then moving apart, some memes may move away altogether, new ones appear and join. If you map a memeplex over time, all the memes moving around look like a swarm of dots (each dot being an individual memes) moving around some central locus. This is why we talk about Data Swarms

This movement map alone gives out a lot of information, for example:

  • Where the locus is, and the bounded area it describes in its movements allows one to make predictive analyses
  • By observation there are visible clusters of memes within the memeplex (sub-plexes), this is usually veryuseful information own its right
  • The shape itself has some message – average distance of separation as well as various Social Network analysis techniques
  • Typically, in the online media milieu, each meme carries metadata (who, when, where) that can be used to create second-order swarms, which again have informative patterns

In other words, the swarm gives a lot of useful intelligence about itself and its components. More interestingly, is that the swarm sometimes exhibits some form of “intelligence” itself. This is not real intelligence per se, but it is a sign of direction and purpose in the underlying memeplex/es, and (in general) is more predictable en masse than for a particular single meme. This analysis can be monitored and projected forward a bit into the future  and (ideally)  predicted over longer time periods (but this is non trivial – prediction is hard, especially about the future, as they say) At the very least the longer the prediction time, the greater the statistical uncertainty.

CCAPWX epa02379626 flock migrating cranes fly over puszta or Hungarian steppe Hortobagy some 200 kms east Budapest Hungary EPA/TIBOR

Migrating cranes fly over puszta or Hungarian  some 200 kms east Budapest, Hungary (credit EPA/TIBOR)

The above picture of cranes migrating in Hungary illustrate a few of the points. Individual birds continually move towards and away from each other, their positions change relative to each other, new birds arrive and other birds leave. The flock itself moves according to some trends that can possible be predicted (migration direction speed, height etc) and metadata about the types of birds (these are Cranes, they have certain food, habitat and nesting habits) can help add to the prediction of the movement of the flock. There are quite a few types of data swarm (and swarm analysis) with different properties, for more reading its covered quite well in Wikipedia over here