In a recent blog, Robin Bloor discusses how parallelism is required for software to go really fast on todays multicore computers. He brings up this point about MapReduce: using MapReduce on problems it wasn't intended to solve is "... like playing golf with a single club". I'd like to expound a bit on this analysis.
MapReduce was most famously implemented by Google to fulfill their need to index the world wide web. Quite an undertaking! And MapReduce proved to be critical to their success. The programming model for MapReduce fits perfectly with the problem of finding words within documents and creating indices for later (very fast) lookup.
However, the MapReduce programming model can be limited when applied to other, more complex problems. Many deep data analysis algorithms require multiple, complex steps to produce their output. In these cases, a more general use programming paradigm is a better and more efficient fit. Hence, Robin's analogy to "... playing golf with a single club".
Robin goes on to discuss DataRush and the capabilities it brings to bear. Based on a dataflow architecture, the programming model of DataRush is much more flexible and general use than MapReduce. I wouldn't use DataRush to index all the content of the internet, but it has proven to be an excellent tool for general data processing and data mining. And it has the ability to utilize all of the cores available on today's multicore systems. Put into perspective, we have benchmarks showing Terabyte an hour (and even better) processing of network log data for a cyber security application on a single box.
So does playing golf with only one club mean you can't play golf? Of course not. But it does mean you can't play as well or as efficiently as if you used all the clubs at your disposal. The flexibility of DataRush allows you to utilize a full programming paradigm especially suited to big data problems.
Robin has a knack for the turn of a phrase. Check him out on Twitter. He also has a very funny (and informative) book out called "Words You Don't Know". I especially like the chapter on swear words.