Microsoft Excel in the age of Big Data

How are traditional data tools adapting to this bold new era of BIg Data?

Let’s face it, Big Data is one of the most overused buzzwords today — but hype or no, Big Data is real and it’s powerful. The ability to quickly crunch large volumes of data in real-time and generate results truly transforms how advisers, analysts, investors and businesses work. Which got us thinking — how are traditional data tools adapting to this bold new era? We’re thinking specifically about a program that we’ve all used: Microsoft Excel.
Savvy analysts and investment managers swear by Excel, and with good reason — there’s a lot of to love about it, and it’s certainly one of our customers’ favorite tools. Unfortunately, their giant spreadsheets don’t quite qualify as Big Data (even if they do sometimes clock in at multiple megabytes). And while no one can dispute Excel’s usefulness, the program has some traits that lag behind recent innovations in data science.
True Big Data horsepower requires capabilities in all of the following areas:
Exploration
Today’s Big Data horsepower can generate insights and analyze patterns with an efficiency and “aha”-inducing elegance that was once impossible. In contrast, while Excel is a great calculation and analysis tool, it’s not really built for creative querying or automatically highlighting notable trends.
Structure
Excel is old school in that it’s purely two dimensional (rows and columns). With today’s databases, data can be organized and assessed along multiple dimensions, using a number of intersecting variables. As data becomes easier (and cheaper) to work with, the notion of manually processing data in a 2-D grid will become increasingly difficult for users to swallow.
Data Visualization
While Excel allows users to build charts, that process is often extremely painful. Mislabeled axes or garbled data have caused us all chart heartburn; and worse, the resultant graphs often look clunky or cheesy. Today’s advances in technology and techniques around data visualization mean there’s no reason to suffer graphics that aren’t easily customizable, intuitive, and beautiful.
Fat Finger Protection
Entering massive amounts of data in Excel is simply mind-numbing. Oftentimes, one wrong input can mess up your whole model, leaving you scouring multiple worksheets for the single guilty typo. Today, that data extraction, aggregation, and checking/cleaning process can be automated … and should be!
Learning Curve Efficiency
Excel can do a lot, but odds are that you don’t know how to take advantage of most of it. The level of training and outright dedication that Excel often demands is uniquely daunting. With everything we know about user best practices and interactive data design today, these barriers to usability seem unnecessary and downright counterproductive.
Don’t get us wrong, we’re Excel fanatics. In fact, Excel is open in the background as we write this (two books, twelve sheets, vlookups as far as the eye can see). But we’re also big believers in Big Data, and in making data science as reliable and accessible as possible.
As best practices evolve — and the role of software in investors’ and analysts’ workflows grows – the odds are that the next must-have tool won’t look all that much like Excel. Instead, 21st-century analysis tools will do more to guide investors to notable patterns in data sets, automate complex calculations, and bridge the gap between dense number-crunching and at-a-glance, multi-dimensional insights.
Lowell Putnam and Niko Karvounis are co-founders of Quovo, an investment insights company that reimagines sophisticated portfolio analytics and data management tools for investors.