Cruise ships, like the Norwegian Epic above, are getting bigger and bigger. The theory seems to be to that if you make it bigger they will come. For the cruise ship industry, that philosophy appears to be working.
Yesterday I attended a lecture at the University of British Columbia Computer Science department featuring local industry experts on big data from companies like SAP and HootSuite. Big data is all the rage right now with everyone jumping on the bandwagon. I’ve been following the big data movement and was curious of what has changed in the last few years.
While technical infrastructure like Hadoop still requires a lot of expertise and thought, the bigger issues involve business and project management. On the use of big data, the panelists suggested the following best practices:
- There must be a clear question that he organization wants answered with a big data initiative.
- It is easy to infer more from the data than is actually there based on our current knowledge of how to interpret big data.
- A lot of the work is in cleansing and organizing the data. This is contrary to many people’s idea that you just throw the data into a tool like Hadoop and start getting answers.
- There is a great need for statisticians and library scientists to make sense of this data.
- Maximum turn around from conception of a question to providing answers must be 90 days.
In terms of managing big data projects, the rules of any major project are still at the heart of success:
- There must be a clear and measurable objective for the project.
- Milestones must be written down and reviewed.
- If the project goes sideways, management must have the gumption to cancel it.
While data is being gathered in vast amounts that none of us could have imagined five or ten years ago, just having the data is not enough. Bringing the discipline to ask intelligent questions and the tenacity to have the right people and infrastructure to drive to the answers is what it takes to be successful with big data today.
