Choose your language:
Everyone has heard of Big Data and its lesser-known sibling, data sciences. But you may be surprised to learn Big Data existed long before the advent of the Internet and has, in fact, been with us for a very long time. The Weather Service is the best example of a well-known Big Data; when they report a new record-breaking weather event for a particular day, they draw on records that stretch back 200 years. Big Data in its current buzzword form might be all about how and where tons of acquired data is stored and the tools used to parse it, but true Big Data is much more than that.
While Big Data may not be new, the techniques to store and analyze it are. The Internet has added new sources of data and made the acquisition, gathering and analysis of it relatively easier. But the real value lies in using those large amounts of structured and unstructured data, either from local or disparate sources, acquired or gathered on demand, and analyzing it for patterns to provide some kind of meaningful answer to a known question—or even one yet to be asked.
Think the cloud, Apache Hadoop, MapReduce, Python, Java, MongoDB: all tools used by the IT community and data scientists to store, disperse and analyze the petabytes of data that's been acquired over X number of years. If you want to learn more, Hilary Mason has done many great presentations on Big Data and data sciences; you can view one of her presentations on YouTube for an idea of what's possible.
I’ve gathered some interesting links on Big Data and some great ways it's used today and could be used in the future.
Due to the large amounts of data that today’s corporations are collecting, Big Data requires an interesting set of skills and opens new avenues of work and challenges for today’s IT professional, which is sure to keep things interesting for the near future of IT.
Don Vaillancourt is a lead software developer at TEKsystems’ Montreal Solution Centre specializing in Java and PHP. In his spare time, Don geeks out about Big Data and Cloud technologies and programming languages. Stay tuned for Don’s blog posts about what’s trending in tech.