Categories: Uncategorized

Big Data Big Science

Big Data is an enormous amount of unstructured data, an amorphous behemoth, usually when people talk about big data, its petabytes and exabytes of data, which is beyond the processing capacity of traditional relational database systems, because its too huge, its fast, and the main point, it has no structure, but still, its important. Big data can reveal insights and critical acumen. It has its uses in analytics, predictive analytics, data visualization and getting real-time relevant content. For successfully acknowledging the values in the data will require a lot of scrutiny and exploration.

As we start, we know the data is towering up and its unstructured and unsymmetrical and it needs to be stored in a way, where it can be operated and scrutinize even in real time. This is where the database and data-ware house solutions like Hadoop, Storm and Greenplum come in.

Nathan Marz, Twitter Engineer and Author of Storm, states, “Storm makes it easy to write and scale complex real-time computations on a cluster of computers, doing for real-time processing what Hadoop did for batch processing. Storm guarantees that every message will be processed. And it’s fast — you can process millions of messages per second with a small cluster.” You can get into more details about the Storm at the Twitter Engineering Blog.

Hadoop which is already quite popular among engineers. The servers do not  share any memory or disk space, these are multiple servers with Hadoop installed, when the data is loaded, Hadoop slices the data and distributes it among the different servers and it even keeps the track of where the data is placed. The most well-known Hadoop user is Facebook.

Once the data is in place and operatable, then starts the procedures of curation, mining, analysis, algorithms and mathematics that needs to be performed on the data. This where the Data scientists and Statisticians come in, the ability to derive the insights from the data, that makes the data relevant. This is someone who can look into the raw potential of the data perform the science and make it pragmatic and advantageous for enterprises, organizations, entrepreneurs and professionals of various different streams and industries.

Finally, understand that big data will reveal a lot of insights, but before that you need to decide, what are you trying to innovate on ? what problem are you trying to solve ? This will make the data relevant and the insights valuable.

Image source: Aviat Networks

Suhail

Suhail is a Startup's guy, an Avid Blogger, Content Writer and Curator. Most of the time found sitting at a wooden desk with a Vaio Laptop and a phone on the side. Tweets awesome contents. You can catch him at his Twitter handle @NafkSuhail or on Email: suhail@nafk.net

Recent Posts

AI goes global: Best universities abroad offering world-class Artificial Intelligence degrees

As we navigate a world with algorithms predicting consumer behaviour, robotic-assisted surgical procedures, and intelligence…

2 days ago

Ness Digital Engineering launches ATONIS: An AI-powered engineering workbench to accelerate product innovation

Bridging intelligence and engineering maturity for a digital-first world Today’s engineering teams face mounting pressure…

6 days ago

War by algorithm: As AI & robotics enter our defense budgets what will our battlefields look like?

As Artificial Intelligence (AI) and robotics redefine the battlefield, how should we feel about it?…

7 days ago

Funding alert: Tech startups that raked in moolah this month

The Tech Panda takes a look at recent funding events in the tech ecosystem, seeking…

1 week ago

AI & technology trends in the glass & manufacturing industry

Walk near the glass facade of any modern building in India today, and it would…

1 week ago

AI literacy for non?tech teachers: A guide to ‘AI for everyone’

Artificial Intelligence (AI) is no longer the exclusive domain of tech experts or computer scientists.…

2 weeks ago