India startup mining ‘dark data’ to sell to businesses

Indian startup Quantta Analytics is mining data from, well, pretty much everywhere. Text messages, images, videos, e-mails. You name it, Quantta has data on it.

Along with this, they are also mining dark data — that is, information found hidden away in the deepest crevices of the deep web, and selling it to corporations all over the country. They are also setting up an office in Silicon Valley – the tech star of the world – so will soon be able to bring this same information to the States.

Currently, Quantta have about one hundred clients and, whilst it is mostly financial institutions benefiting from this, they have also shared the data with retailers and restaurants, such as McDonald’s and Starbucks.

It’s also possible that, in the future, intelligence agencies will be able to get their hands on this in order to fight terrorism, and other crimes.

The deep web often gets a bad rep, what with its associations with illegal weapons trade, drug markets and sex trafficking. In this respect, Quantta could actually end up saving lives. They could put an end to needless suffering of innocent people. As mentioned, there is the potential for intelligence agencies to utilise their data to track and pinpoint areas in which these crimes occur.

If its predicting software is accurate, we could also see a kind of Minority Report-esque situation, where people are caught before the crime has been committed. But we’ll have to wait and see.

The dark web contains 99.97% of all information available on the web. The remaining three percent, known as the surface web is what we find whenever we type a query into Google. Although the results still only count for 16% of the surface web.

Popular Science has compared this to “fishing in the top two feet of the ocean – you miss out on the virtual Mariana Trench below.” — source

The majority of information on the dark web is password protected, or needs specialist software to be able to access and view it. Quantta is able sidestep this and is looking to mine the 44 zettabytes [yes, zettabyte] of information that is projected to be on the dark web by 2020. If that sounds like a heck of a lot, it is. For comparison, in 2013, the digital universe contained just [ha] 4.4 zettabytes of data.

So how, exactly, is this going to be useful? Well, according to founder Ritesh Bawri, companies will be able to buy and use this data to tailor their products for the consumers. Stores can already see how many people are in their store, what they are buying and how long they are staying there.

But if a company wants to grow, that isn’t always enough. By working with Quantta, they will be able to see potential customers as well. Quantta’s software not only analyses the store, but also the surrounding area, this is done in areas of one square kilometer

Of this, Bawri says “I will add data about what else is happening in the world around those Nike stores. I can tell the company that around this store, where you clocked sales of a million Indian rupees, are potentially 500 customers who have not bought from you so far.”

Quantta’s website states that they focus on Location and Predictive analytics in order to help companies be prepared for any eventuality. They also look into Personalisation and look to understand customer behaviour to further satisfaction for their clients.

This data will be important in predicting human behaviours. It will help companies get ahead of the curve, to predict trends, see what’s in style, and what is not. Instead of focussing their attention to just their stores demographic, they will be able to look at the demographic of the area in order to further push their products.

So, it sounds like a very effective tool for businesses. But what about the everyman? The guy on the street searching for flowers for his wife, or looking for birthday presents. Privacy in technology is a huge issue nowadays. The use of VPNs and Tor browsers have shown that we don’t really like our private searches being collected and analysed.

Whilst Quantta may not know the whos, they certainly know the whats and the hows. In the long run, the average consumer may benefit from this, eventually. Think tailored products for customers.

In the meantime, it’s primary function is to assist corporations become more effective in assisting their customers. And from the looks of it, consumers will always be below businesses in the hierarchy of beneficiaries.