Fractal Analytics Blog

IoT – Big Data, Analytics and Challenges

IoT – Big Data, Analytics and Challenges

By Archit Agarwal and Praneet Jain
March 17, 2016

“Information is the oil of the 21st century, and analytics is the combustion engine.” – said Peter Sondergaard, SVP, Gartner Research.

With billions of objects expected to be inter-connected within the IoT space, one thing is for sure, there will be an abundance of Information. Imagine, the variety of objects, tracking and exchanging data about an individual and their environment, every second of every minute of every day. Eventually, they will have produced an unprecedented volume of data whose impact will be felt across the big data universe, which intrinsically, has always been an evolving landscape

What is more mindboggling is that Cisco GCI estimates that more than 500 zettabytes (42.3 ZB per month, 1 zettabyte = 1 billion terabytes) will be generated by all people, machines, and things by 2019, up from 135 zettabytes (11.2 ZB per month) generated in 2014. This will be 269 times higher than the amount of data being transmitted to data centers from end-user devices and 49 times higher than total data center traffic by 2019. Figure below shows examples of the amounts of data that will be generated by planes, automobiles, and buildings, among other things and systems (Full report)

image 1

“The whole is greater than the sum of its parts” advocates Gestalt psychology. Onset of IoT connected devices will generate high velocity high veracity data. And the application of analytics on top of this foundation, presents unending possibilities to leverage this data and weave it into a more complete story than ever before.

Advantages of IoT analytics are many folds. From real-time marketing to enhanced user experience, from process optimization to optimized resource consumption, analytics will become the enabler for efficient downstream consumption of IoT data. Let us see some examples where analytics will play a major role and the challenges associated with them –

A. Real – time analytics means real – time insights

Traditionally, data has been stored and analyzed. However, the amount of data that IoT is expected to generate, demands for analytical solutions and algorithms to run in real-time providing ultra-complete results based on the latest data feed available, as opposed to running at regular intervals. The deployment of such analytical solutions and algorithms will have to take place in the cloud, enabling real-time indication of errors and therefore more real-time corrective actions rather than post facto analysis . For e.g., General Electric Durathon battery plant has 10,000+ sensors continuously monitoring humidity, air pressure, temperature and other operating data, providing opportunity to adjust processes in real-time resulting in elevated performance and reduced failures.

To enable this kind of smart scaling, every component of technology will have to fast-forward their innovation plans. Data collection mechanisms will have to change. Moreover, data streams coming in from multiple sources will be tough to integrate. Hardware industry will have to step up a few notches, to keep pace with IoT expansion. It will be interesting to see how other industries poise themselves alongside IoT

B. More accurate results via self-learning algorithms

Variety of IoT data allows analytical algorithms to leverage multi-dimensional view of the real world, to tweak and learn from latest data streams with better accuracy. The streaming data will enable algorithms to learn continuously, thereby creating an AI system which gets better by the second. These algorithms will recognize sophisticated patterns and take intelligent decisions with minimal human intervention

But as someone rightly said, “All that glitters, is not gold”. The challenge here lies in identifying the most useful data. The algorithms will be bombarded by numerous data streams, they will have to develop the capability to identify and attenuate irrelevant data streams.

C. Hyper – personalization of products

Hyper personalized marketing will move to hyper personalized products that will serve every customer based on their preferences. Analytics in IoT will revolutionize the development of products. Companies will have visibility to the daily journey of every customer, identifying individual patterns, which will act as feeds for product development. This will increase the overall product usage and customer satisfaction as these products will already be well – suited to their needs

However, there is a big trade-off here. The minute by minute access into customers’ lives, is a feast for hackers and cyber criminals. Traditional security mechanisms cannot address these new privacy challenges. Access to personal data will eventually lead to identification of hidden patterns and unexpected correlations in customers’ lives which will lead to total loss of self-image and the concept of personal life.

D. Reliable and faster decision making

IoT analytics will revolutionize the way companies react to insights and take action. Seamless data streams and dynamic results will help drive actions, as and when required, hence reducing uncertainty and the turnaround time. Continuous monitoring of vital statistics will help forecast diseases and conditions at a premature stage, reducing loss of lives. This will also help the healthcare insurance industry. Better prediction of health outcomes will reduce the risk for insurers providing more coverages for less

However, due to immature implementation of IoT, it still needs to be seen, how profitable it can be. Most of the smart devices are not yet commercialized. It is good that industry think tanks are making such innovations but it is best if people are able to benefit from these. This demands a huge upfront investment with a promise to prove cost-effective in the longer run.

In addition to these, there are other hindrances to IoT analytics as well. Such large scale integration and deployment will require a uniform set of standards that govern all implementation. Lack of data scientists with enough skills to design analytical algorithms and solutions is also a major detractor. Not to mention, better analytics tools to handle the data, will be required and have to be upgraded as the implementation of IoT gains momentum.

The advantages of analytics with IoT are not limited, and nor are challenges. This is just the tip of the iceberg. IoT provides analytics to be applied in areas where it has never happened before. A lot will surface as the technologies, hardware and software, evolve over time, where analytics will play a centerpiece role in tapping the full potential of IoT in shaping our future

In part 1 of the series, focus was on understanding the concept of IoT and how it will inevitably affect all factions of the society. Building on that foundation, this part focuses on Analytics acting as a catalyst in enabling IoT transform our future




About the authors:

archit v1 Archit Agarwal

Archit is a Senior Consultant, working for one of the largest technology companies in the world. His role primarily focuses on identifying relationships between customer attitude, purchase intent and buying behavior. He is an experienced and accomplished analytics professional with 5+ years of experience in customer and market research helping Fortune 500 clients take informed decisions.
He is specialized in leading engagements involving advanced analytics capabilities for solving complex business problems.

Praneet v2 Praneet Jain

Praneet is an Analytics Lead at Fractal working for one of the largest global healthcare companies. His role demands analytical evaluation and modelling for healthcare process improvement. He is an analytics professional with 2.5+ years of experience in customer and market research helping F500 clients take informed decisions.

Post Comment

Leave a Reply

Your email address will not be published. Required fields are marked *


Institutionalize Forecasting Within an Organization

Download Paper



    • collapse2017 (7)
    • expand2016 (7)
    • expand2015 (43)
    • expand2014 (15)
    • expand2013 (47)
    • expand2012 (15)