Blogs

Overcoming the challenges of mining big data

Big data remains of critical importance to every organisation operating today, no matter its size or location. Data volumes continue to rise, says Bill Scudder, SVP and general manager, AIoT solutions, AspenTech. Big data growth statistics from Statista reveal that data creation will be over 180 zettabytes by 2025.That will be about 118.8 zettabytes more than in 2020. Industrial organisations are among those most significantly impacted by data growth.

Making the most of data

The impact is a mixture of opportunity and challenge. On the one hand, big data has an important role to play in arming organisations with the resources and information they need to enable data-driven decisions that can improve business-related outcomes. When analysed properly, the benefits of big data can include optimising production, real-time visibility, and enhanced decision-making, allowing teams to be more productive, effective, and innovative. These benefits can make the difference between an organisation’s success over its competitors.

Indeed, for capital-intensive industries such as manufacturing and industrial facilities, big data is essential to operations. It can help with predictive maintenance, so supervisors can schedule plant downtime to repair assets before unexpected costly breakdowns occur, provide anomaly detection to alert workers to small deviations from the norms of quality, and predict with greater certainty around international supply chain management challenges.

That’s essentially the opportunity. However, the problem often lies in harnessing this data and then making use of it to advance the organisation’s goals. With the Industrial Internet of Things (IIoT) and capital-intensive industries amassing more and more data, they’re having to face the increasing challenges of being unable to manage it or leverage it effectively.

Starting to address the challenge

More data isn’t always better data. With the influx of data that organisations have received through digitalisation efforts, many have found themselves in the middle of a “data swamp” with every piece of possible data included.

We are seeing signs of progress. COVID-19 undoubtedly accelerated the digitalisation of organisations across the world, down to the way they store and access their data. However, this transformation also revealed the limitations of the traditional model of data management, where data is siloed by teams, sources, and locations. This kind of data gatekeeping significantly hinders visibility, as only certain people with unique access or domain expertise can understand or even access data sets that may be relevant to others across the enterprise.

Industrial organisations must switch focus from mass data accumulation to strategic industrial data management, specifically homing in on data integration, data mobility and data accessibility across the organisation. They need to bring together data stored in various silos, often at a range of different facilities worldwide with the goal of using AI-enabled technologies to unlock hidden value in previously unoptimised and undiscovered sets of industrial data. In effect, they need to start moving to an approach based on big data mining.

Debunking the myths

Before embarking on this approach, however, capital-intensive organisations need to counter the myths that have hampered the approach in the past. When it comes to big data mining specifically, one of the biggest challenges organisations face is operating under the assumption that there is a “one-size-fits-all” solution.

This is not true organisations must continuously re-evaluate their workflows and processes for collecting, storing, optimising, and presenting data to ensure they’re reaping the greatest business value from it. This shows up in practice when thinking about auto discovery for example. Many IT leaders believe that there are tools that will auto-discover relevant information across all data, where in fact there is an age limit on what these tools can work with data that is past a certain time frame is usually undetectable, for example.

Another challenge comes from a generational, operational expertise gap. Many organisations are having difficulty finding the right people who have the means and knowledge of where data is stored and what format it’s in. This all circles back to making sure they have the correct data integration strategy in place it makes it infinitely easier on the employees when a set plan has been made and executed on.

Executing on a strategy

Once organisations are fully aware of the myths of data mining and prepared to counter the challenges, they can start on the process of making more of their big data stores.

To help facilitate this change, enterprise IT teams should put a clear strategy in place to ensure that they’re implementing all the proper tools they need to get adequate information from all sources. A big part of this strategy should be to implement a data historian. These tools have evolved, moving beyond standardised aggregations of process data to become the anchor technology for industrial data management strategies. In today’s world, the data historian serves as a democratising force, making it possible for data to be accessed and actioned on by anyone in the organisation.

Next, organisations need to start applying Industrial AI to make data more visible, accessible, and actionable across the enterprise. By building a cloud-ready industrial architecture that connects the latest AI technologies with the IIoT can not only collect and transmit data but also turn it into intelligence to drive smarter decisions.

This emerging confluence of AI and IIoT offers capital-intensive businesses a range of benefits because they can create a sustainable advantage when analysing large volumes of industrial data for real-time reporting, automation and decision making and then integrate it across assets, plants and sites. They can then build on the advantage by visualising data to identify trends, outliers, and patterns to drive mission-critical apps and actionable business workflows and collaborating across AI-powered apps to help achieve safety, sustainability and profitability goals.

The strategic data management approach that this combination of data historians and industrial AI delivers also helps to bridge a growing skills gap. As veteran employees with years of expertise retire, replaced with younger employees with much less experience, an AI-powered, data–driven approach ensures that critical, historic knowledge is preserved and shared widely across the organisation regardless of team, geographical location, or siloes.

A positive future approach

Big data will continue to play a mission-critical role in arming industrial organisations with the resources and insights needed for making data-driven decisions tied to concrete business value outcomes.

This could be about helping with predictive maintenance so supervisors can schedule plant downtime to repair assets before unexpected costly breakdowns occur, providing anomaly detection to alert workers to small deviations from the norms of quality, and predicting with greater certainty around supply chain management challenges.

It could also mean anything from optimising production lines to providing real-time process visibility, all to help teams become more productive, effective, and innovative. But to reap the most value from big data and apply it meaningfully to industrial applications, capital-intensive businesses must switch their focus from mass data accumulation to more thoughtful, strategic industrial data management homing in on data integration, mobility, and accessibility across the organisation. By deploying technologies like next-gen data historians and Industrial AI, these businesses can unlock new, hidden value from previously unoptimised and undiscovered sets of industrial data.

The author is Bill Scudder, SVP and general manager, AIoT solutions, AspenTech.

Comment on this article below or via Twitter @IoTGN