How businesses can create internal teams of data experts
Ben Hemo of Rivery
Data offers huge opportunities for businesses, but most leaders face growing challenges in unlocking the business potential of the data at their fingertips. In part, this is due to the sheer amount of data available to businesses, says Itamar Ben Hemo, the CEO and co-founder of Rivery. In the decade between 2010 and 2020, the creation, capturing and copying of data went up by 5,000% worldwide and reached 59trn gigabytes, according to IDC Research in 2021. But business leaders also face a shortage of talent to make sense of this data, with few around the world.
Data experts come armed with expertise from fields such as information science, machine learning and computer science, and focus on converting big data into usable information in the form of dashboards, reports and visualisations. But new technologies, job roles and ways of thinking about data increasingly offer business leaders ways to take advantage of the data available to them, without relying on a centralised team of data experts to decode.
In a world where data powers everything from cybersecurity to helping companies hit their sustainability goals, there’s never been a better time for businesses to reshape their data stack and rethink how to extract business value from the data available to them. The vision comes to life as teams train their internal talent and hone their skills towards becoming experts who will help their organisation grow.
The changing role of data experts
Data roles are already changing within organisations, with a general move towards making it easier for business users to ‘self-serve’ with data. One manifestation of this is the rise in analytics engineer roles (among many others), which helps to bridge the gap between IT and the data consumers within an organisation.
In this particular case, analytics engineers sit between data engineers and data analysts, building data models for teams across the business. These engineers don’t analyse the data themselves but ensure the data is of high quality and served up in a form that business users can extract value from. The engineers offer companies an easy way to set up and activate the modern data stack helping business users access the data they need.
Upgrading internal skill sets
Cultivating core skills among existing employees will be key to this shift. The burden will fall on business leaders, with data literacy courses becoming the norm among many companies, and large organisations such as Adobe and Bloomberg creating in-house digital academies to teach workers how to read and interpret data.
Rather than hiring in outside expertise, business leaders should seek to train existing workers with data skills (combined with the domain expertise to take advantage of it). Using low-code and no-code solutions, these ‘data citizens’ will be able to format data, and derive business insights from data in their area of the business (for example, sales or marketing) without waiting for data experts to execute the activity.
That’s not to say their importance is insignificant. Data experts and traditional business intelligence (BI) dashboards will still play a crucial role in helping companies plan and anticipate future events. But over time, these data experts will help inform core decisions within the organisation, across almost every part.
The tools to unlock the potential of data
The right technological tools, including data management and tools, are key to unlocking the business value of data. Data management deployments automate several tasks and procedures into one framework, making it easier to store, transform and extract value from data within a business. Additionally, it includes components such as data quality management, where data sets are monitored to see that they meet the needs of data consumers within the company, and data validation, which ensures that data sets abide by certain preset rules. Ensuring that data is in a condition where it can be used by teams and stakeholders within a business democratises data access and removes bottlenecks.
Take ‘Reverse ETL’, otherwise known as Extract, Transform, Load. In normal ETL, data is first extracted from a source system and then loaded into a data warehouse. In Reverse ETL the data flow is reversed and is transferred from a data warehouse to third-party systems such as ERP (Enterprise Resource Planning) or CRM (Customer Relationship Management) software. This strips out bottlenecks where teams on the ‘front line’ of the business have to wait for data experts to deliver reports: instead, the data is delivered directly into the software which they use day-to-day, creating a more ‘horizontal’ data culture within the company. Rather than data being a treasure hoarded within the centre of a company, it reaches every team, helping to operationalise business insights derived from data.
Data as a product and the ‘data mesh’
This ties into two new ways to think about data: ‘data as a product’ and the ‘data mesh’. While ‘data as a product’ is not a new idea, the shift in perspective offers a new way to understand how data is used. Data (the product) has to be in a state where it can be used by the consumer, whether they are an analyst or a salesperson. One of the benefits of thinking of data as a product is that each domain within a business handles its own data pipelines, as compared to previous monolithic top-down ways of managing data paving the way for an entirely new way to use data.
A data mesh is a decentralised ‘self-serve’ approach to data where business users access and query data where it lives within the organisation, without relying on a centralised data team to deliver it. Rather than one data warehouse controlled by hyper-specialised teams, data flows through a ‘mesh’ covering the whole organisation, accessed via shared protocols.
The move towards a true ‘data mesh’ will take years, but companies can’t afford to be complacent about their data stack. It’s no longer enough to be awake to the potential of data; in the coming years, companies will compete on how quickly they can derive business value from it. To do so, companies will need to have the right tools, the right ideas, and staff with the right skills.
By equipping ordinary business users with the tools they need to self-serve, and adopting technologies which help to avoid bottlenecks such as hyper-specialisation, companies can equip themselves for a future in which every employee will be a data citizen. These future experts can help companies make absolute sense of their own data, ensuring they’re consistently driven forward by insight.
The author is Itamar Ben Hemo, the CEO and co-founder of Rivery.
Comment on this article below or via Twitter @IoTGN