Data teams have long championed consolidation, standardisation and governance. This was often very challenging and time-consuming work. In seeing it through, data teams laid the foundations for more accurate reporting, dashboard proliferation and advanced analytics on an enterprise-wide basis.
But for every data team or professional that’s earned praise and plaudits for their work, there are those that are effectively victims of their own success.
By realising value early, small teams of data professionals became de facto centres of excellence. A pipeline of ideas to execute was quickly assembled from across the business, but due to a variety of factors – including the high cost of, and competition for, talent – scaling up the size and capability of the data team to meet the rush of demand for data services was difficult.
The effects of this were felt by portions of the business.
Those who contributed ideas to the pipeline, only to see those ideas assigned a lower-than-expected priority, started to view the data function as a bottleneck to their ambitions. Backroom discussions – without the data team – gained momentum. Pitches to executives along the lines of, ‘Having the data team build a dashboard is slow and costs x dollars, we can shortcut all of this and reduce cost by using this tool instead’ ensued.
Circumventing the data team became attractive – and data democratisation was touted as the answer.
What data democratisation is and isn’t
Data democratisation has its merits: it promises users the ability to self-serve some of their most common or straightforward data needs, which can go a long way to resolving resource constraints and growing pains felt by centralised data teams.
But there’s also a tendency among some users and business functions to interpret this empowerment too liberally.
A common interpretation of data democratisation is the circumvention or removal of the data team altogether and the provision of Excel or a low-code/no-code dashboard assembly application so users can self-serve all their analytics needs instead.
Giving users carte blanche access to everything solves the problem of the data team being a bottleneck. Data access is also technically “democratised” – but in doing so, the enterprise winds back decades of foundational work.
Data democratisation is not a data “free-for-all”, and allowing the business to self-service their most common data requirements should not come at the cost of abandoning long-established best practices.
Data democratisation should instead be thought of as empowerment to leverage data that is compiled using best practices that the data profession has built up over time. Democratisation is about bringing the tools, culture and skills of data science to a broader array of people, rather than trying to reinvent the wheel on data access.
When the lived experience of data professionals around education, tooling, culture and other supporting tasks is ignored or overturned, democratisation can become its own bottleneck – and the business quickly finds it is no better off.
Lessons learned from data democratisation’ misses’
The foundations that data professionals put in place around structuring data, standardising key definitions across business units or functions, governance and security were put in place for a reason.
Data preparation, such as cleaning, integrating, transforming, simplifying and formatting, has long been the most time-consuming task of data professionals to improve the accuracy and efficiency of any subsequent data analysis and decision-making processes. While the situation has improved over time – going from 80% in 2016 to 39% in 2021 – this still represents a significant chunk of time. It also shows this work is not set-and-forget: as new data sources emerge, systems evolve and internal needs change, ongoing maintenance effort is required.
The extent of this behind-the-scenes work is often not fully appreciated or understood widely across the business by those seeking unfettered access. It’s often only when they encounter problems with source data that it becomes apparent why best practices that were built up in this area over time remain critically important to data access and usability.
When confronted with that reality – of the effort needed for data preparation and upkeep, the complexion of those pushing free-for-all data democratisation quickly changes. They want access to data to get faster turnaround times on analytics and reports but do not want to deal with preparation issues – of incorrectly formatted data, data migration and other tasks that the central data team would ordinarily take care of.
The realisation sets in that to achieve their vision of data democratisation, they still need the support and structure of the data ecosystem set up by the data team. Once that realisation is made – and all of the business is aligned on what they want to achieve and the building blocks required to achieve it, Data Democratisation can make progress.
Data democratisation holds immense potential for businesses across all industries. By dismantling data silos, fostering a data-driven culture, and empowering employees with the necessary skills, organisations can unlock the true power of their data assets.
However, the journey requires continuous learning, collaboration, and a commitment to responsible data practices. As businesses embrace this shift, they position themselves to make informed decisions, drive innovation, and ultimately achieve a sustainable competitive advantage in today’s data-driven world.
This article was originally published on 3 May, 2024 in ITBrief Australia.
Want to understand how we can help your business with its data needs? Contact us