Earlier this month I attended Gene Leganza’s keynote at the Infogix ICON conference. Gene, who is a VP and Research Director at Forrester Research), discussed the top trends in data management, and how that is driving the evolution of data governance and data quality as a team sport.
Obviously, Forrester’s Research is heavily led by US and EU trends, and here in South Africa we are often a few years behind.
Yet I have noticed some of these trends emerging strongly in South Africa too.
What are the top data management trends?
A by-product of the increasing shift to digital business is the variety if data that must be accessed an analysed. Gene calls this trend that of connected data. The challenge here is not necessarily volumes, but rather the complexity of finding and integrating all the data needed to inform decision making, AI models and the like. Data cataloguing and data preparation are two areas where modern data management capabilities are necessary to deliver these connections.
Self service analytics has been a trending topic for years, but definitely hot at the moment. We have a number of clients where the ability to support self-service is a primary driver for data warehouse modernisation programs and the like. Data governance and data quality are critical enablers to build trust in the self service data stores. This is to lower the barriers to entry by making it easier for decision makers to find, understand and trust the data they need for analytics. A related trend is the trend towards Artificial Intelligence – leveraging data and machine learning, for example, to make decisions. The old adage, garbage in, garbage out, remains extremely relevant to AI and delivering quality training data sets remains a key focus of data governance and quality.
Another trend that is very hot in South Africa is the shift to hybrid / multicloud environments. From a governance perspective organisations must grappel with data movements and data security, on-premise and in the cloud, to optimise their data architectures and comply with data regulations.
Tightly related to cloud is the trend towards fast data – typically real time data streams. A key challenge here is to manage failures in the data stream, for example the intermittent loss of a network connection, without loosing data.
Cost reduction and faster time-to-value seem to be ongoing trends, not just for data management, and certainly very relevant in a South African economy very heavily impacted by COVID-19. Data governance platforms are intended to make it easier to understand the impact of data changes, making it easier for IT to adapt system to evolving business requirements.
Gene also discussed the emerging/aspirational discipline of DataOps – a collaborative and interdisciplinary approach to data management that is practiced by a small group of data leaders. Crowdsourcing data governance means that the right decision makers and subject matter experts are involved at the right time to ensure that the best data is found, that the right context is understood, and so on.
Gene spoke to the reality that cross functional teams are rare in business. Most of us work in functional silos that are well suited to the operation delivery of our job requirements. However, data driven leaders are leveraging data governance technology to create virtual teams that break down silos and deliver enterprise level insights. Definitely an area that is slowly emerging and hopefully we will start to see these cross-functional teams building in South Africa.
It is interesting to note that many of these trends are interconnected – in other words the trend towards self service BI, for example, is fed by the need for both faster time to value and cost reductions.
Which of these trends are affecting you?