INSIGHTS FROM THE DATA EXPERTS’ PERSPECTIVE

Speaking of the current insights and trends, and whether they foresee any potential paradigm shift that could radically affect the big data ecosystem, the majority answered they don’t anticipate any paradigm shifts in the short term.

However, several data experts mentioned that a paradigm shift/disruption won’t be what will influence the big data ecosystem. Yet, it will be influenced by the growing adoption of IoT, Edge Computing, ML, 5G, also, traceability and other novel value models linked with Blockchain/DLT, for example. They believe that the combination of these technologies or its commoditisation will accelerate data availability and enable real-time (or near) processing which will foster the creation of novel businesses, and business models, especially by combining sources/domains and distributing value across the value chain. That being said, it is evident several domains (e.g., health, pharma, fintech, insurtech, media) are eager to use big data to support wiser decisions.

Moreover, data experts mentioned that data security might drive a considerable paradigm shift in the big data ecosystem. It is noted that the ecosystem misses standard and reliable policies to safely manage data that are spread in a heterogeneous cluster of digital devices. For instance, the data expert expects huge advances in the following years coming from the field of homomorphic encryption.

Ecological transition, carbon taxes, Green IT, sustainable development and other energy-related matters were also mentioned. Nonetheless, the opinion is that the ecosystem is still at a point where current approaches and technologies have to finish maturing and start to be used massively.

Data Value Chain experts were asked if they had noticed any signals of disruption and what those were. In a nutshell, there are several opportunities under the topic of Big Data, especially related with traceability, trustworthiness and monetisation, and from the combination of multiple domains.

It is mentioned that disruptions could start even from the first part of a data value chain, which is data creation. Data is produced but still is not defined if we exploit all possible data sources or if we create and collect really valuable data. Furthermore, other disruptions may be caused by security, privacy and governance issues related to data.

Speaking of data creation and data sharing, there are two cases: companies that don’t have data because they cannot produce it but need data and it’s difficult to access third-party data. And on the other hand, companies that have the data but do not create correct data spaces with governance and do not exploit the data value. Moreover, it is important to recognise data sharing as a new dimension in the value chain, and that definitively adds a lot of implications to the process.

Nonetheless, some experts notice that the main problems within the data value chain space are the uncontrolled growth of stored data (data lake) and the concerns about GDPR and Data Privacy. From a technological point of view, the never-ending growth of data has pushed the performance boundaries of legacy data storage and management technologies. This has driven the emergence of new technological solutions that sometimes go against well-established principles (e.g., NoSQL or data denormalisation).

From an industry point of view, the disruption could come from the emergence of a plethora of data-driven companies and business models. Some data experts see a huge chance that nobody is leveraging, which is a company including data analysis in their own business as a mandatory and regular activity, to support decisions and efforts. However, businesses working in the data space find the governance and creation of processes to check the quality of data as the main issue for further disruptions.

Data experts were asked to share what are the main barrier(s) that they see to participating in Data Value Chain, and what stops people from investing themselves in the Big Data domain.

The main barriers mentioned were data sharing, data decentralisation, data ownership, and data privacy. In addition to the limited availability of valuable data, lack of interoperability in data and systems integration is also an issue, since there are no established working data spaces, only some first pilot cases/examples. Also, there is not an established app store/marketplace for solutions that will enable in a standardised way the data value chain creation.

Some experts stated keeping the status quo as one of the barriers since business users do not want to change their minds easily and to be data-centric. Moreover, the fear that making part of the internal workings of a company visible (process, clients, etc.) can make them lose competitiveness and differentiation.

Important barriers, such as lack of standards and lack of sectorial glossaries shall be addressed swiftly since there is a lot of jargon used, no standardised bureaucracy and ambiguous concepts around what’s expected from data experts.

When asked if they have noticed any signs of opportunity within the data value chain space, it was mentioned that clearly for some years the perception of the data/information/knowledge has increased with the growing adoption of AI. It allowed decision-making support in wider domains and due to computational power commoditisation, its inclusion in real-time operations. Several companies are growing due to this combination of data availability and usage and AI.

One of the opportunities stated is porting every private cloud to AWS or Azure Cloud, which can result in better data sharing. This can lead to data experts being able to experiment in a safe environment, sharing data, accessing data from others, and developing applications using the data. Nonetheless, this has to follow the awareness of the industry (mostly small players) about the importance of data, and new regulations (e.g., GDPR, Data Governance Act).

Data experts also comment on what opportunities real-time applications offer, and that they are able to leverage data in a shorter time and in a more meaningful manner. Moreover, there is growing attention regarding the cost of data and enterprises need to gain value from it. Besides, there are more and more open-source software, use case examples and pilots that can exemplify and encourage more companies to enter the world of data spaces.

In a nutshell, in order to seize said opportunities, DVC stakeholders need to know and understand the various data value chain stages (data collection, curation, storage, analysis, cognition, visualisation, etc). Furthermore, they should understand the business perspective of an application in order to create solutions that create actual impact for the end-users.