By 2024 it’s predicted there will be 149 Zettabytes of data*.
The emergence of new data, combined with traditional data, has created an opportunity for companies of all industries and sectors to make more informed business decisions and find ways to uncover a competitive edge. However, effectively unlocking these data-driven insights still remains a challenge for many.
In this Q&A, experts from S&P Global Market Intelligence, Snowflake, and Blackstone discuss trends in data management, how Snowflake has helped solve client challenges and more.
*Statista. Information created globally 2010-2024, published by Arne Holst, July 7, 2020.
Migration to the Cloud
Question: What are the trends you’re seeing regarding data management practices?
David Coluccio, S&P Global Market Intelligence: According to 451 Research, it was found that about 60% of enterprises had said they are moving existing on-prem data platforms to off-prem cloud solutions over the next two years.¹
If I was to go back five years, I'd say a lot of our conversations with clients were around utilizing the cloud for disaster recovery purposes. But if you fast forward to today, we are hearing more and more clients moving towards the cloud for production. I think a lot of that has to do with scalability. I think the key being that that moving to the cloud does not have to be a binary switch. Instead, focusing on your data assets and your analytical workloads, while also providing availability for on-premises applications to be connected is key.
I think the idea of moving to the cloud and being able to have portability, not only just across regions, but also to be able to connect to tools that are in the different cloud platforms and being able to have your data seamlessly available, wherever it needs to be, is key.
I think another key reason for that has to do with another core trend in the market today, and that is this explosion of data. Data is now available from many different sources and different vendors. A lot of these new data sources are semi-structured, unstructured and massive in size. As clients are trying to bring in these new data sources to try to gain some additional insight as they merge it with a lot of the traditional data that they used in the past, they're finding it difficult and/or inefficient to actually run a lot of this analysis on traditional on-prem services. And so that is why they are looking towards the cloud to be able to help them analyze a lot of these new data sources.
If we look at the different data sources that are available, the hot trend that we are seeing in the market is definitely around unstructured data, particularly textual data. There's been a lot of demand from our clients around accessing textual data, being able to analyze the text versus what they were previously doing with the numbers. I think a lot of that has to do with Natural Language Processing (NLP) and many of our clients becoming familiar with NLP and analyzing textual data. With that, we've been seeing huge demand in our global transcripts offering.
I think another key trend is around clients trying to push nonproprietary work off their plate. Many clients don't want to manage that process of taking data from the file transfer protocol (FTP) site and loading it into their local environment and then linking that data. As they're bringing in a lot of these new data sources, many of them aren't linked. And they are looking to vendors like S&P Global to link that data and manage that loading data from FTP into a cloud-hosted solution to provide that instant access.
I would say the last trend is around clients moving away from a lot of the traditional tools that they used in the past to analyze data. As I mentioned, the growth of interest in textual data would, historically, require individuals to read through the documents themselves. Today, many clients are looking to utilize newer tools like Python, R, and Tableau, to really improve their workflow and make it much more efficient. And they're looking for vendors to kind of deliver their solution seamlessly and integrate it within those tools.
Matthew Glickman, Snowflake: I'll double down on one, and I'll give you one more. So, what you said, Dave, about not focusing on things that are proprietary, I would say not focusing on things that are not differentiating. I think more and more companies are really focusing on what is their core business is how they can differentiate.
The second thing in your point on data. I think the big trend is so much focused on building, training and leveraging machine learning AI models for predictability. And those models are hungry, right? They're hungry for data. They're hungry for connected data, the mosaic of data that brings together different data sources in a way that can be used to predict behavior, both in markets and customer behavior. The more data you can get into these models -- and getting that data into a format that is consumable is also nontrivial. The more you can take away, the more friction you can remove to do that, the better.
And then lastly, reducing friction overall so that you can try before you buy data sets is becoming the norm. And the more we can work together to remove friction so that you can now get access to data, trial data, experiment with data and reduce what was typically your theory to execution from months to weeks to days to hours to instantaneous, the better.
Thomas Pologruto, Blackstone: I've been doing data and data science for 2.5 decades as an actual scientist back before data science was a phrase that one could use. And I’ve seen a longer-term trend over the last 20 years.
If we think about what the Internet, and what the worldwide web did in getting this insatiable amount of data somewhere that would be ultimately accessible. The way the data ended up being available via the Internet and the worldwide web was truly spectacular.
But it really wasn't until we organized all that data; it was Google that came along and organized the data. And suddenly, that became very powerful. So, you have to have data, then you need an organization of that data to be able to actually do something with it.
And the last trend that happened was the mobile revolution. It's about 10 years ago now. And what that did was it really made information available from the cloud. So, you need all those things: you need data, you need data organization and you need, ultimately, data availability to go do something with it.
The symmetry between the technology we expect in our pockets, in our iPhones and devices, has gotten really, really out of sync with what we do in the workplace. Ultimately, at the workplace you go into the workplace and you're back into a state of affairs that's looked relatively the same for many decades. But there's been many advances on your mobile device, where all of your data just walks around with you.
By pushing all your data into the cloud, you get that ability to make it generally available. So, when we're saying that, it's not only if you do analysis at your desk, but it also puts it into your pocket. I can pull up my iPhone and open reports in any number of BI tools or run analytics right on my phone; we've raised the bar for what it means to analyze data or to do work. And I think that really paints a pretty picture about a great starting point for what the next five years look like for the workplace.
¹451 Research. Voice of the Enterprise Survey – Data Analytics/Data Platforms.
Convergence of the Cutting Edge: Q&A with the Experts
Learn More about S&P Global Data via Snowflake