Using Real-Time Data to Accelerate Digital Innovation

Redis – Splunk

MICHAEL KRIGSMAN: How does data drive, make possible, enable digital transformation? We’re talking with Shawn Bice, the President of Products and Technology at Splunk, to dive into this very important topic. Before we start, a huge shout out to Redis for making our conversation possible. Shawn, it’s great to talk with you. How are you today?

SHAWN BICE: I’m doing great. Thanks for having me.

MICHAEL KRIGSMAN: Shawn, tell us about Splunk.

SHAWN BICE: Well, Splunk is helping customers all around the world turn their data into doing. We’ve been in the industry now for almost 20 years. So when you hear of companies that have a big data problem, they really need to get insights out of that data, I get a chance to work with the builders that are connecting with those customers each and every day so they can take their data and actually turn it into a solid asset.

MICHAEL KRIGSMAN: Shawn, you’re President of Products and Technology at Splunk. What does that role encompass?

SHAWN BICE: Yeah, so when you think of the people in my organization– this includes product managers. These are the folks that sit down with customers and really take the time to understand their requirements. The whole engineering organization is here– so from software design to build to all the operations in the cloud. So basically all of Splunk’s products get built and operated by our team.

MICHAEL KRIGSMAN: And it sounds like you and your team spend a lot of time with customers.

SHAWN BICE: Oh, yeah. Yeah, no, we spend– everything we do works backwards from our customers. So we are with them every single day.

MICHAEL KRIGSMAN: Shawn, let’s talk about data and digital transformation. Data is not new but has become this very alluring and popular topic. What’s going on to have brought data to the forefront, as we now see?

SHAWN BICE: Yeah, well, data– there has been an explosion in data systems. What I would say is, data has gone from a record of what has happened into the most vital information that you need to make decisions. So if you kind of just think about, in the past, when people thought of data, it most often was just, hey, that’s a record of something that happened.

Today, a lot of people have so much data from so many different systems. These businesses are trying to operate in almost near real time. So really, what has happened is now data really, truly is the most vital information for an organization in terms of its decisions that it makes.

MICHAEL KRIGSMAN: You described a shift that’s taken place from data being a kind of record keeping system to data being strategic. Can you elaborate on that and also weave in the technology aspects of this change?

SHAWN BICE: Yeah. Well, the technology aspects of this change is huge. When you think the data industry, if you go back to the early ’70s, from pretty much 1970 almost all the way up to the year 2000, if we were to ask somebody what their data strategy was or what they were doing with data, most of those people would have built an application on a relational database system.

So pretty much every scenario that you could think of was built against a relational database. And then as we got closer to the 2000 era, all of a sudden, all new types of databases have come about, like a key value system or a time series database or a document database.

And today, there’s over 380 different systems tracked by dB engines, if you use those as a proxy. So really what’s happened is we used to have this one size fits all database for almost three decades, and today it’s really these purpose-built systems that are very specialized for a given access pattern. That’s the world we’re in now.

MICHAEL KRIGSMAN: Has the change in technology been the driver of this new kind of use of data, or is it the business needs that are creating this? Or do those two work together?

SHAWN BICE: They do work together. But I would tell you there’s maybe a bigger emphasis now on the types of things people are trying to do with data. So for example, if you and I were building an application 20 years ago, there was no such thing as Lyft or Uber or Netflix. So if you and I were thinking about scale, we were probably thinking about our network, like how many users were in our enterprise connected to our network and what environments were they in. That was the scale footprint.

But today, you look at the type of applications they’re building, like think of Snapchat. Who knows when the Chicago Cubs might win the World Series and millions of people pick up a device and want to take a selfie? But when that happens, your infrastructure better be ready to go. So the type of applications that exist today, they are far more scalable. Many of them now have global scale. The speed requirements is off the charts.

You meet people today– they’re talking about millisecond latencies, where in the past, it might have been seconds. You hear people having from one to hundreds of millions of users anywhere around the world. So that kind of footprint, that kind of scale is really what’s behind a lot of applications that are built today that look nothing like they did 10, 15, 20 years ago– and the technology that has evolved to support this.

MICHAEL KRIGSMAN: I’m also old enough to remember that when building an application, especially in a startup, meant buying expensive servers and the people to run those servers, whereas, today, you spin up a cloud instance.

SHAWN BICE: Yeah, yeah. [LAUGHS] I remember those days too. Yeah, I’m glad you brought this up because the world today is known as fully managed APIs. And a lot of people will ask, well, what does that actually mean? And when you think about 10, 15, 20 years ago, setting up a data environment was hard. You’d have to get hardware.

You would stack these machines, provision them, and then if you had multiple of them– I mean, it was a lot of undifferentiated heavy lifting that way. Or if you remember sitting in design meetings for nine months planning out what you’re going to buy– and the world we live in today looks nothing like that.

If you and I have an idea, with the advent of the cloud and all of its wonderful characteristics, one of the best things that’s happened in the cloud is a lot of these service providers offer these database platforms that literally are fully managed. So to you and I, it’s just an endpoint. We can connect to it start, coding against it. We can scale inevitably.

If we had a bad idea and needed to punt– no big deal. We didn’t buy anything. So it is a dramatic change. This is why– 20 years ago, if there are 380 different types of databases, forget it. But this is why that exists today, because they really truly are exposed just through a simple API with nothing to manage.

MICHAEL KRIGSMAN: And the impact on business innovation is tremendous because it’s so much easier and faster to spin up an instance and have the tools you need, the databases, like you’re describing.

SHAWN BICE: Oh, absolutely. Think of the pandemic. And I have heard so many customer stories where the general story sounded like, hey, this changed. We need to serve our clients in a very, very different way, and it needs to happen in the next month. And that same project before the pandemic might have taken a year or two. And I hear story after story of these companies that literally have had to change on a dime, reinvent themselves, create whole new experiences, and they’ve done it in the matter of months, which is pretty darn remarkable. But I would, then again, point out the reason they’ve been able to do it is by and large because they’ve had a really good data foundation, and they’ve taken full advantage of the cloud.

MICHAEL KRIGSMAN: Well, you have not used the term digital transformation, but really, that’s what you’re talking about. And so what is the role of data in digital transformation?

SHAWN BICE: Yeah, great question. To tell you the truth, I don’t know that everybody even knows what digital transformation means. Sometimes I think people hear these words and they don’t necessarily think about it to the point where they go. oh, yeah, I know exactly what it means. They’re trying to put it together.

I often answer this by just using an example that we all know. Disney is a wonderful company– 97-year-old enterprise. Disney Plus a great example of an old enterprise reinventing itself and creating a whole new customer experience in less than a year.

And I always use this as an example because before Disney Plus, if you had said to me, hey, imagine just staying at home on a Friday night and a major motion picture– you don’t even have to go to the theater– it’ll just be streamed over your television, most people might have laughed at that. But you look at the services and how these companies have– that is a great example of a digital transformation.

And how does data play in that? Well, all the quality of service, how fast those bits come down to you, if you have latency, are you able to skip through chapters the right way, are you pausing– if you pause, do you restore? Is audio coming through? So imagine all the telemetry that comes off that system, and basically they just want to make sure the quality is good.

Or another fun digital transformation example is in Formula 1. So these cars have been around racing for a long time, but today, these cars have 300 or 400 sensors on them. They are collecting data from everything that’s happening– the engine, the wheels, the steering, brakes, acceleration. They have so much data on the car.

But as you know, in Formula 1, a blink of an eye can be the difference between winning and losing, and in this case, they are heavily relying on data. They have like 900 engineers back in– McLaren does, back in England, that are processing this data during the race and making adjustments in real time. That would be another really good example of a digital transformation and how data is playing a huge role in Formula 1 today.

MICHAEL KRIGSMAN: How much of this business shift rely on data for transformation is real time data?

SHAWN BICE: Yeah. Yeah, I mean, a lot of it is becoming more real time every day. If you think back, what would be an example of real time application, real time data? I think one of my favorites would be stock trading. That’s a very real time thing. You’re measuring down to milliseconds for trades.

But the type of real time applications that exist today– I mean, you see them springing up everywhere. Take for example a rideshare. When you take your phone out and you order a car, most people are looking at that phone to just track the driver to see exactly– or a food delivery service. You can literally track the car all the way to you.

That kind of thing wasn’t– that type of requirement, that type of app didn’t really exist 10 years ago. So if you think of every application that you have on your phone where there’s some type of real time activity happening, that just gives you an idea of how common it is becoming. So in some sense, I would say, in the past, real time– maybe a few applications. But today, it’s really becoming– quickly becoming a part of the norm.

MICHAEL KRIGSMAN: I think that customer experience is a very important part of digital transformation. And as you’re talking, I’m thinking that as consumers we have come to expect this type of real time interaction, which, of course, has major implications for the type of applications we’re building and the infrastructure and the data itself.

SHAWN BICE: Oh, yeah. I mean, think about how many services sit behind some of these massive consumer applications. You and I might know it as an email service– or if you’ve ever use Zoom for video conferencing, it’s just a video service.

But think of the pandemic happens– before the pandemic, had you said to me, every student in the world tomorrow could be video conferencing for education, or the entire workforce is now going to be remote worldwide– imagine an event like that. Nobody’s thinking something like that could ever happen.

It happens, and of course we as humans figure out how to navigate through it. But in reality, what you have going on is literally, fulls-blown scale going out– just massive scale. And you look at a customer like Zoom– think of how much complexity is behind that system. And now students just need it to work so they can interact with a teacher.

If you and I were having a work meeting, we’d need the audio and the visual to just work. We don’t really care what’s behind the scenes. But now somebody like Zoom– they need to have the right set of tools so that they can observe that entire environment. You can think of so many parts in that environment– no human being could process it all in their mind.

That’s why tools are so important today. But those tools help you observe it all, pinpoint those problems. They’ll tell you what’s happening, why it’s happening, how to remediate it. So these systems are bigger and more complex than ever before, thus the tooling that is available today now is pretty darn important.

MICHAEL KRIGSMAN: Very different from traditional data and analytics, say, for reporting on historical data or historical events that simply don’t have a real time component.

SHAWN BICE: Oh, yeah, yeah. I think of– jeez, when I first started, a lot of– I think reporting was a big deal, and reporting with visualizations was a big deal. And putting a piece of paper on a table with a pie chart or line charts and when it was in color– that was a big thing.

You look at a productivity worker today, they’re looking at multiple dimensions of data on the fly, touching screens, in full context moving through entire business workflows by the touch of a finger. So yeah, the world has changed quite a bit.

MICHAEL KRIGSMAN: Shawn, several times you’ve mentioned speed meaning performance and scalability. A lot of that relies on infrastructure. So how important is infrastructure in general and also to you at Splunk?

SHAWN BICE: Yeah, I mean, infrastructure– or you think of a data platform. One of the most important things today is that customers really do have a strong data foundation. And sometimes people say, hey, what does that actually mean, to have a strong data foundation?

I like to put it in plain English, so the way I would say it is, if you’re trying to create a new customer experience and you constantly hear, no– like, hey, we need to recommend something, or we need to check boxes– our shipment every second down the assembly line. Whatever it is that you might be doing, when people come back and go, no, we can’t do it, or we can’t get that data, or it’s not possible, that’s probably a good sign of not having a strong data foundation.

But on the flip side, customers that have a strong data foundation, they say yes a lot. They come up with new ideas that they weren’t even thinking about that morning. And when they go and say, hey, could we make a recommendation here, or if we wanted to build a new video streaming experience– they often find a way. Oh, yeah, that’s another fully managed API that’s part of our architecture.

So in that context, these foundations are critical. The customers that I’ve seen really thrive through the pandemic– they’ve had a strong data foundation. The folks that I’ve seen struggle– they don’t. They often are constantly bumping into no. But at the end of the day, with that strong data foundation, you really can turn data into doing. That’s how we like to talk about it at Splunk.

But at the same time, a lot of customers fundamentally need to understand– they need to observe that whole infrastructure. They need to make sure it’s secure, so at the end of the day, their whole systems are constantly up and running.

MICHAEL KRIGSMAN: So what is a data foundation? For example, you have a data foundation that you use at Splunk. Can you drill into that a little bit for us?

SHAWN BICE: Yeah. So when people talk about a data foundation from a customer’s point of view, at the end of the day, a lot of customers today are trying to reinvent themselves or create new customer experiences. Signs of not having a strong foundation is when you’re constantly told no. In any idea you have, any experience you’re trying to build, it’s constantly like, oh, we can’t do that. We can’t do that.

Customers that find a way where they say yes a lot, like here’s a new experience. You’ve never done it before. We want to go build this. And their architecture or their data foundation allows that to happen. That’s a really good sign.

But from a Splunk perspective, Splunk is all about helping customers turn data into doing. The last thing you want is data sitting around and you can’t do anything with it. It’s all about getting the maximum value out of your data. So that’s one aspect from Splunk. The other is just the basic fundamentals of making sure you can operate your environment and keep it secure.

MICHAEL KRIGSMAN: All right, so we have our business goals. We have our data foundation. What kind of team needs to be in place in order to do all of this effectively?

SHAWN BICE: Yeah, this is a great question. And I usually answer this one pretty privately in a meeting. And what I would say is, as a lot of people are trying to modernize their technology, you also need to think about how you’re going to modernize your workforce. There’s one thing to always think about here is remember, in data, there was kind of one way of doing things from almost 1970 all the way to 2000.

So for that many years, people just kind of got used to doing data one way. And you don’t want to let familiarity stifle innovation or become a blind spot. So I always encourage folks– really, really do not let yourself fall into that trap where it’s like, oh, that’s how we’ve always done it, and that’s the way it’s going to be.

I see this happen because a lot of folks have had people on their DBA teams or in their IT departments that have been there for 10, 15, 20 years. And sometimes those people have so much institutional knowledge, it can be great. But if somebody like that doesn’t really want to embrace some of this new technology, you really could find yourself stuck in the past. That’s what I mean by, don’t let familiarity because a blind spot that stifles innovation.

So you really, really truly want to be very intentional on how you modernize your workforce. Because when you do, you find these change agents that– they’re the ones that will go figure out a new graph database or a time series database, or they’ll explore ledger. And then they’ll compose it into your data architecture so when your businesses start coming up with new applications, those are the folks that are often saying yes. That’s why you absolutely want to modernize your workforce.

MICHAEL KRIGSMAN: I’ve also seen business leaders, very experienced business leaders, who say, yeah, that data you’re giving me– I know it’s not right because my intuition about this is always spot on. So how can a data team member deal with that kind of historicalist approach?

SHAWN BICE: Yeah, I’ll use a Formula 1 example. So Zac Brown is the CEO of the McLaren Formula 1 team. And we are having dinner together just a few weeks ago. He’s shared this story publicly, so I can do it here. But one of the drivers on the McLaren F1 team– his name is Lando Norris. He was racing in the Russian Grand Prix. He has not won– finished pull– he has not finished in first place on a race yet.

And he was leading every single lap of this Russian Grand Prix. There’s about five laps to go in this race, and rain starts to move in. The rain clouds are coming in. And the data strongly suggested that McLaren should have pulled into the pits, changed the tires to wet tires or tires that can handle wet roads, and they would have had a very good chance of winning this race.

But they did not follow data. They actually followed that gut emotional, we’ve made it this far. We can finish. And then it started raining, and next thing you know, this car couldn’t even stay on the track. And then the Mercedes team was right behind Lando. They followed data, pit, changed tires, and they won the race no problem.

So what Zac will tell you is they should have followed data. And in this case, he believes they would have won the race. And that doesn’t mean your gut can’t be right. Oftentimes, your gut can be right. But boy, oh, boy, was that a lesson learned for them on just trusting that data and following it.

MICHAEL KRIGSMAN: I think it takes time to develop a kind of data culture where confidence in the data is pervasive and we feel more comfortable submitting to the data rather than trying to fight the data.

SHAWN BICE: Yeah, and I think a lot of this comes from that old world of spreadsheets. So you remember the day where you would get a customer list, and it would be attached to an email? And then you’d mail it off to another person, and before you know it, this email with this spreadsheet– is that the source of truth and what I should be making decisions on? So where I’m going with this is governance.

And today, there are so many different data systems. And where I’ve seen companies struggle– or they don’t have a lot of confidence. When you really sit down and look at the details, what you’ll find is they actually don’t have good governance. They can’t classify data. They can’t see a topology of it. They don’t understand how it moves around. They couldn’t really tell you who has access to what.

And in that particular case, then, yeah, those people don’t trust data. Things go really slow. But if you have really good governance, then that is when you know, like, hey, I can see exactly who has access to what data. I can see its topology. I know where it is, how it moves around. And when you have good governance, then you can have a ton of confidence, which is going to give you that trust to make decisions.

MICHAEL KRIGSMAN: Well, who’s responsible for building that kind of program of data governance?

SHAWN BICE: Yeah, well, in larger organizations, oftentimes– I’ve seen governance teams in a smaller organization. It could be in whatever group is covering technology. But it’s almost always a combination of somebody who’s in your core architecture with good oversight from a security team and whatnot because they’re so– think of all the ransomware stuff that is happening now. And boy, oh, boy, it’s maybe more important than ever to be able to secure and protect your data. but most commonly, to answer your question directly, it’s almost always a combination of a core infrastructure team partnered with a security group.

MICHAEL KRIGSMAN: And then what happens in machine learning and AI environments where, if you don’t have a handle on your data and your models, you can find creeping bias and other issues that come in that you may not even be aware of until some problem happens down the road?

SHAWN BICE: Oh, absolutely. Yeah, when you think of machine learning in the context of governance and your whole data environment, think of– well, you could take an easy one, and you could start to think about the quality of your data and using machine learning to just follow patterns for your data.

And then you could understand, like, oh, imagine if a customer had changed– imagine somebody got married, their last name changed, and then is that two different customer IDs, or is it one? And how would you track that. And when you can use machine learning, machine learning will oftentimes be able to follow patterns like that for you and identify it and point it out to you. It’s like finding that needle in the haystack.

Or take for example anomaly detection. Imagine where in your environment all of a sudden a particular set of data is now being accessed fully in a way that it’s never been accessed before. With something like machine learning, you could on the fly detect an anomaly like that and do something about it.

So we’re in the early stages as an industry of machine learning, but I would tell you, I think this is actually going to be one of the most important things that people really, truly understand and embrace because it’s really going to be that assistant, if you will, that is helping you reason through millions and millions and millions of files of data.

MICHAEL KRIGSMAN: And certainly, when it comes to machine learning, explainability is more and more and more important.

SHAWN BICE: Yeah, I often like to share this thought that– I think I read somewhere where a human can process 60 bits of information a second. And imagine a data center or a cloud environment that has billions of events happening every second. There’s no way any one person could ever process that. So having something working with you to find those anomalies, present it to you, and explain it in a way where you can now find that needle in the haystack and make sense of it, now that becomes more important than ever.

MICHAEL KRIGSMAN: Shawn, with your customers at Splunk, do you make a distinction between investments in data that support innovation versus investments in data that support improved efficiency?

SHAWN BICE: Yeah. We try not to think of data like that, in a sense. We always go to data– data is– there’s two things you can never ever do with data. You can never lose it, and you can never give back the wrong answer. If one of those two things happen, that’s just not good. That is the ultimate trust buster.

So as long as you practice very, very good rigor and diligence around your data– that is a non-negotiable, needs to happen every single day– then you get to think about the innovations you want to do and improvements to the system and so on and so forth.

But at Splunk, when we think about innovations, it really, once again, goes back to our customers, what are you trying to build for them. Sometimes it results in us building new capabilities. Sometimes it’s extending something that was there. And at the same time, we’re always doing maintenance on our code and our data to make sure that we’re hardening those systems each and every day.

MICHAEL KRIGSMAN: Shawn, what advice do you have for using data to drive digital transformation?

SHAWN BICE: Well, the advice that I would share with people, take advantage– take full advantage of the cloud. Take full advantage of fully managed APIs. Don’t settle for the way it was. You don’t really want to do that undifferentiated heavy lifting. Take advantage of that.

And don’t be afraid of exploring difference. Just remember, all of these new purpose-built systems are there for a reason. Really, what a developer will say is, I love this because I don’t have to trade off functionality, performance, and scale. And they’re able to build these applications faster than ever before.

The third thing that I would say is, do not pick technology first and then figure out the use case. Today, I would always start with what is the use case. What are you trying to do? Because if you start with the tech first, whatever the limitations of that tech is could constrain your application ideas, and then you’re kind of– you’re really creating headwind for yourself.

So when you really explore and take full advantage and embrace new and you have these new systems, then the next thing you know, you’re able to build applications that you weren’t able to in the past. The other point I would make is, especially if you’re an enterprise that’s been around for a long time, do not let familiarity become a blind spot that stifles innovation. It’s always going to be there. It’s always going to be in your face.

But the customers and companies that I’ve seen push through it– they’ve modernized their work forces– boy, oh, boy, are they moving fast and doing things that they never thought they could do. So those are some of the points that I would share or advice that I would provide to anybody who’s going into the cloud with data.

MICHAEL KRIGSMAN: Shawn, as we finish up. I have to ask you, where is all this going?

SHAWN BICE: Yeah. I think where all of this is going is a work environment that maybe is moving faster than ever before. Maybe the norm back 10 years ago is things would take, hmm, a year or six months. I think those projects that used to most often take a year– those might take a month in the new world.

I think where it’s going is businesses are going to be able to move faster than they’ve ever been– than they’ve ever moved before, which means they’re going to be able to respond to their customer base and almost real time. I think it also is going to mean customers are going to be able to build new experiences, iterate on those things.

The day of putting something new out and then updating it a year later– I think those days are over. You’re going to put something out, you’re going to learn, you’re going to iterate, you’re going to learn, you’re going to iterate. All those things, I think, will become common practice for all. And if that all truly comes together, then at the end of the day, all of these customers will literally have turned their data into doing.

MICHAEL KRIGSMAN: And thus, we come complete circle with data being the support enabler and driver of digital transformation.

SHAWN BICE: Absolutely. It really, truly is. And then when you have that digital transformation– I see people so excited because it’s always starting with, gosh, we never could do this in the past, and look what we’re doing today. And there’s nothing better than seeing somebody use technology in a way that is having a profound impact on a business. It’s just– it’s a real delight to see.

MICHAEL KRIGSMAN: It sure is. Shawn Bice, President of Products and Technology at Splunk. Thank you so much for taking your time to talk with us today.

SHAWN BICE: Thank you.