Cloud Data Summit Podcast
Aug 29, 2019
Eric Axelrod interviews Rasmus Bang, founder of Bang Insights, about some of the challenges with running Business Intelligence, Analytics, and Data Science workloads in the cloud.
Rasmus is a highly skilled Big Data consultant based in Denmark. He has worked with some of the largest companies in the country like Lego.
Hint: most problems are still organizational
Watch the full video:[WPCR_SHOW POSTID=”9803″ NUM=”5″ PAGINATE=”1″ PERPAGE=”5″ SHOWFORM=”1″ HIDEREVIEWS=”0″ HIDERESPONSE=”0″ SNIPPET=”” MORE=”” HIDECUSTOM=”0″ ]
Eric Axelrod: This is Eric Axelrod with the Cloud Data Summit podcast, and I’m interviewing Rasmus Bang today. And Rasmus, can you tell me a little bit about yourself
Rasmus Bang: Of course. Uh, as you said, my name is Rasmus. I am a, an independent consultants, uh, specializing in, in all things, eh, Bi and big data, Eh, master data. I’ve, um, I’m working mainly as an architect and I’ve worked as a, as an architect for many different projects in many different organizations. I’m based in Denmark, in Europe. So, um, so I’m a, um, I’m very, uh, curious to, uh, to share some, some, uh, experiences from here and see how they resonate with where you’re, uh, where you’re from. Mary.
Eric Axelrod: Excellent. And, uh, and tell me a little bit about more about your company. So you do architectural work, you know, w what is, what’s the most, um, what’s the most kind of common common types of projects that you work on or common types of roles that you end up working with
Rasmus Bang: Well, it typically, it’s, it’s rules where organizations say we need to do something about the way we store data or the way we analyze data. And we’ve been, you know, we’ve been working in the, in the traditional bi paradigm for a long time, but a s something seems to be happening. Something seems to be shifting and we don’t know what to make of it. So typically I go in and I help them try and find out, well, where would you, um, where would it make sense to use a traditional bi, um, platform, a data warehouse Where does it make sense to use a, a big data platform Uh, what technologies are we looking at When should we consider, uh, uh, working in a cloud setting When does it make more sense from a, um, an on premise setting So it’s, it’s very much sort of a drawing the, uh, the initial things, um, eh, for customers and organizations who know that they need to do something, but they don’t know exactly what it is and they don’t know exactly what it means.
Eric Axelrod: Excellent. And I know that’s a very, very, very common problem with all sorts of companies, even though even the very, very large ones have, uh, have very similar sorts of issues that, you know, that, that we run into often. Um, you know, and whenever you’re actually working with your customers, what types of Bi and big data technologies do you usually see them using and what do they have installed in their, in their, in their legacy stack
Rasmus Bang: Well, I think one of the things that, that might be a little bit different, uh, compared to where you’re at is that, uh, uh, I mean, being based in Europe and being based in a small country, just north of Germany, we see an awful lot of SAP. Um, so there is a lot of, um, in, in many Danish, uh, production companies, you see a lot of SAP business warehouse, you see a lot of Hana, you see a lot of business objects and I think probably much more than you do many other areas in the, in the world. I think for instance, something like Teradata is more or less absent from the, uh, the Danish it landscape. Um, so a lot of, uh, a lot of SAP, um, technology basically on the, on the Bi side, um, uh, Microsoft tests as gained much more of a foothold in the last, uh, I’d say in the last, I don’t know, four or five years, uh, than before with, uh, with the whole power bi suite, uh, power bi world. Um, I, I, I think my impression is that it’s gone from being something that, um, that architects life like myself, I have tried to push out, uh, to being something that is, you know, allowed in a lot in the door. Yeah. Because I think for two reasons, I think people have, you know, have realized that there’s really nothing, there’s no way to fight it. And also because some of the maturity issues that have been been there earlier are, you know, it’s, it’s getting, it’s getting more to an enterprise ready state.
Eric Axelrod: Right. So is that something that they, that they’ve been struggling with maybe for the last few years is trying to implement something like power Bi and this figured out that it wasn’t quite ready yet and now they’re a little bit more, uh, they’re a little bit more apt to, to go to go down that road
Rasmus Bang: I think it is, and I think it’s, I think it’s very much to do with the, with the, um, you know, the traditional thoughts of trying to, you know, if you try to implement an enterprise data warehouse, then, um, how do you, you know, how do you deal with people using a, a power bi platform to, well, how do, how do you avoid the situation again that you’ve had for years, if not decades, where people have had their excel solutions and their, you know, power bi and their tableau and their, whatever it might be, and have tried to make these, you know, I don’t know, shadow bi platforms. Uh, and, and that’s where I, I see some of the, you know, a tipping point here is that, that, uh, at the moment it’s actually, it’s, it’s being allowed in the door, so it’s not, it’s no longer something that people try and fight. It’s something where people go, all right, well, you know, if, if, if we have to, then let’s just, yeah.
Eric Axelrod: and that hasn’t been a pretty big change that I’ve seen as well over the last, uh, really over the last five years. I think it’s been really, really, uh, big words, been a pretty much a shift from like the monolithic enterprise bi design where it’s something that is fully controlled and fully governed by the it department to where it’s a much more self service model and there’s a lot more organizations that have been a lot more, um, open to the idea of, of doing that. And of course, I think a big part of it is the realization that, um, that they’re business people use excel anyway. They’ve been using excel for a very, very long time and really since day one. And that none of their old, none of their legacy bi platforms have really ever met the needs of their business teachers. Because if they did, they wouldn’t be resorting to excel, which is the way that it’s always been.
Eric Axelrod: Right. And so I think that’s been a big realization, uh, for a lot of companies that said, wow, you know, we have this big thing, you know, and everybody around here calls them spread marks where we have, you know, our organization has hundreds of thousands of spreadsheets that we use on a monthly basis and we want to, we want to get out of that negative, probably never going to get out of it. Right. But, um, but we want to decrease that number. We want to get that to a more governed platform than where it is that enables the business users. Right. Uh, versus, um, versus what they’re doing now is we’re just going to go into business objects and you’re going to download a bunch of rows of data and put them into excel and then do whatever with the right. Exactly. Yeah. So I think it’s a, it’s a very, very interesting, um, very interesting paradigm over the last few years with that. You know, it’s, um, I, I, I find it interesting that you’re seeing the same thing on that side of the pond as well, especially with a heavy, had very heavy SAP usage.
Rasmus Bang: Yes, definitely. Yeah, and I think on the, um, on the, on the, on the big data side of things, uh, I think what, what I typically experience are different flavors of Hadoop. Um, I know it’s a, it’s something that’s shifting a lot these days. Um, uh, but, uh, typically, uh, it’s, you know, it’s Cloudera, it’s Hortonworks, I guess it’s all Cloudera right now. Uh, but, uh, but, but those are the two, you know, the two big things. Um, I’ve seen a little bit of um, but, uh, but I think, I mean the big, the big three or the big two or however many there are right now is, is typically what, um, what people are, are using from a, you know, more of a big data, uh, point of view.
Eric Axelrod: And I’m curious just because of kind of where you are geographically. Do you see XSL as well
Rasmus Bang: Sorry,
Eric Axelrod: you tried to see a fair amount of XSL as well in your part of the world
Rasmus Bang: Ah, it’s not, it’s not something I’ve, uh, I’ve run into, uh, uh, apart from conferences. Uh, it’s not something I’ve seen seen out in the wild here.
Eric Axelrod: Interesting. I don’t remember exactly. I, they’re based somewhere in Germany. I don’t remember whether where their home base is exactly. But, uh, I figured that they would be pretty similar to SAP and have a pretty good footprint on knowing that part of the world, um, because you know, and that, and that’s something that I’ve seen for a long time and it’s not even just a, it’s not even just a country based. There’s a lot of cities in the u s that have been very, uh, heavy to adopt certain technologies. And so like for, for example, um, I’ve been in the St Louis area for a long time in St Louis, has been very traditionally heavy on the SAP side of ERP and has, has been very, uh, traditionally, uh, it was, uh, we were a big business object city and back in the day and IBM barely had any presence at all with Cognos and in our part of the world and m and M.
Eric Axelrod: Um, yeah. And so th that’s just kind of something, you know, that’s kind of the way there’s always been. And then if you go to the coast as well, if you look at new, you know, New York or San Francisco or one of those, you’re going to see a completely different type of technology landscape that companies are 10, 10 to use. And it’s not even just startups, uh, you know, it’s a lot more of the enterprise tech companies as well. They’re going to be a lot more apt to do custom code. They’re going to be a lot more apps to use to do technologies versus what we’re going to see around here, where people are on SAP, they’re on Oracle, they’re on Microsoft, you know, and that’s just, it’s a very kind of interesting geographic paradigm based upon, I think it’s really, it’s really kind of based upon where the big models I’ve been focusing their sales efforts. Right. And where they get traction. Yeah. So it’s a, it’s a very interesting, it’s a very interesting paradigm.
Rasmus Bang: Yeah.
Eric Axelrod: How do you typically see, um, collaboration at your customers Uh, whenever, you know, whenever you have different, different departments that are trying to get things done, you have, uh, you know, different roles or trying to work with each other. How do you see that playing out Whatever you are on the ground working with customers
Rasmus Bang: Yeah. Yeah, that’s actually a very good question. Uh, I think it’s, um, it, it varies, I think from you. I mean if we mainly look at it from the, maybe from the Bi side of things, then you would typically, what I see is some kind of BICC ish model where you have your, um, in your back end people, your front end people of from on the Bi side. And then you have some kind of BICC or center of excellence somewhere in the middle, um, between in the interface between it and the sort of the quote unquote business. Um, and that’s where, uh, you know, that’s, I guess that’s sort of the, the clean model. But then the interesting thing is, well, well then what do you have on the business side of things because, um, uh, one of the things I did before I, um, uh, before I started out on my own was I was an enterprise architect for, uh, for bi and analytics at Lego, the toy company.
Rasmus Bang: And, um, and they had at that point, a very, I almost want to say sprawling model for business collaboration because apart from the BICC and it than they had maybe four or five different centers of excellence out in the different business areas. So in finance and HR and supply chain, all these different areas. Um, so, so there it’s, it, it tends to be quite long chain of, uh, of people needing to collaborate and all to get this and this done. Whereas if you look at it from the, uh, sort of the more big data, data sciency, Eh, set of things, this is one of the areas where I, where I see may organizations struggling. Um, and I think it’s mainly that to do with, uh, not technology more, you know, organization in politics because you’re in a situation where in, in for many years you’ve had it and you’ve had a business area with a need.
Rasmus Bang: Whereas here you, um, do both need, you know, the, the technical side of things, the it side of things you need to, people with business understanding in the business process problems. And then also you need your, um, statisticians or data scientists or machine learning people or whatever. We’ve, you know, whatever it is, uh, who, um, tend to be at least in some cases, in a completely different area. So it’s a, it’s one of those things where you can’t have just two of the three because you don’t get anywhere. Uh, and, and in, in sort of larger organizations where you have a, have a very, uh, robust, maybe a little bit, um, uh, dusty way of, um,
Eric Axelrod: right aligning and allocating people, they go, all right,
Rasmus Bang: well you can have our, you know, our big data team, three weeks starting Monday in the new yeah. But you know, the,
Eric Axelrod: they are designed to start ready by then. So, so I think that’s a, I think that’s one of the things that is, that just takes a, it takes a long time for, for patients to get used to. So, so, so really what you’re seeing is that they, they’ve done a, they’ve done a decent job with traditional bi kind of establishing those cross functional roles where they do have to be a center of excellence. And they do have, they do have stakeholders that are crossing over between bi and the business. But then when you get to machine learning and, and, uh, machine learning and AI and data science, they’re not really doing that, right. They’re kind of working in a silo and they’re really kind of struggling to be able to deliver a business value, right Cause that’s really what it’s about. And then lifting the exact same thing as well to, there’s so many organizations that go out there and they think that hiring a data science team is going to be a panacea to whatever they do.
Eric Axelrod: And then they give them, they get a bunch of data, they give it to the data side and then, you know, and then the, then the, the, um, and then the data science team, uh, they don’t really necessarily know what to work on. They don’t know where the business either, where the business pain is. They don’t really know anything about the business process. They don’t have any collaboration. Right. And then, so they end up, you know, really in a lot of cases just kinda spin spinning their wheels, not really knowing what they’re after. And, um, and I think that’s a, that’s kind of been a big, a big flaw for how a lot of organizations have really constructed their data science and machine learning departments. Because if you don’t start with that business problem, then you’re not going to let, you know, you’re probably not going to be successful. You’re probably not going to figure out what that business problem is just by looking at the data and you know, and if you don’t know what the business problem is, then you’re not going to be able to solve it. Right,
Rasmus Bang: exactly. Yeah. I had a, um, I worked with a, with a very, very bright guy, um, within a data science at one point. And, and he, he always said he really liked the, like the phrase data science, uh, and, uh, simply because it implied that you were working with data in a scientific manner. So you created your hypotheses and you gathered data and you tested it and you refine it and you accepted it, or you discarded it. And, uh, and I think it’s, you know, the way he said it, it was like, oh yeah, well, of course that’s the way it is. But if you compare that to the, um, compare that to the, uh, maybe the, uh, the business case process and the kind of mental process is many organizations then, then many, um, you know, top level decision makers aren’t going to be satisfied with an answer of, you know, maybe this is going to give us some huge insights. Maybe it’s not, who knows So that’s a, I think there’s a, um, I, I think there’s some room for improvement in the, uh, maybe appetite for uncertainty, uh, from any of these things.
Eric Axelrod: Absolutely. But I, you know, I, I do think that that has, that has a really big part of, of, um, knowing why you’re starting out a data data science initiative, right It’s not that we’re going to hire a data science team and then we’re going to get, there’s going to be this, you know, there’s pot of gold at the end of the rainbow that we’re going to find. Eh, you know, it’s that we know, you know, you, you, you need to know where your problems lie, you know, and our problems are that our manufacturing plants are backed up. We can’t get orders out fast enough. We’re always short on raw materials where whatever the problem is, right. It’s like we need to focus on, we need to focus on things that are, um, the things that are contributing to that problem, and then try to try to reverse engineer that rather than just, you know, nebulously um, going after, you know, just, you know, mining data, which I guess is, you know, um, uh, you know, very much the idea, you know, very much kind of a, a, a 1980s, 1990s idea where you can just kind of give a bunch of data miners some data and they’re going to give you a bunch of insights and it didn’t work back then and it
Rasmus Bang: doesn’t work now either. Ready Exactly. Yeah, yeah, yeah. Yeah.
Eric Axelrod: So that’s very interesting. And then, so on the technology side, um, how have you experienced the shift going from a traditional bi to big data over the last, uh, five, five, 10 years
Rasmus Bang: Well, it’s, um, uh, the, the areas where I’ve seen it, it’s actually been, uh, actually coming back to your, uh, to your point, it’s typically been, um, been been driven by a, I think two different forces. It’s been driven by a, I, I’d say a curiosity or a readiness in, in the it areas where they’ve said, well, we can see something’s happening out there. We can see theirs, we can see, I mean, we reading the same inflight magazine as everyone else and, uh, there seems to be something that’s just different. Uh, but then, uh, that has, you know, typically only really gained traction when it’s also been paired with a very, um, with a very sharp, uh, you know, business challenge or business need of some kind or if it be it, uh, finding out the, uh, the challenges in your sales situation or, um, I think I’ve seen it from a supply chain point of view at one point as well where, uh, where they’ve, where the someone from the business as said, all right, well we, we really, really need to get this thing done.
Rasmus Bang: And if you’re saying that this, you know, big data thing is the, is what’s going to get it done for us, then you know, let’s do that. But then I’ll also say that, um, but I’ve really, um, I’ve experienced areas where, um, where once you have to have these things, once you have the, um, um, the experts with both the, you know, have the platform set up and you have your data scientists and, uh, it, it has really been able to show some results that are, that really tend to blow these a decision maker. I have at one point a, um, um, I had at one point an example of a um, move sitting with uh, some customers and looking at their, um, ah, it was something with fraud detection on a web shop. Uh, and it was something to avoid a certain type of, uh, a certain type of, of purchasing.
Rasmus Bang: And, and this organization actually had a, um, it had a mechanism in place to, you know, prevent this purchasing behavior. And then we had the data scientist pro show the finding and they said, all right, and then we had this guy and he’s done this, you know, from the same IP address 76 times in a row in the, you know, the people that are like, what Because you know, they, they didn’t, they didn’t think they had a challenge anymore. They thought they’d fix it. But when you’re able to really go in and say, you know, there, there’s something here you need to do something about. So, um, so, so that’s just to say that that has really tended to, um, when you have these moments to really accelerated, uh, whereas when you don’t have the business side of it, then it just, you know, it tends to, to just be like an, like an it pet project.
Eric Axelrod: Yeah, absolutely. But then like if you, if you kind of kind of, um, um, you know, wrap that around, you know, w w the, the idea is around, you know, customer is running traditionally, they’re running SAP, they’re running oracle, the running, you know, all of our, all of our legacy technologies and then they’ve been over the last decade going to something else. Whatever that is, whether it’s music or whether that’s cloud or whether that is, you know, whether you’re using spark or something like that. Um, have you seen a lot of momentum on that Uh, as far as, you know, people that are actually taking those initiatives and saying, okay, we’re going to go rebuild our core platform on some other, uh, you know, and some other, uh, some other plot or,
Rasmus Bang: yeah, I th I’ve seen it, uh, maybe not the, I think people are still hesitating at least of the people I’ve seen on, on the call platforms, but I definitely see it a lot on the, um, you know, on the sort of the edge platforms. So you have things like, um, I mean obviously salesforce, uh, you have things like, um, uh, cloud solutions for, um, for HR, uh, success factors and workday for, uh, what’s it called, the, um, uh, the buyer portals. So, so most places I’ve been people are, they’re, they, they’re getting an appetite to sort of, uh, try out many of the new or, um, uh, I guess software service solutions for, uh, for non core areas. But for the sort of the key ERP, I don’t think I’ve seen a lot of, lot of organizations who’ve really, really been, uh, you know, gutsy enough to say, all right, we’re going to do it.
Eric Axelrod: Yeah. And I think we’ve, we’ve seen something very similar, uh, in, you know, in the states as well, where, um, a lot of what a lot of companies are doing specifically on the Bi side, is they’re implementing what a number of organizations have termed a tier two data warehouse where they’re not going to retire their legacy. Let’s see. One, if they’re, if they’re running an oracle data, where else since the 1980s, they’re going to keep that on and they’re probably not going to change anything about it. And then what they’re going to do is they’re going to take another new technology and laid on top of the old one. So they’re going to, you know, they’re, they’re, they’re gonna buy, um, you know, they’re going to buy one of the newer technologies or they’re gonna buy a cloud platform or something like that. And then they’re just going to kind of migrate, uh, uh, a subset of the data up to that new platform.
Eric Axelrod: And then they’re going to buy another set of tools because they don’t want to impact the, the legacy platform. They’ll buy a Tablo, they’ll buy a looker or they’ll buy a click, they’ll do something like that and lay it on top of their new data or else that’ll be their new platform. But the old one is still around, you know, and so, um, I, and I, I’m sure just because of the time and expense to be able to, uh, migrate off of that, that that’s what organizations are really doing all over the world. Because, you know, I, I haven’t, I had been at a number of them where, uh, you know, number 14 of a hundred sites orgs where we’re in there kind of looking at their infrastructure and they’re like, you know, we, we want to get off of on legacy platforms entirely, no budget.
Eric Axelrod: There’ll be like, wow, this is a a hundred million dollar project. And they’ll figure that out. And like, okay, it’s going to be four years just to rewrite our ETL jobs in another tool or something like that. And then they’re like, okay, well we’re not, we’re not gonna do that. And so instead they’re just going to say, okay, let’s go rewrite some parallel things and load them, et Cetera, and then we’re going to keep this, you know, legacy system kind of on life support indefinitely because that’s going to be so expensive to move off of it. And it’s just not, it’s probably not practical to do. I mean, even if they’re paying millions of a year in license fees and infrastructure fees, it’s going to be a really long ROI to be able to turn those legacy platforms off if they really have to spend double digit millions of dollars on project work to rewrite everything. Right. Yeah. So that’s definitely been a big, definitely been a big thing that we’ve seen as kind of been, I’m not going to say that’s been a barrier, but you know, it’s, it’s a very interesting paradigm where a still of other problems whenever you’re running parallel systems, um, there’s not only technical issues, but there’s a lot of business process issues that you run into. So it’s pretty interesting thing.
Eric Axelrod: You’ve Kinda kind of touched on his knees a little bit. Do you see a lot of Duke ecosystem technologies that are actually in use And I, and I’m going to use that with a qualifier because what one of the things that we see here is that there’s a lot of companies that talk about it, um, that say, hey, we’ve, you know, they’ll go to conferences and I’ll talk about how we’ve implemented x, Y or z thing. And you’ll find out once you get into that, don’t actually have any of that. It’s all, it’s all vaporware and there. And so, uh, there is certainly a lot of, um, a lot of Duke, uh, technology that’s in production, but it’s a lot less than what you would think right from the wild. Curious if you kind of see the same thing, um, over or whether there’s just a lot of organizations that are still, you know, just mostly running on legacy systems and then they’re kind of claiming to run on, on, um, a big data platform. No, I think some of the things I’ve seen, there are places where, where, where you really get the impression that they are, that, that they are quite far. Um, and where I think for me, the, the key question is how many sort of more or less business critical things are running off the, uh,
Rasmus Bang: off the platform. But I’ve also seen areas where I’ve seen organizations where, um, where they have implemented a, a platform, eh, but they are still, you know, they’re really struggling with, uh, for instance, um, they’re struggling with, um, the data loads basically. So, you know, we have to, we use, uh, you know, do we use scoop, do we use knife id, we use whatever. And when you say, well, maybe we should, you know, maybe we should look into this technology, they go, Ooh, that’s probably,
Eric Axelrod: well we want to be doing right now.
Rasmus Bang: Oh, well, you know, there’s this use case for, um, for real time. Um, uh, you know, real time feedback and all that. And they go, yeah, but we can do it with a, so it’s, so, it’s not real time, but it’s within 15 minutes and we can do that on our old platforms. So let’s do again. So I th I think that comes, it comes back to the um, uh, what’s it called, the maintainability of it. Um, that I definitely see, I see organizations that have said, all right, let’s, let’s do this, let’s implement this. And then they find out that the, I, it’s probably not fair to say maturity, but definitely the, um, the, the way of, um, the steps needed to keep this running and keep it watched and scripting and all that. Right, exactly. Yeah. Is much more, it’s much more challenging than I think they initially thought. Um, so, so they’re, I mean that’s examples of organizations where, I mean there is, there is a platform and there are things running, but I think you could probably, you could probably discuss the, uh, sort of at least at least the short term business benefits that come.
Eric Axelrod: No, I, I can, I can definitely see that. I mean, that’s something that we’ve seen a lot of organizations struggle with as well. Even the ones that I actually do get something up and running. And I think that’s one of the factors that really inhibits the adoption of the new platform as they try to pile more work onto it and they find out that it doesn’t perform as well as they thought it would or doesn’t deliver all of the, you know, kind of the magical value that, that it was promised. Right. And then they start to pull back a little bit and they get a new workload and they’re like, well, do we want to put this on a duper, do we want to put this on a legacy platform And a lot of times on the legacy platform, right Yeah.
Rasmus Bang: Yeah. I think it’s, I think it’s also a, um, at least what I’ve also seen in, in, in, uh, in those types of, uh, those types of situations is the discussion of saying, you know, if we have the situation where we, we have a, we’re implementing a solution here, we need to store the data. We need to, you know, analyze, react on the data. It’s relatively, it’s relatively small, um, uh, data sets. It’s not very complex. Uh, so we know we can, you know, we can solve the use case using our, our traditional bi world. So we’re gonna do that. And then the question becomes, will do we also store the data in our data lake, a big data platform because we know that we have a, we have a strategy of saying we want to store all the data there because we want it to be able, uh, we want it to be available for, for data science, for machine learning, all these things. And there, I think it’s always interesting to see the, um, you know, the, the discussions of, well, how much does it cost What’s the politics I mean, how strongly can we really enforce this principle or we want to put it in and, and, and who’s going to pay for it
Eric Axelrod: It’s always about politics. Right Exactly. Yeah. We talked a little bit about, about Hadoop, but if we, you know, kind of pivot the other direction, what are, what are some of the things that are leading co companies down the cloud path And can you really kind of explain what the shift is from on premise or private cloud systems down to public clouds when we’re talking about AWS as your Google cloud, maybe Oracle Cloud, IBM cloud, some of these things where the architecture is, is kind of different. Like what have you seen over the last five to 10 years on, on that front
Rasmus Bang: Well, um, well the first time I’ve was, uh, was part of implementing a Hadoop platform, uh, we were quite, um, we were quite clear that we wanted to go the, to go the cloud direction, um, for different reasons. But I think, uh, the main one was the scalability and the flexibility, um, because, uh, what we had heard and seen from others, uh, on the, you know, the capabilities and the, and the possibilities. Eh, this just seemed to be the, uh, it seemed to be the right way to go basically. And, and there, I, um, I remember that was a lot of, uh, it was a lot of discussion, uh, mainly from the sort of the infrastructure a world where people were, I think it’s probably been the same in more or less every single company, but where, where the first impression is, well, why would you want to do that
Rasmus Bang: We have a, have a perfectly good terrace on the right here. Um, so, so I think there was the, uh, the um, um, what’s it called The, the change management part of it. Then, uh, another thing that was a big inhibitor to begin with was, um, was finding out how do the, you know, contract models work and work here. I remember, um, uh, without, without naming any names, I remember that one of the big cloud providers, uh, had a, there was something around privacy of data where, um, at least the, the in house lawyers and, uh, the vendor simply couldn’t agree. Uh, and where we ended up saying, alright, it’s been good talking to you. We’re going to go w w we’re going to go with someone else. And so, so I think that there has been a, um, I think that’s definitely been one of the, um, one of the shifts there.
Rasmus Bang: And, and, and from what I’m seeing now is that that’s changed, I mean tremendously. Um, over these, over these years, people aren’t afraid of cloud anymore in the way they were. And I think, uh, I think for me the big, the big shift here is that, uh, infrastructure organizations aren’t afraid, afraid of cloud anymore and, and, and security organizations aren’t afraid of cloud anymore. I think they were, at least in my experience, the ones that tended to, um, to, uh, to hold, to hold these initiatives back, uh, at least, at least from, from what I’ve seen.
Eric Axelrod: Exactly. And that’s one of the things that, uh, that I’ve seen as well with S. Um, and one of the things that’s kind of surprising me almost how quick it happened is health care organizations, particularly hospitals, health insurance companies and things like that. You know, because in the U s we have HIPAA, which, you know, the, the, the federal government mandates a lot of things about data storage and privacy. And data destruction, the word, and we’re talking about physical like, you know, uh, data center level things like access to hard drives and things like that. That’s all regulated by the federal government. Right. And, and then as soon as, uh, um, as soon as AWS and Microsoft as your got certified for HIPAA, then the game changed. You know, because now they can come down to these organizations and say, all of your, all of your, um, all of your, your withholdings that you had about going into the cloud, the government says they’re okay.
Eric Axelrod: They’ve certified us, it’s fine now. And then that was a huge, you know, a huge mindset shift I think for a lot of companies to say, wow, they, they say it’s okay now, so maybe we can do this even though we’ve been resisting this exact thing for the last 10 years. Yeah. And so it’s been, you know, it’s been a very, very, and we’ve seen the same thing. Uh, I mean obviously like, um, commercial banks and things like that don’t have a HIPAA regulations to worry about traditionally. Uh, but they’ve kind of been in the very same boat. Where are they for the most part, been very, you know, very on premise data center and there have been some pretty substantial moves towards the cult club. Like I think it was a, what was it in 2016 I think that, uh, I believe it was, uh, JP Morgan Chase invested just a massive, a massive amount of money and snowflake, you know, with the intention of moving a huge part of their data warehouse, you know, to AWS on snowflake, which was a, and that was a huge, huge move because I don’t believe that anybody had done that before.
Eric Axelrod: Uh, and you know, especially, you know, in the banking sector and the financial services sector, it’s all very much, uh, you know, uh, historically very much a, uh, an on premise, uh, fully managed data center type operations that they’ve been running. And just in the last, like in the last three years, it’s been, it’s a massive, massive mindset shift for people are really now start to explore these technologies and say, wow, these really are safe. All of the, you know, all of the, um, uh, all the things that are, that we were worried about in our legal department or worried about, these are not a concern, you know And so, um, it, I think that that’s, that’s made a big, uh, this made a really big impact on the adoption from what I’ve seen in the last few years.
Rasmus Bang: Yeah, yeah. No, I completely agree. It’s, there’s been a, to call it the sea change, uh, sort of in, in that, uh, in, in that area. I think also one of the, a sort of specifically from the, um, uh, um, from, from the, I guess the, the big data world. I keep on trying to find a better word than big data because it’s very 2016, but, uh, I just, you know, it’s not data science. It’s not machine learning. It’s not the warehousing. So, yeah. Anyway, um, so from where I was like, oh, yeah. Um, so when we, when we first did this, uh, uh, implementation of Hadoop, uh, uh, it was just, um, you know, it was, it was complete standard. It was, uh, it was a Hadoop installation, a on the, uh, you know, the cloud vendors, um, infrastructure basically. Whereas, uh, I think the, the biggest, uh, the big changes, uh, coming recently here is the, um, you know, the maturity, the, in a sense, massive maturity that have come from the, you know, the, the offerings from Amazon, from, from Microsoft, from Google, where, where you don’t know all the things that we had to, uh, you know, mess around with to get this working is really, I don’t want to go as far as calling itself as a service right now, but it’s, it’s definitely much, much closer and much, much mature, much, much more mature than, than it was.
Rasmus Bang: Uh, and it was at the time. And I think the, you know, the scale of scalability and the flexibility that we were looking at at the time of which was something that needed to be done with, uh, scripting, setting up clusters, showing them down, or is, is, is today more or less even, you know, built into the whole, um, the whole world So I’ve, I’ve, I’ve seen, uh, I’ve seen examples of, um, I mean that’s more or less the, the standard, the standard now that, you know, the fact that you can, the fact that you now just can, I’m, what’s it called, spit up the, the storage and the compute parts of it is just a, it’s such a, it’s, it sounds like a small thing and I guess know conceptually it’s just a, it’s a very big thing and it’s something that, eh, I don’t see happening in an, in any kind of on premise world without much more, um, much more hassle.
Eric Axelrod: Yeah, absolutely. It does require a huge investment. No, I think you’re, you’re absolutely right. That’s one of the things that it does. It sounds, it sounds simple, but it’s such, it’s actually a game changer for what your ability to do because, uh, you know, and I’ve seen a lot of, you know, even even before a lot of these, uh, platforms were service, I guess I should say, um, uh, you know, software as a service ready, uh, for some of these tasks. You know, I’ve seen some people in the earlier days of AWS that implemented, uh, dynamically scaling ETL clusters, which was really, really interesting because, you know, we’re talking about some very heavy ETL tasks that would traditionally take, you know, many, many, many hours on, on premise, you know, system where you’re constrained by a certain number of resources and then we can, we can put it on to elastic elastic compute environment.
Eric Axelrod: And the speed in which we can get it done is only limited to how much we’re willing to pay concurrently for that. And, you know, and then we don’t have to keep those servers on when we’re done with them. And so, you know, um, you know, and that’s one of the great things about the, uh, the idea of elastic compute is that, you know, for the next 30 minutes, I need a hundred servers. While you can get a hundred servers for 10 minutes and then you only paying for that 10 minutes of use, then they go away. Right And so, um, you know, that’s something that you can’t do with on premise because you have to have those hundred servers, you have to have them rack running and ready to go, which means that, why wouldn’t you just, you know, why wouldn’t you just be running your applications on those all the time.
Eric Axelrod: And so, uh, and I, I have seen some of the, some of the very large organizations that do have private cloud and still have some on-premise architecture, they are moving to Kubernetes and docker swarm for their internal applications. So they can do some of that with their, with their bigger operations, but they’re still nowhere near the scale, uh, that you can obtain through, you know, through the public cloud providers. Because, you know, with pretty much any provider there is really practically no limit. I mean, any, any workload that you want to throw at it, it’ll be able to, it will be able to gobble it up, you know And so I think that’s the really interesting part versus the old, the old world would say, wow, we’ve got, you know, we have 24 nodes in our data warehouse cluster and, you know, once they’re matched or maxed and you know, we can try to we can try to tune our indexes, we can try to optimize our partitions.
Eric Axelrod: You know, we can try to go through our explained plans and make these things as efficient as possible, but you know, at some point you’re gonna reach that point of diminishing returns, right Where you just can’t really make it any faster or it’s not going to, you know, you, you can’t get the games that you need to get to be able to take a workload that’s going to take, you know, 24 hours and get it down to 10 minutes. It’s just not gonna happen. Right. Uh, but if we go to elastic cloud, we can do that. Uh, and it’s very, it’s very, um, it’s actually very brief, feasible to do it. In a lot of cases, it’s actually substantially less expensive to do that in a cloud and throw a hundred nodes at it in parallel rather than running our, you know, our small cluster that we have on premise or something like that.
Eric Axelrod: So, um, it’s, it’s been a very, very interesting model that, uh, that I’ve seen on the, on the, on the, uh, on the, on the cloud side, you know, and of course that’s, that’s one of those big reasons, kind of what leads them. They’re writers with them inspecting their, their workloads and saying, well, we really need to speed up our delivery. We really need to speed up our SLA, our business users or our, you know, our business users need to have this when they get in at six o’clock in the morning and, you know, we can’t have it to them until noon. We’ve got to cut six hours out of our process. You know, how, how are you going to cut six hours out of your process Because that might be reducing it by 75%. Right. And so, um, you know, and so I think that that’s good. That’s one of the big things where you, that you know, you can actually accomplish that with elastic compute, um, in the public cloud. Whereas you, you’re, you’re, you’re probably not gonna say that you can’t do that with on premise, but it’s not, it’s not going to be, it’s not going to be practical to do it with on premise because you have to own
Rasmus Bang: and maintain those servers. Right, exactly. Yeah. Yeah. One thing I would say though on the other thing is I heard from other people, for many people that, that oftentimes you, it would be nice to have some kind of training wheels thing in the public cloud because you know, when you put in your credit card, if you don’t know what you’re doing, then you very strong risk that you’re going to need to run to your boss and say, oh, by the way, I tried this thing out. Um, you’re going to get an invoice of $10,000.
Eric Axelrod: And I, and I’ve seen ’em definitely seen the same. And it’s totally, I’m totally off topic from what we were talking about, but I’ve seen some organizations that did lift and shift projects where they just said, you know, I’m going to take my existing Oracle server and we’re just gonna put it in AWS. And whenever they do something like that without doing any rearchitecture or without putting it onto the lastic platform, their cost will three, four, five x. Because, you know, they go the old way. Cause I mean, they’re, they’re not eliminating the need for a DBA. They’re not able to necessarily, um, you know, reduce any staff or anything like that. So it’s still the, you know, the same staffing level. And instead of taking a server that we physically owned and we had to pay, uh, you know, we had to pay, you know, cooling and power for, and we had to pay for parts.
Eric Axelrod: Um, you know, instead of doing that where there’s, you know, they’re very little, very little recurring costs to run that. All of the sudden now we’re putting into this environment where it costs, you know, thousands of dollars a to run and you know, and so a lot of the, and then I’ll, I’ll, I’ve seen some companies that have done that at scale, whether they said, let’s just take all of our private cloud infrastructure or all of our pumps, infrastructure, let’s put it up in the cloud. And the bill increases substantially and they don’t get any benefits from it because they didn’t, they didn’t fix any of the problems, you know, they, they didn’t, uh, give me like, and what they really need to be doing for those, it’s kind of looking at those opportunities for, uh, to, you know, to be able to use Alasta compute. And if you’re just moving a VM, then that’s not going to get you there.
Rasmus Bang: Right. Yeah, yeah. Yeah. So that’s very interesting.
Eric Axelrod: What are some of the challenges that companies face when moving to the cloud other than unexpected costs Kind of like we touched on, or maybe the, you didn’t get the performance benefits that you thought you would because you did a lift and shift. What are some of the other things that you kind of see, um, that are, that are, that are challenges that
Rasmus Bang: companies are facing Well, that’s a good question. I, it’s actually, it’s, um, typically for me at least, the main things I see are not so many challenges in moving to the cloud, but more the concerns, uh, you know, before, before moving to the cloud. Uh, I think there’s, um, there’s sometimes, I mean there’s, there’s the latency issues, but, um, but, but again, I have to say and that if, if I look at the amount of people concerned about latency before a cloud migration and compare that to the amount of people who actually experienced challenges with latency after a cloud migration, it’s, it’s maybe a 10th. Um, absolutely. So it’s a, I think, I think the biggest, um, the biggest, uh, challenges are mainly from a, uh, what’s it called A luck in point of view of saying, you know, if we’re, if we’re going to go to this, uh, this solution, what are the, the options that we have for, you know, for going somewhere else down the line. But at the same time, I’d say, well that’s, you know, that’s the same. It’s not really, it’s not a cloud specific thing. It’s the same if you, if you, if you, if you for instance, decide to implement SAP, then it’s in for a penny in for a pound.
Eric Axelrod: Exactly. And then on that note to those, um, those, um, um, I forgot, I was going to say there for a second. Those, those same organizations, um, uh, oh, I’m sorry. W uh, what we can do now is with, is with, um, a mole cloud. You know, there’s a lot of architectures that support spanning dynamically across, um, you know, across infrastructure. So even if, you know, even if we’re running multi-region in AWS, for example, we can still set up, um, either fail over or dynamic scaling over Google cloud. Right. And then, so, you know, even if AWS were to go down or an entire region where to disappear or something like that, yeah. You know, we can very, very easily shift if you build your applications and the right way to be able to kind of dynamically just redirect all of your traffic over and then scale up on the other side.
Eric Axelrod: Yes. Um, that’s a very, they know that that’s a thing that’s a relatively new, uh, concept that, uh, you know, really in the last couple of years has really started to get a lot of traction and it’s got, you know, it’s received a lot of vendor support as well with the, you know, vendors that really realize how, how powerful this is because it completely eliminates the vendor lock in argument. Um, and it completely eliminates the idea that, you know, what happens if there’s an AWS outage Well, we can take care because we’re going to fail over to a completely different provider. Right. So, um, so kind of, it really kind of neutralizes a lot of those, uh, a lot of those concerns around, around vendor lock in or anything like that. As has more data solutions are moving to the cloud. You see any changes to, uh, associated disciplines like data quality, uh, metadata management, data governance, all of the things that, you know, impact companies whenever they are, whenever they’re private cloud or whatever, whatever their on premise, does any of that change whenever they go to the cloud
Rasmus Bang: Hmm. Well, I mean we, we talked a little bit about, about Microsoft and Power Bi and, uh, and the, uh, what’d you call it The spread marks, uh, earlier. And I think that’s, um, that’s probably, that’s probably one of the things that are, that, that I see the most is, um, um, seeing as how it is now much, much easier for a business user in an organization to say, I need this solution. I have a mastercard, I am going to go purchase this thing, Eh, where you don’t really need the, uh, you don’t need the, uh, the explicit consent from it. Or maybe you do, but you’ve had an opportunity to bypass it. I think that’s where the, uh, I think that’s where the biggest, um, uh, maybe strain on, um, on these, uh, on these disciplines come because, um, because the, the bigger the, um, the bigger the possibility is for you to actually, you know, go out and, and, and make these solutions on your own.
Rasmus Bang: Um, bigger the risks of, uh, of unclear definitions and, uh, uh, faulty metadata and, uh, missing governs all this is, it’s going to be, and, and again, it’s not a, it’s not a, it’s not a cloud challenge. I mean it’s, the same thing happened when, you know, when excel came, the same thing happened when, when tableau, but how can I say it Maybe the, the, uh, the, the capacity and the capability that, Eh, you can now go out and, and get, um, as a, um, as a business user, um, is, is much, much higher and much, much stronger than it was maybe in the past. Um, so there, I think there are definitely some, some things where we’re organization should, should I think, I think they should have a very, a clear strategy for saying, you know, what do we have from a data point of view, from an insights point of view, which is controlled, which is clear.
Rasmus Bang: We have the definitions, but we know that when we say financial turnover, we mean this and this and this. But we don’t mean this and we don’t mean this. So that you know that when you have the board meetings and people are sitting with the, with the reports, you don’t have to say, well, mindsets were up five points and the other guys were, well according to mine were down two and then I spent 45 minutes arguing over. Um, so, so I think that’s, um, uh, that’s maybe the, uh, that’s the, uh, for, uh, for the, um, data, data governance area to, to rise to the challenge and be able to say, we know that there are needs for, um, we know that there are needs for, uh, for people to, to, um, to analyze things and to do stuff on their own. But at the same time, if this is something that many people will, will use to make decisions, then it needs to be, um, we need to be able to trust the definitions. Uh, that’s definitely a, um, an area that hasn’t gotten easier.
Eric Axelrod: So do any of the, uh, any of these associated disciplines like data quality, like metadata management, like data governance, do you see any of these disciplines becoming redundant over time Because either because of the cloud or just because in general, you know, we don’t need these things anymore.
Rasmus Bang: Um, well maybe, you know, maybe in a perfect world, but I’m, I’m, I’m, I’m not sure. I’m not sure if, if it’s something that’s, that’s really gonna happen anytime soon. I saw a, um, I saw linkedin, um, posts here, uh, a few days ago where, um, I can’t remember his name, but it was, um, it was someone, I think, product manager from Informatica maybe who said he just had a, a, an informal Twitter poll asking his, uh, data scientists, uh, friends and contacts, how, something along the lines of how much time did they actually spend doing analysis and making models and how much did they spend gathering data, cleaning data, um, preparing data. And from what I remember, he said that 78% of the time, uh, was spent gathering data and cleaning data and 22% was spent analyzing. And I thought, wow, that is, I mean, that’s almost uncanny when you think about the, uh, how many years we’ve had this thing where we said, well, analysts send to spend 80% of their time gathering data and 20%, uh, analyzing. So that means that we’ve made an improvement of 2%, uh, percentage points
Eric Axelrod: in the 20 years. Um, hopefully. So, so
Rasmus Bang: that I think, uh, you know, when we work with data, when we talk about data, I think we just, we need always to, to, to remember that, um, that we’re up against a, we are working with, you know, people and their people’s nature because it is in people’s nature. If you’re in an organization to say, oh, by the way, it could also be interesting to, you know, to when I create new products or whatever to have a fuel covering this or to have something covering this. And it’s also in our nature to say, I need some information about this. Let me try and, you know, gather some stuff and then I’ll, you know, mock apart and then we’ll, we’ll, uh, see what comes out. And it’s also in people’s nature not to document
Eric Axelrod: anything that’s about right. So I think, I think we’re just a, I, I think, I think we’re struggling against people
Rasmus Bang: nature. I do think that, um, maybe, um, some of the metadata disciplines are going to get easier. I think it might become easier to, to align and harmonize. Um, uh, I think many cloud technologies will definitely be able to, to improve these things. Um, so hopefully it’s going to get, hopefully it’s gonna get easier, uh, in the future, but, uh, yeah, but we’re not there yet. Right. Maybe not quite.
Eric Axelrod: Well, and one of the things that I know is a big impact with this as well is, you know, and this is, this is one of the reasons why the, uh, called the data disciplines are different than a lot of other types of software engineering types of roles. Because, you know, if you’re building a front end customer facing app or you know, even if it’s an internal app that you’re kind of building for use, uh, for your users, then you as the, uh, development team kind of have full control over how you want that thing to behave. You know, and so they have the opportunity to enforce those rules on the front end. But whenever we’re working with data on the backend, we don’t, you know, and we have to deal with, we get and um, then some at some front end applications that are built well from a data governance point of view, uh, some are not.
Eric Axelrod: And uh, you know, one of the ones for example that I’ve seen, um, I’m, I’m certainly not a, not a Navision expert, but I’ve seen a number of Navision implementations. This was back in the day. I don’t know if it’s still like that where, um, in the gooey, in the ERP system, you know, if you have a screen where you can enter customers and you hit a dropdown that um, you know, so let’s say example, for example, let’s say you drop down of countries and you can either pick one that’s in the list or you can just type anything in that field. And if you do, it’ll save whatever you typed. And then, so, you know, everybody says, well, I don’t understand what the problem is until you go back and you look at the data and you profile, okay, well what countries do we sell to
Eric Axelrod: And you have to spend the next three months cleaning your country list to figure out, you know, okay, there’s a bunch of stuff in there that makes no sense at all, but it’s in our data. So what do we do with it Right. I’m like, that’s a problem that you can only fix on the front end, you know, as far as, as far as, um, uh, you know, as far as actually implementing them in governance. But frankly, so many organizations don’t do that. You know, they’re just going to say, just, you know, you’re the data team. You just deal with it and you go, go build the, just fix it. We’ll end up building some, you know, some, um, uh, some mapping rules that, you know, they work. Sometimes they don’t work the other times, but the problem is that we can’t predict, um, you know, we can’t predict what the, you know, um, what the business is gonna do next.
Eric Axelrod: You know, they might end up with some new customer or some new product or something like that. Maybe we just expanded into a new region and all of the rules that we worked, you know, they don’t, they don’t work anymore. All the rules that we built because we have a whole new set of things that we need to account for. And then, you know, the David, he doesn’t find out about it until well after it’s happened, you know, and until, you know, it’s like, oh, we’ve been live in this thing for a month and now we have a month of bad data that we’ve got to go clean up. Right And so those, those types of problems I think are, are always, you know, those are kind of really always going to exist because the people who are building and directing the construct of those front end apps, data governance, there’s really not a concern that I’ve ever seen come from kind of the software development world.
Eric Axelrod: You know, they’re usually a lot more concerned about user interface and, you know, general usability and certain features that data governance is not even on the radar. Right. So I think that’s one of the things that for, for, um, I, I mean I really hope that we’re going to have some crossover from the data world and be able to kind of loop in a lot of those, you know, a lot of the custom developers and, you know, kind of help them understand why enforcing data governance rules and MDM rules are so important and, uh, you know, kind of, you know, cause if we can propagate some of those things to the front end, then it’s gonna help. Tremendously. Problem is, is that we have all of these legacy systems that have been around for decades, right, that are, that are going to continue running. And even ones that were implemented five years ago, they’re probably going to be around for another 20 years then.
Eric Axelrod: So, um, you know, cause I mean, there’s still a shocking number of applications that are still running today that were built in the 1980 or the 1990s. And they’re still an operation, you know, and uh, they’re still being used on a daily basis. Maybe, you know, they’re not maybe using the same capacity, but the point is, is that they’re still there and all of the balls that were rolled out in 1982 whenever they built it are, we’re still dealing with them today. And so, you know, and so I think that’s going to be, you know, I, I’m Kinda hate to say it, but I almost want to, you know, think that I’d be, I’d be really surprised if these problems are solved within our lifetime, just because of the longevity of these systems. Um, and, you know, as they tend to happen, uh, you know, it would be great if we could just, you know, flip a switch and then all of our data quality problems are gone. But as long as, as long as they’re able to, um, as long as they’re able to continue allowing the bad data in, then we’re going to have to deal with it. Right.
Rasmus Bang: Yeah, exactly. Yeah, exactly. Yeah. And I think it’s also, I think it also in a way ties back to the, you know, when we talked about the organization, a part in the beginning where if you’re, if you’re developing a, a front end for something, then there’s, you know, as a customer and then there’s a development team. It’s, it’s very much a one-to-one. Whereas with the in the data world, the insights, well it’s, it’s this, it’s this long chain or a web or whatever we want to call it of different people who are interested in data from different areas in order to gain different insights. And, and that’s just a, um, it’s just a whole nother, um, it’s just a whole nother world.
Eric Axelrod: Yeah, it absolutely is. And then, you know, and then the other side of those, those types of, um,
Rasmus Bang: okay,
Eric Axelrod: well, so that even if there is, you know, even if somebody is on the process of building a new system right now, a new front end system for customers or end users or something to enter data into, then, um, the idea that the data team is going to be able to come in there and influence it whenever they’ve already set a budget, they’ve already set a timeline and then we as the, as the data architects are going to come in and go, okay, time out. You can’t do that. We have to be designed and let, but the budgets, the budget. Right. And, um, it just because the data team doesn’t like, it doesn’t mean that the budget is going to change. And so they’re, you know, so the, the, you know, the team that’s building an app is going to do whatever they’re going to do and then we’re going to, you know, we’re going to deal with it on the back end. Yeah. What were that The, I, I don’t think I’ve ever seen an instance where, um, where, uh, somebody in a data role was actually able to go in and, you know, successfully make that argument and get more money and get more time for a project because of, you know, um, uh, because of
Eric Axelrod: Spec of data quality issues. I should, I guess I should say, right. Uh, after what’s I said, we
Rasmus Bang: do try though. We do try, but I could be wrong that I can’t remember a single instance where anybody ever successfully made that argument and actually got a project delayed or, or, um, or more money allocated or something like that, so they could deal with that. So, um, it’s, um, you know, unfortunately, right. You know, until, you know, and I think this is, especially in larger organizations, this is really kind of an enterprise architecture, um, issue because it’s really kind of something that they need to be, you know, they need to be having ownership of. But I that, you know, even so, a lot of those organizations, the enterprise architects, they’re in the same boat where they don’t find out about it until the product is in flight. You know, the, the budget was written a year ago and they get to deal with whatever they get, you know, to deal with.
Rasmus Bang: And, you know, and this is something that I’ve seen working for a number of very large organizations in the past as a consultant. We’ll, we’ll come in and we’ll build something and then we’ll try to get it into production. And then they’ll say, okay, stop. You’ve got to go clear this with the enterprise architects and then you’ll go bring it to them. And they don’t even know what your project is. They’ve never seen this before. Right Yeah. And they never signed off on it. They never got clearance on it. And here you are, you ready to put the one in production and you know, and then you have all of these things that they want you to go do, but guess what, you know, the company didn’t budget for that. So, um, you know, so you know, you, you, you, you kind of have to deal with the most critical issues and then get those over the, you know, cause, you know, get those over the goal line and, you know, but realistically, a lot of those, you know, kind of thrown in the side type issues, like their quality are usually not addressed. Or the other thing is documentation because nobody ever budgeted for documentation. Right. Um, that’s not a thing that ever happens. It’s like, okay, you’re done building the app project over. Uh, even though, you know, you probably need another 20% of the budget to actually document how the thing works.
Rasmus Bang: Uh, but, uh, you know, that’s one of the things I’ve, I’ve never seen a product actually had documentation written into the budget where, you know, and, and actually had a timeline that says, okay, here’s the time we’re going to do this. You know, and even if you end up running a little bit of buffer into it, uh, you know, as far as the, as far as the estimate goes, that usually ends up getting swallowed up by something else. That’s more important than documentation and, uh, you know, usually it’s bug fixing more or something along those lines. And, um, you know, you know, if you’re lucky you can get a little bit of documentation done. Right. So it’s, uh, it’s, uh, a very interesting organizational problem that, uh, unfortunately I don’t see going away anytime soon. Yeah, no, I think you’re right.
Eric Axelrod: Oh, how, um, do you see the maturity of cloud based solutions It’s some of the companies that, that you’ve been working with
Rasmus Bang: Well, I think it’s, um, I think we took, touched a little bit on this earlier. I think it’s, it’s definitely something that’s increasing quite dramatically. Uh, I mean the, um, there’s the, uh, there’s the maturity and then there’s this, the, uh, you know, the, uh, daring to do it. And, uh, and what I see is, um, is also this where you people start relatively small. They say, well, we have a, um, what’s it called A, an expense management solution. So let’s try and buy, you know, something cloud-based and then you buy that and you realize, oh, things doesn’t, things don’t fall apart. Uh, and then you try with maybe an HR system or you try with a CRM system. Uh, and, and, um, I think the maturity in organizations is definitely, uh, definitely growing, um, in, in, in this, in this space. And also as we talked about on the, on the data side where, where it’s going from, um, yeah. It’s going from, um, where, where you needed previously to, to build a much more yourself, where you actually able to get much more mature solutions right now, which can, um, which can help you, uh, which can help you with your, uh, with your data analysis need and their processing needs in a way that, uh, that you could only have dreamt of, uh, five years ago.
Eric Axelrod: And what are, what are some of the reasons where, um, you were, you have seen a cloud migration really kind of any kind be deemed as a failure Uh, you know, after they, after they, they went through a substantial portion of the project, they tried to put a system on it. They came back and said, this isn’t just gonna work. We need to roll this back.
Rasmus Bang: Um, I’m just trying to think here. Um, I, I think most of the, um, most of the failures I’ve seen have not have, have mainly in related to, um, you know, business process areas or, uh, missing alignment or missing data. Um, I typically don’t see a lot of, uh, a lot of challenges where you can say, all right, well that is, that’s, uh, that’s because of the, uh, the decision to use use cloud solution here. Um, the only things I can think of is that, um, is that, um, especially maybe especially to begin with, people tended, we talked about the, we talked about the infrastructure side of things earlier on and we talked about how there was some, um, uh, hesitation of, uh, of adopting a cloud. And I think that in, in many of the organizations where I’ve been, uh, people on the infrastructure side of things, I’ve also, uh, become much, much more, um, aware and mature when it comes to cloud, uh, cloud adoption. Um, so, so, uh, so that’s also, that’s also definitely helped in that area. Um, but, but typically, um, well, a few times, uh, I’ve, I’ve seen, uh, performance issues, but actually not so much in the data space, but more sort of in the, um, and sort of things like office three, 65, uh, those types of things. So it’s, it’s happened, but it’s, um, I’d say not more than, you know, any other average ID project.
Eric Axelrod: Absolutely. Yeah. I, I, I’d, uh, I would agree. I’ve seen, I mean, there’s been a few issues that I’ve seen where companies, um, they didn’t really put enough thought into their infrastructure and they weren’t doing a full cloud migration because usually, you know, companies don’t do that. They’re going to do, they’re going to do a little bit at a time. And whenever they do that, they haven’t really dealt with the capacity or bandwidth issues that they’re dealing with on their existing infrastructure. And so they, you know, and that’s probably the most common case that I’ve seen, is where they just really can’t get data out of their old systems fast enough and over the wire and landed onto the cloud platforms. And then they’ll look at the cloud system. And Go, wow, this cloud thing was a failure. But the failure is really what their ability to, you know, it’s really, it’s really, well it might be to do with the upstream network pipe.
Eric Axelrod: It might be to do with their on premise database that they’re running because that’s a very common problem obviously with, you know, with, with the, you know, existing data warehouse and data lake systems that are just capacity constrained and um, and uh, you know, and then putting cloud on top of that isn’t going to help because you still have to get the data there. So those are, those are some of the, those are some of the, the, the common ones that I’ve seen. And then I, I guess I, I probably wouldn’t call it a failure, but I would probably say, you know, along the lines, like I talked about earlier with lift and shift where, you know, yes, we can technically take all of our infrastructure and put it in the cloud, but we just for x, our costs or something like that. And so it’s not really, it’s not like it doesn’t work. It’s just a lot more expensive because we didn’t go through that effort to Kinda figure out the right way to do it whenever, wherever needed that migration. Right. I think the finance area would probably call that a failure. They probably would. Yeah. But so like I think that’s, that’s one of those gray areas where it is going to declare it a success and then, and then, yeah. And then finance is going to say, no, no, that was, that was bad. Right.
Eric Axelrod: Um, but, but on the other hand, but you know, just because it does cause more doesn’t mean, um, you know, I mean they, they could have very well solved some of their other problems definitely related to outages or related to the ability to take good backups and things like that. Because obviously that’s, that’s still a big issue with a lot of on premise systems, especially with relational databases to be able to, you know, have enough enough horsepower, enough disk io and enough CPU to actually be able to take those backups without, you know, causing a lot of other problems. And, um, and so it’s very, very possible that you could lift and shift one of those databases and practically eliminate that bottleneck. Right. Because, you know, you can through the, the, uh, hardware at that, that you need to be able to do that. So, you know, so that might be one of those things where finance says, I don’t understand the benefits, but it says, look, we have backups that work now. Right We didn’t before you pretty good risk mitigation there. Exactly. What are the biggest inhibitors that you see today and really probably for the next few years that, that, um, are keeping companies from being truly data-driven
Rasmus Bang: Um, there’s a, there’s a large, I think there’s a large technical component to it of actually getting the, uh, getting the analytic solutions up and running at the big data solutions. Uh, Eh, getting the, the right, getting them fed with the right data. But I think actually the, um, I think organizationally there are, there are big challenges, um, both from a, you know, resource perspective of how do you get the rights, you know, the right data engineers, how you get the right developers, how do you get the right data scientists, but, but, um, but some of the things that I’ve seen as well are, uh, have to do with, uh, with mindsets because, um, because being data-driven is, eh, it really is. If you’re a, let’s say you’re a sales or head of sales for a large company and you’ve been there for 20 years and you’ve worked your way up and you’ve, you know, all the ins and outs and you know, all the small things and you know what this customer want and you know that we need to do this in Q3 because of such and such.
Rasmus Bang: And then at some point you have this, um, data scientists type come in and say, oh, by the way, everything we thought we knew about sales is wrong. Um, so we should do it this way instead. Um, I think, uh, I think that there’s a, a large, uh, resistance to change on the decision maker level. Uh, and of course this is an extreme example, but it’s something that, uh, that I see in organizations sort of all the way down, where if a, if you have a data scientist saying, oh, by the way, I’ve made a new model for how we do our supply chain planning, then you can be absolutely sure that the a supply chain planners and probably also a, the, the it guys manning the supply chain, I’m going to say
Eric Axelrod: that can’t be right, exactly. You’re obviously missing something. This, this is bad. I don’t know. And whenever you do an ERP implementation, I’ve seen this numerous times with SAP, whenever you’re coming in to implement SAP for the first time, and it’s very much the same way, they’ve got a very specific process for how supply chain works, right And most likely that’s not how your department already operates. And so it’s this idea of going to the business unit and saying, okay, this is how you’re going to do things now. And usually that doesn’t go over very well, you know And so, you know, so you have to go do that with every single business. You’ve got to do the finance, you’ve got to do with manufacturing, with logistics, with customer service, with order fulfillment, you know, all the way down the line and kind of do that exact same process.
Eric Axelrod: And so it’s very much, yeah, it’s very much an organizational problem. Um, you know what I mean And then, and not even, not even necessarily, you know, whenever you’re implementing SAP, I’ve been on a few of those implementations, and that’s not a data project. That’s not anything that they’re looking at the data and saying, this is how we need to do it. You know, they’re, they’re putting a cookie cutter, you know, implementation for the most part on top of them saying, this is how you run a manufacturing business. It doesn’t matter what you’re making. This is how you run it. And, uh, you know, and it’s something that’s worked very well for a lot of organizations for 30, 35 years. Right. And so it’s not something that they’re, it’s not something that they’re really, uh, you know, like there’s not a lot of debate as to whether this is the right thing to do or not. But whenever you actually go and implement at a company, they’re like, well, wait a minute. This can’t be right because this is how we do things. Right.
Eric Axelrod: Um, and, uh, you know, and there was another conversation that I was actually having just yesterday with, uh, with, with another, uh, with another great, uh, um, uh, data leader that I know, and we were kind of talking about the idea of, uh, the idea of hippos that are sabotaging, uh, projects either intentionally or unintentionally for a lot of reasons. One of those reasons is because it kind of whatever you’re doing doesn’t align with what the way, you know, the way that they think things should work or the way they’ve done things in the past or something like that. And, um, and, uh, though those end up being, being, uh, you know, being really big problems to actually getting things done. And I actually, one of the things that see, and I don’t, you know, I can’t actually think of a company that I have, uh, that I have worked at or, or consulted at that wasn’t like, as we didn’t have at least one variant person in the company that’s a VP level or an executive director or a cxo that is the quote unquote expert in the application that you’re using.
Eric Axelrod: And as soon as you bring a data quality to them and you say, look, I found, you know, millions of dollars in sales in a place that you say that you don’t sell to. And they say, that can’t be right. That’s impossible. The data’s bad. You know, you need to go, you know, look at this again, or whatever. And it’s usually much more, that usually ends up being much more, um, difficult in that, you know, I’m putting it very, I’m putting it very, very lightly about how some of those conversations go. And, you know, but the idea is, I mean, the data doesn’t lie, right You can clearly see these transactions. You’ve got sales orders, you’ve got purchase order numbers, you’ve got skews, you know, these things exist. You bought them, they’re in your system, you sold them or whatever. And I’m showing that to you and you’re telling me that it’s wrong.
Eric Axelrod: Right Yeah. And so, um, there’s, there’s just a huge amount of that in organizations and as soon as you start to kind of roll out these systems that really, um, you know, kind of wipe that lens clean and allow the whole business to see how things are really working. There’s a lot of people that, um, that feel threatened by it because it really kind of upsets the way that they are used to doing business. And, um, and so, um, you know, the, that I think that that’s one of the big things that have, that have really kind of it, you know, inhibited that adoption is the yeah. Is I, I mean, I, I say it’s, it’s, it’s part of, it’s part politics, but it’s also part of the idea that, you know, we, we don’t like the way that things are going to work in the future because we have, you know, we have a way to do things.
Eric Axelrod: Um, and we don’t like the idea that this is going to be visible to other people in the business, you know, for, for all sorts of reasons. Uh, you know, and so some of them are, some of them are, um, are innocuous and some of them are not an oculus. I mean, I’ve certainly seen numerous examples in the past of just outright fraud that’s been implemented by, you know, a pretty high level executive that we found pretty quickly and pretty easily, you know, and then we show that to everybody and they say, no, you can’t show these numbers to anybody, you know, and so, you know, I was like, what if we, you know, you get understanding if we, if we rolled the system out, this is something that, you know, it’s going to be very obvious to everybody that this is happening. Right. And so that, that’s, that’s been one of the, one of the really big, um, one of the really big, um, blockers that I’ve seen for a lot of projects.
Eric Axelrod: And I have seen that numerous occasions where executives pull the plug on a project because something like that’s happening because, well, wait a minute. You know, we thought we were going to get something, uh, you know, um, you know, quote unquote actionable out of this. Instead, we found that one of our guys is committing fraud or that we found, you know, that we, you know, that we’re just have this ridiculous waste over here, but, you know, but this has happens to be an influential person in my company. We’re not going to do anything about it, you know, and we have no intention of doing anything about it. We just want to like, we want to do these other things and so, you know, and then they’ll kind of look at the whole product and say, no, this is gonna, you know, this isn’t gonna go over well politically for us to roll the system out.
Eric Axelrod: And then so they’ll, they’ll either go, either roll it back or they’ll do something along those lines where, you know, you don’t end up actually going live with whatever thing that it is you’re working on because you’re stepping on somebody’s toes and you know, and you’re, you’re making somebody look bad again, whether intentional or not, it doesn’t really matter so much. Right. Cause there’s, you know, you’re either exposing, um, you know, in some cases to grow some competence or you’re exposing, you know, business process failures or you’re exposing fraud or who knows what you’re going to find. Right. But, um, it absolutely happens. And that, so that’s one of the things that I’ve seen that have been big, um, that have been really big blockers for actually getting, you know, for actually getting data products and data platforms, you know, getting, getting the business to actually finish implementing them and start using them and actually start using them to change their business. You know, you’ve really got to have, you’ve really got to have that top down directive from the c suite that says, yes, we’re going to do this. And everybody really kind of needs to do to agree, or at least the CEO needs to have the authority to stand up and say, we’re going to do this. You know, and you can’t cave to that political pressure in your org, in your organization. Because if you do, it’s gonna, you know, it’s not gonna work.
Rasmus Bang: Exactly. Yeah. Yeah. I think also the, um, sort of the mindset thing, Eh, at the, at the sort of the business analyst level of, of people tending to, to, you know, try and find answers on their own, which in and of itself is a, is a fantastic trait. Um, but, but, but which often gets us to, um, to a lot of eh, faulty conclusions and a lot of reports which are just, uh, incorrect and inconsistent. Um, so I think a lot more, a lot more, uh, understanding and, uh, support in saying, well, this is, you know, this is it, eh, and it’s data, but it’s also, uh, something completely different. It’s also, you know, people trying to get answers and we should try and help them get the right answers,
Eric Axelrod: trying to get correct answers right back then.
Rasmus Bang: So, you know, the, the flip side of that coin is, uh, get them to help define, well, what is a correct answer because, which of these five definitions of financial turnover is actually the right one,
Eric Axelrod: which, which is another thing that we’ll, we’ll, that will, and, uh, you know, in a lot of cases arguing with some soon, some influential hippos, amount of the number of organizations where, you know, you’ll go to somebody and you’ll say, well, you know, how do you measure employee turnover And then they’ll come back and they’ll, they’ll say, well, what do you mean there’s only one way to do it And then go, go survey their organization, find out that they are using least, you know, four, six, eight, 10 different ways to do it depending on who you ask, you know And so, um, you know, that’s a huge, huge mindset shift to be able to, you know, throughout the whole organization to say, okay, we’re going to have a standard definition for how we’re going to do things. And on top of that, you’ve got to have, you’ve really got to gotta have standard systems because as you know, as long as you’re allowing people to kind of do things in excel on their own, you really can completely through that visibility what they’re doing.
Eric Axelrod: And you can’t, I can’t really audit it. Whereas if you have it in a govern system, you know, it should be working the same way throughout the entire, you know, right throughout the entire, um, a life cycle of, of that data, you know, and um, you know, and that doesn’t mean that people can’t create over on dashboards. They absolutely can and they should. But you know, we need to make sure that those, that all of those definitions, you know, that you have a data dictionary that defines what employee turnover is, how you calculate it. And that is enforced uniformly throughout the organization. And that’s that. And that’s one of the things that I am, I think, I would say very, very few organizations I think have done ever really kind of done correctly because there’s almost always some, you know, some people in some, some departments or some business units that can kite the kind of do whatever they want to and they’ll just, you know, download all of the data.
Eric Axelrod: They’ll do all of their work in excel or they’ll build their own access databases or something like that. Oh No, they have the, even though they already have a thing that does this. And uh, you know, and then they’ll try to recreate the wheel and they’ll end up recreating it. Not to necessarily say that it’s wrong, but it’s different. And uh, you know, and so now we’d, now we’re back to having multiple definitions of what is employee turnover and how to calculate this number. And then corporate says, you know, give me your employee turnover number and here’s a number, which is fundamentally calculated differently than the rest of the way the rest of the business does it, you know And then, and then they’ll start to measure you with those KPIs and say, okay, business unit, you know, you need to get your number from x to y.
Eric Axelrod: And, um, you know, in, in, in, in, in whatever that happens, right In performance management or organizations, that’s another, you know, another problem on that side where most of those performance management people don’t understand that this is a problem. You know, they don’t, the concept that you can measure this, this metric two different ways doesn’t even, you know, doesn’t even register. And that’s how they’re met. That’s how they’re measuring their employees performance. And so, or that’s how they’re measuring the business units performance. And so it makes it very, very easy for, for them to, uh, for them to, um, you know, have, having correct data. It’s also easy for them to manipulate their numbers, uh, either intentionally or unintentionally with, with exactly what it is they’re doing. And, um, and, uh, you know, and I’ve seen a number of organizations that, that this is well that, um, next to, you know, kind of went, tried to go through this process and said, okay, Lena, we need to standardize the way that these things are calculated.
Eric Axelrod: And they just, you know, they would spend a lot of money and a lot of time on it and they figured out it was just kind of too difficult to do politically and they kind of just gave up, uh, you know, as far as going to all of the business units and say, you know, okay guys, you need to number one, stop using excel and access and all of these to do these things get on and everybody needs to be using the same platform. But then number two, you know, you, you have to get everybody to agree on how these things are calculated and you know, create standard processes for collecting this data. And, you know, and you know, usually, unless it’s a really, really, really bad problem and organization usually doesn’t want to go through because it ends up, especially the big companies. I mean very, very intrusive and expensive to do.
Eric Axelrod: Um, one of the big areas where I’ve seen this a fair amount as well as in as human resources where, you know, w where w where you’re, you’re, you’re trying to get you to some very basic data about, you know, when people are hired, when they’re fired, they’re not even taking into account salary data or anything like that. It’s very, very, very difficult to get human resources data and, um, you know, and so, you know, when you’re, whenever you’re going and implementing all these change management processes that says, okay, we’re going to do this thing, we’re, we’re going to start measuring our business units on employee turnover, for example. And then we want to actually quantify that data. It becomes a very, very difficult thing to do because, you know, you’ve got the people on the business side that they don’t want to give you the data and, uh, you know, they’ll just kind of spoonfeed you a little bit that say, Oh, this is, this is all that you need.
Eric Axelrod: And, uh, you know, and that completely, uh, completely, um, uh, eliminates that data governance and change management side of it because, you know, there’s nobody, there’s nobody double-checking these numbers to make sure accurate, you know, and, um, you know, it, you know, and I think in a lot of cases you’ll find out when, if you actually do get access to it, that they’re not. And, you know, and a lot of those cases, they’ve never been right. And I’ve, I’ve, I’ve seen, I’ve seen quite a bit of that we look at and go, wow, you know, your numbers are off by, you know, by, you know, 20%. And they’ve been like that for decades. And, uh, you know, and, and that’s another one of those, one of those, those, um, instances where, you know, the, the, the stakeholders don’t want to though, they don’t want to believe. You Think, oh, that’s not possible. And you know, we’ve been operating since the 1960s and this is never been a problem. I’m like, well, you, just because you weren’t looking at it doesn’t mean it wasn’t happening. Right.
Eric Axelrod: So, you know, from, from, you know, from, from my point of view, even though those are some of the really big ones for, for, for inhibitors on being data-driven, it’s just the ability to, you know, is the ability to trust your data, to understand your data and then be able to not be threatened by it and ultimately ultimately act on it. Right. Is that okay, we’ve got, we’ve got a system, we have a uniform way to agree on what employee turnover means. Now we’re going to actually go use this data to do something that it sounds very trivial, but there’s just so many steps that happen that you have to get through to get from point a to point B to point c, um, and get everybody in alignment before you can finally say, you know, yes, we can really, you know, we can really execute this. Right. Yeah, exactly. So that, that’s excellent. Um, that was my last question. Do you have any, any final thoughts The floor is yours.
Rasmus Bang: Oh, I think we’ve covered it pretty, uh, pretty well. I, I think I’ll say that, that, I mean, given how much this area in this domain has changed and evolved in the last 50 years than the last five years, I’m very, very curious and excited to see what happens next. Um, uh, because I think that there are going to be some, some big shifts. And at the same time, I think that, you know, the things we discuss here at the end, I think they’re just gonna. I think they’re everlasting. So, uh, so I think we should, I think we should schedule a followup and, uh, five years from now and then see, look, it,
Eric Axelrod: I, I agree. I have a, I have a hunch that, uh, that most of these organizational problems that are not going to be any different than they were five years from now and, uh, and that, uh, and that our technology, our technology landscape is going to have another pretty dramatic shift probably in five years. But it’s not any of the fundamental problems around, around data quality or around the adoption or the things that cause the things that cause, um, projects to succeed or fail in the eyes of the business. Right.
Rasmus Bang: yeah.
Eric Axelrod: very interesting. Well, so I, I think very much for your time. Uh, Rasmus, this was great. I know it’s a, I know it’s a little bit late over there, but, uh, it’s been a, it’s been a, it’s been a fun conversation. Hopefully we can do it again. Uh, hopefully we can do it again sometime. And, um, and we’ll, we’ll, we’ll make sure to put all of your contact information in the, um, you know, in, in the link, in the, in the post, in the videos so everybody can go meet the, everybody can go meet Rasmus. Thank you very much everybody for watching the Cloud Data Summit podcast.