How APM, Observability and AIOps drive Operational Awareness

Ron Williams explains all to Jon Collins

Jon Collins: Hi Ron, thanks for joining me! I have two questions, if I may. One’s the general question of observability versus what’s been called application performance monitoring, APM – there’s been some debate about this in the industry, I know. Also, how do they both fit in with operational awareness, which I know is a hot topic for you.

Ron Williams: I’ll wax lyrical, and we can see where this goes – I’ll want to bring in AIOps as well, as another buzzword. Basically, we all started out with monitoring, which is, you know: Is it on? Is it off? Just monitoring performance, that’s the basis of APM. 

Observability came about when we tried to say, well, this one’s performing this way, that one’s performing that way, is there a relationship? So, it is trying to take the monitoring that you have and say, how are these things connected? Observability tools are looking at the data that you have, and trying to make sure that things are working to some degree.

But that still doesn’t tell you whether or not the company is okay, which is where operational awareness comes in. Awareness is like, hey, are all the things necessary to run the company included? And are they running okay? That’s what I call full operational awareness. This requires information that is not in it to be combined with information that obviously IT operations has, and AIOps tends to be the tool that can do that. 

So, Observability solutions serve an important function; it allows you to see the technical connections between objects and services, and why and how they may work. Awareness includes that and adds functional analysis, prediction, and prevention. But I’m not just talking about operational awareness as a technical thing, but in terms of the business. Let’s look at HR – this has an IT component, but nobody looks at that as a separate thing. If HR’s IT isn’t working, and if I’m the CEO, as far as I am concerned, HR is not working, and so the company is not working, even if other parts still function. 

So, how do I gain awareness of all the pieces being brought together? AIOps is a solution that can do that, because it is an intelligent piece that pulls data in from everywhere, whereas observability is taking the monitoring data that you have, and understanding how those data relate to each other. APM gives information and insights, observability helps solve technical problems, whereas AIOps tools helps solve for business problems. 

AIOps platforms are one tool that can combine both data sources real time IT operational awareness and Business operations awareness. Together, these constitute Organizational Awareness, that is, awareness across the company as a whole. 

Jon: For my take on the benefits of observability platforms, bear with me as I haven’t actually used these tools! I came out of the ITIL, ITSM world of the 1990s, which (to me) was about providing measures of success. Back in the day, you got a dashboard saying things aren’t performing – that gave us performance management, anomaly detection, IT service management and so on. Then it went into business service management, dashboards to say, yeah, your current accounts aren’t working as they should. But it was always about presentation of information to give you a feel of success, and kick off a diagnostic process. 

Whereas, observability,… I remember I was at a CloudBees user event, and someone said this, so I’m going to borrow from them: essentially, that solving where things are going wrong has become a kind of whodunnit. Observability, to me, is one of those words that describes itself. It’s not a solution, it’s actually an anti-word, it describes the problem in a way that makes it sound like a solution, actionable insights. It’s the lack of ability to know where the problems are happening in distributed architectures. That’s what is causing so much difficulty. 

Ron: That’s a valid statement. Operational awareness comes from situational awareness, which was originally from the military. It’s a great term, because it says you’re sitting in the middle of the field of battle. Where’s the danger? You’re doing this, your head’s on a swivel, and you don’t know where anything is. 

So operational awareness is a big deal, and it feeds the operation of not just IT, but the whole company. You can have IT operating at a hundred percent, but the company can be not making a dime, because something IT is not responsible directly for, but supports, is not working correctly.

Jon: I spoke to the mayor of the city of Chicago about situational awareness, specifically about snow ploughs: when there’s snow, you want to turn into a street and know the cars are out of the way, because once a snowplough is in a street, it can’t get out. I guess, from the point of view that you’re looking at here, operational awareness is not the awareness that IT operations requires. It’s awareness of business operations and being able to run the business better based on information about IT systems. Is that fair?

Ron: Yes. For the monitoring, are my systems OK, and is the company? Observability is, how are systems and the company behaving, why are they behaving that way, and what’s their relationship? Can I fix things without anything happening, and causing incidents? Awareness is a whole company thing – are all parts performing the way they should? Will something break, and if so, when? And can I prevent that from breaking?

That’s why operational awareness is more than situational awareness, which we can see as helping individuals – it’s aimed at the whole company, working with business awareness to drive organizational awareness. I’m not trying to invent concepts, but I am trying to be frank about what’s needed and how the different groups of tools apply. Operational awareness includes observability, monitoring, reporting and prediction, which is where AIOps comes in. You get all the pieces that we all know about, but when you put them together you get awareness of the operation of the company, not just IT. Observability and monitoring doesn’t include anything about business operations. 

Jon: Is there another element? For the record, I hate maturity models because they never happen. But this is a kind of developmental model, isn’t it? From monitoring, to observability, and from this ability you want to improve to awareness. What you can also do is think upwards, from basic systems management, to IT service management to business service management. 

Business service management was great, because it said (for example) people can’t access the current accounts. That’s really important, but what it wasn’t telling you was whether or not that’s doing you any damage as a company, so you can work across monitoring, through observability to operational awareness.

Another question, then, where can you get this operational awareness thing? I don’t suppose you can go down to Woolworths, pick up some operational awareness, stick it on a pallet, and wheel it home, so what do you do? 

Ron: For a start, you must have all the pieces – if you don’t have monitoring, observability and all that you can’t get there, right? But then, one of the biggest pieces that’s missing is business awareness. The business, generally speaking, doesn’t communicate its operational state. This makes it hard – if my database isn’t running, what’s the impact of that? What does it mean to be fully aware? We can see this as a Venn diagram – if I draw another circle, it’s the whole circle, it’s the company.

Jon: Hang on, this is super important. If we go back to the origins of DevOps (we can argue whether or not it’s been successful since two thousand and seven, but bear with me on this), the origins of it were things like, “Black Friday’s coming up. How can we have the systems in place that we need to deliver on that?” It was very much from left to right – we need to deploy new features, so that we can maximize benefits, we need to set priorities and so on. 

But the way that you said it was the business is not closing the loop. It’s up to the business to say, “I’m not able to perform. I’m not able to sell as much as I should be at the moment. Let’s look into why that is, and let’s feed that back to IT, so that I can be doing that better.” You’ve got the marketing department., the sales department, upper management, all the different parts of the organization. Then all need to take responsibility for their part in telling everyone else how well they are doing. 

Ron: Absolutely. I almost put a fourth circle on my Venn diagram, which was the business side. But I decided to leave this, as it was about awareness as an intersection. It’s odd to me that many companies are not aware of all the things that are necessary to make them function as a company. They know that IT is a big deal, but they don’t know why or how or what IT’s impact is.

Jon: Yes, so bringing in elements of employee, experience and customer experience, and all those sorts of thing which then feeds the value stream management, strategic portfolio management aspects, knowing where to make a difference, shifting our needle according to the stakeholders that we have. 

Ron: Yes, and all of that’s in awareness, you know!

Jon: That’s a great point to leave this, that the business needs to recognize it has a role in this. It can’t be a passive consumer of IT. The business needs to be a supplier of information. I know we’ve said similar things before, but the context is different – cloud-native and so on, so it’s about aligning business information with a different architecture and set of variables. Thank you so much, Ron. It’s been great speaking to you.

Ron: Thank you for letting me share!

The post <strong>How APM, Observability and AIOps drive Operational Awareness</strong> appeared first on GigaOm.