Ep. 100: Breaking Down the Barriers of Traditional Architecture

Peter: 0:05

Welcome to Definitely, Maybe Agile, the podcast where Peter Maddison and Dave Sharrock discuss the complexities of adopting new ways of working at scale. Hello and welcome to another exciting episode of Definitely Maybe Agile with your hosts, Peter Maddison and Dave Sharrock. So what are we talking about today, Dave?

‍Dave: 0:21

So, Peter, I've been spending this week at Agile 2023 just to put a sort of time stamp on today's conversation, surrounded by a lot of people talking about agile practices and so on, but not so much on the technology side. There are technology talks here. I'm just not necessarily going to some of those, but what's happening is in the conversations in dinner. Technology is still a big conversation, Architecture, things like this so let's maybe revisit that. I don't think we've touched on that for a while.

Peter: 0:49

Sure, and it's an interesting one, right, and I see this a lot where new ways of working come to the forefront and it's like we're all going to do agile, so we're going to form scrum teams, and the scrum teams are totally capable of delivering everything all the way into production, except the underlying architecture and technology doesn't support that, and it's very, very difficult because you're working across organizational boundaries in any kind of large system. That makes this tough to do, and I've seen organizations when they start down a transformation journey where they've pretty much gone down the route of well, we're going to fire all the architects because the architects weren't doing anything useful anyway.

Dave: 1:28

Dangerous words, that one isn't it and also that, I think, creates whether it's that specifically or but a lot of the time the reality of that architecture and how tough it is to change, creates a barrier to that change and becomes an excuse or a reason to slow things down or to only take the change to a superficial level, for example. So not only is the architects accused of not doing things or not doing anything visible, but they're also. Everything that they have done is slowing the organization, the transformation down, which I think is definitely wrong.

Peter: 2:02

And I think that some of that comes from a, if you want to call it, an old school architectural way of working, where it would be. You have an architecture team that is sitting in an ivory tower sending out edicts and for anything to move forward in the organization it has to be ratified by the people in gray cloaks and hoods who sit in that ivory tower and cross dictates on and I'm thinking of the. I know one organization that I've worked with that has two layers of this, where you go through the first sort of technical review and if it's deemed that your sins are bad enough, you get sent to the really bad guys who like do a bigger review. But then all of this requires like 120 page decks and a lot of conversation. And if there's feedback, if that larger group of people say only meets once every couple of months or once a quarter and you have to go through a couple of iterations of that, you can very quickly see how this is just grinding everything to a halt in the organization.

Dave: 2:59

Well, and I always, I always want to draw attention to the fact that I think that's a consequence of the system they're working in. So sometimes it feels like we just point our fingers at whatever it is. And, of course, the architecture function is about stability, about scalability, and it has to be secure and stable and changed in a thoughtful, deliberate way. So back in the day, that was how they maintain that stability. But I think when you're, when you're responding to change as rapidly as so many organizations, when you're many of your, your solutions, your systems are in the cloud, or actually in the cloud. There's a lot more moving parts, there's a lot more responsiveness needed in that architecture function.

Peter: 3:40

I completely agree, and I think that's often where that juxtapose is coming from. If you have architecture hasn't evolved with the way that workers evolved and the rate of changes evolved into the systems, then it is going to become that barrier. But many organizations have overcome that and there are lots of other good models out there where we start to look at architecture in a slightly different light, where it isn't just the people with the expertise over here and everybody else has to listen to what they say. There's this. I mean, one part of that and you and I have talked about this in the past is this concept of white box and black box architecture thinking of the different layers at which architecture occurs in the organization, because architecture's architectural decisions are occurring all of the time throughout the organization. We look at how are we going to have this service? Talk to that service. What are the? How are we going to build this thing? What does that all look like? And so architecture occurs within the teams, at the team level, where individual delivery teams are working at how they're going to solution the particular stories that they're working on at this time. It also then occurs at the system level, within solutions architecture, where we think about all of the components of the system. How do those components talk to each other? And that's all sort of what's happening inside the system. That's the white box architecture, the stuff that we see inside the system. And then you have the black box architecture, where we can't see inside it. But we have all of these components, these systems, across the organization where we should need to talk to each other, and that's where enterprise architecture starts to come in. And the other side of that that you're seeing now over the past. So what is, as well as the concepts around business architecture and how business architecture aligns in with this as well, and I think, it's almost a product of the more broadly adoption of technology into different aspects of the way that an organization operates.

Dave: 5:34

Well, we've talked many times about technology being a partner, a strategic partner, and not something that is a service function, and what you've described me, that that means using, I think, a lot of the practices and the principles that we expect to see in a technology group but really across the business in some ways, just because they're there for a reason, they allow that standardization and repeatability and a lot of these different things that digitization is going to break together as one.

Peter: 6:02

Yeah, and then it's that reduction in risk that comes from having standardized pieces that we know that we can pull on to put together. It both helps us accelerate, but it also reduces the amount of risk there is in the system because we've got more confidence in how these are going to behave. So there's all of those pieces that come in it also I know I've encountered this on a number of occasions. It seems to be one of the most common ones is where somebody goes out and tries to reinvent something that's already built into the system, a classic example being the authentication system. It's not going to rely on the authentication system exists. We're going to build our own, and then you're spending all of these cycles building something that doesn't really add any value to the product. They could have just used what's there. So there's the pieces of that. So having that visibility into those is another critical aspect of what architecture is there to do.

Dave: 6:54

So talk a little bit about what you want to see If you go and talk to an architecture group. What are the things that you're expecting to see if either? Yeah, what do you want to see in a responsive, aligned architecture group for an organization that's changing rapidly?

Peter: 7:11

So it's happening at a different layer. So from the team level of architecture, where they need to have accountability and ownership of how they're going to put these together. So they need tools to be able to do that. They need to know what parts are available for me that I could start to build things out of. What patterns should I be following? What does that look like? So architecture, providing them with that and being available for the conversations around how should I go about doing this is definitely a key element that we're looking for At the solutions level architecture, solution architects. One aspect of this is, especially at that level, that solution architects are existing within a, not like a centralized team that gets pulled off in a million different directions all of the time, that they have time to work within the domain, to understand it over a period of time, that they can see how the system starts to come together, because I've seen that as a common barrier to success as well where, essentially, you pay by the hour for your solution architect and the solution architect then goes off and does something else, at which point the context switching to another project. But I need another decision made and now the context switching is going to kill you, you're not going to have the informed understanding of what the system is and how the system is evolving. So having better alignment between architecture and domains that they're operating in, I think is critical so that they can learn them in more depth. And then, at the enterprise architecture layer, it should be a more pull mechanism than a push, where they enterprise architecture is looking at understanding the systems, pulling information from what's occurring to build up what the model of the interactive components are, and not preparing 120 page decks to submit to a quarterly architecture review board. Those occasions where that type of congruence is necessary should be very, very rare and very, very far between. And there may be reasons that you would need to do that, like there's a sufficiently large architectural decision which will alter the direction of the organization that you feel that everybody needs to come together and have this. But if it's a recurring event all of the time, then that's not necessarily a good way of going about it.

Dave: 9:24

The enterprise architecture level that you're describing now. I've always liked the idea of architecture as a service and what I think of there, and I don't mean any technical solution in that sense, but what I mean it's almost like just in the same way, a product owner is going to go out and talk to their customers and their clients and their end users to find out what problems they're dealing with and how can the product better serve the space that the end users, the customers, are working in. The architecture team, being a really actually set about pulling the ideas in, is going out to see what the business is needing, where things are going, what's happening, so that the architecture becomes something that serves the greater you know the direction of that, of those systems and the problems that they're trying to solve, rather than being something which is disconnected from what's really going on and is a sort of thrown over and is the building within which the teams have to work. But they've not had any input into how it's been put together.

Peter: 10:22

I don't know if that makes sense, but it does, and I think architecture also has that a role in going outside of the organization too, building the Chancellor Gallo. What else is there that could solve some of these problems, like what are we looking at, what's coming down the pipe and how do we bring those things into the table as well?

Dave: 10:39

I mean the base of change in technology means that's increasingly essential, right? So thinking how many organizations get caught out with new technologies coming to the table? Just all the time we see that the ones that make it in a popular press, but also lots and lots of changes under the hood as well that don't make it into the popular press. That you know architectural teams need to be aware of as they're coming through so they can make good decisions.

Peter: 11:03

I think there's a piece too here where there's clear understanding at every layer of the organization as to who's accountable for the technical architecture, like who to go to to ask questions of. So there's that clear. This is who I need to tap on the shoulder when I have questions and making sure that that is sufficient slack in the system to enable that so that we can look at how we do things and learn and move forward and grow the organization. That sense as well. And of course there's also the size of the organization plays a big part of this too. Right, it's in smaller organizations. It's much easier to keep all of that aligned. In larger organizations you need more layers and more people and more scale to like create that alignment across everything.

Dave: 11:46

It's interesting you talked about slack a couple of times now whether it's overloading, that they're sort of architected by the hour problem, or just slack for architectural teams to look ahead and learn and discuss things. And a lot of that has to do with, like I see, the best organizations. Architects are continuously communicating, the architecture group continuously communicating to the development teams, to people who have to use that architecture to understand their thinking, to understand where to go, how to solve problems, what the intention is, where it will work really well. So that education piece of communication and knowledge sharing is also such a hugely important piece. Sometimes we get bogged down with the problem in front of us but we're not taking that time to look ahead and look around and learn how people are solving problems and where the challenges are coming from, and so one of the key artifacts, of course, is design documentation and architecture, which always brings up an interesting piece right.

Peter: 12:50

There is a piece where, if I've got, I'm going to have a smaller number of architects than I do teams almost certainly and for the architects to be able to understand what the systems are doing, they need to be able to have some place they can go and look at that or people they can go and talk to to understand what is the architecture that we're working towards. How do these moving parts fit together? So there is also, from a risk perspective and many in a more regulated industries, there is regulation around ensuring that. Do you have up-to-date design documentation that demonstrates what the architecture of your system is? And so there is reason to maintain this, because external regulatory bodies see it as a risk too. So I can say it's not just me, but this sometimes comes into conflict with the interpretation of agile is not documenting anything.

Dave: 13:44

Well, I mean, I think it's more nobody really. I mean, there are definitely people who love documentation, but I think many people who are at the cutting edge of working on things, the last thing they want to do is sit down and write everything thoughtfully down. So getting that, the right amount of documentation, is so critical, of course right.

Peter: 14:03

Yeah, it's a balance. I mean, I've got to be able to have something that I can look at to understand what are the core moving parts of this system and how are they being put together? I mean, it's a certain aspects and from a security architecture perspective, for example, if we look at things like threat modeling, to do threat modeling at scale across organizations, you need teams to be able to take that on themselves, which means they need to understand what that means and, as a part of that threat modeling exercise, they're going to have to create models of the data flows of the various component parts of their application to understand what talks to what and how does it talk to. How do each of the pieces talk to each other? Because, I mean, those are some of the key things that we're going to look at for from a security perspective, like what does authentication look like? How are we handling transfer of data between different components within the system? How is data being stored at rest, that type of thing? So where is it being stored? So all of that type of information is critical, and so to capture that, you're going to write it down, and so we need to know that that, but we also got to make sure that's being kept up to date, and so that requires that there is some effort being put into that. So there's a bit of overhead in ensuring that that documentation exists and is being kept up to date and that we're maintaining that in some fashion, and there are a few different ways of going about this and there's definitely tooling in the industry now that can take some of that burden off. But there's always and I think this is the other critical part in that architectural spaces there's always that need for the conversation, because we can't necessarily 100% trust what the documentation is going to tell us. But we also need that relationship to say hey, are we, are we talking often enough about the kinds of things that you're doing? And there's how fast moving space that you operate again, and what else could I do to help you, like looking at your environment and helping you solve any problems that you have in front of you?

Dave: 15:59

So I think we've kind of explored quite a range of different things there, and it's as always. I always love these conversations because we're opening up the doors to so many other things that we can dig into and chat about. A couple of key takeaways you're working for. Any of our listeners are working in an architectural group. Is there questions they should go and ask or things they should go and look for? What would you suggest?

Peter: 16:20

So I think some of the key pieces for me are always like do we know who the architects are? Are architects aligned to a particular business domain or a particular business architecture, or are architects basically you dial them up and you get them for an hour and then they disappear again, because if you're operating in that latter model, then you're what you'll find is that the architects that can have little understanding of your actual domain, so what they may end up suggesting as a path forward may not even be the right way. So thinking about how do I better align my what I'm doing from business architecture perspective to the systems that I'm building, I think is a critical part. Thinking about the conversation, the questions, the different layers of the organization, acknowledging that architecture is occurring everywhere, and how do we capture that information and radiate it. I think my third point would be avoid ARBs as much as you possibly can, because they at least sequential ones. There are reasons that there are reasons that it would make sense to come together and review an architecture in depth, but those tend to be. They should be few and far between. It shouldn't be that normal modus operandi is that we have a architecture review board every quarter, which would then set the cadence for any kind of innovation in the organization, anything you would add.

Dave: 17:39

What I took away is that idea of Slack, of just giving and I'm just coming from conversations around huge capacity problems in so many of the organizations that we work in and I think that recognition of Slack is so important. And I also love what you said about knowing who the architects are, because it's easy to push them off into a corner and we should not. We should be engaging and we should be bringing them into the conversations, because they need to be more involved not less involved, definitely so with that.

Peter: 18:06

Anybody has any feedback, they can send it to us feedback at feedback@ definitelymaybeagile. com. And don't forget to subscribe. Hit that button. It usually has a big thumb on it or something, I think. Until next time. Until next time, thanks again. You've been listening to Definitely Maybe Agile, the podcast where your hosts, Peter Maddison and David Sharrock, focus on the art and science of digital agile and DevOps at scale.

‍