From good to great #2: Fail or prevail.

As I mentioned in the first part of this article series, I truly believe that the agile approach to software development has been one of the greatest things that has ever happened to the industry. However, there is still quite a lot of room for improvement in the agile methodologies and their underlying principles.

Is agile agile?

Today's business climate is more competitive than ever. The world has shrunk significantly due to globalization and there is always a bunch of competitors breathing down your neck, threatening to do things as well as or even better than you. Competition will be even harder in the future since each and every company is streamlining its business and trying to find an edge, differentiating themselves from their competitors. No one expressed this better than the Canadian prime minister Justin Trudeau;

- The pace of change has never been this fast, yet it will never be this slow again.

To be able to stay ahead of, or at least to keep pace with, their competitors, businesses must be proactive and responsive. This is where business agility becomes important. Companies that aren’t able to innovate and to handle changes rapidly will be outcompeted.

How does this concern us as software developers? We can’t hardly be responsible for our customers’ businesses, or can we? I’ll say we can, and we are. Essentially every part of today's businesses is somehow supported by software. If the business changes, it’s highly likely that the supporting software needs to change as well and it needs to change quickly not to become a bottleneck for business agility. Software solutions must be designed and structured in such a way that it aligns with the business it supports. It must be possible to identify, implement and release necessary software changes in an efficient and effective way that is long-term sustainable and doesn’t accumulate technical dept.

Agile software development can actually have a negative impact on business agility.

So, does the fact that we are developing software based on agile methods mean that the resulting software allows the business to be agile? I’ll say that it doesn’t. Agile methods and business agility is not the same thing. On the contrary, agile software development can actually have a negative impact on business agility. Agile methods have a strong focus on delivering software fast in short iterations. Delivering software fast is of course in itself good for business agility, at least short-term. The problem is that prioritizing fast delivery in short, time-boxed iterations strongly invites us to sacrifice long-term quality, well thought out structure and architectural software design. Thereby, there’s a significant risk that the resulting software will be messy and increasingly hard to maintain and develop further over time.

Quality in this context is much broader than writing bug-free code. Off-course the code should be as bug-free as possible but quality in the context of software supporting business agility is all about structure, structure and structure. Oh, by the way, did I mention that structure is really important?

The important stuff

Now you may be asking yourselves why structure is so important and what it actually means. Well, structure is about how software solutions and systems are divided into smaller and larger components and how they depend on - and interact with - each other. It concerns everything from how the software is connected internally on a low level to how major components, such as systems and services are connected and communicate with each other. Structure is extremely important since it determines how well the software solution will age and how long it can actually live. Every component, from small, low-level entities in code such as classes, methods and functions to larger units such as services, should have a clear responsibility and should be responsible for one thing, and one thing only. Related business logic and functionality should also be clearly gathered and isolated within one single component. As a fitting example, let’s consider a service-based e-commerce solution. A service responsible for handling orders should definitely not also handle products. Business logic related to orders should also not exist in other components external to the order service.

Furthermore, each single component should be as self-contained and independent from other components as possible. Changes in one component shouldn’t result in a cascade of changes in other components. Software solutions that are structured in this way will support business agility. If and when (and trust me, this will happen often) the business changes, the required software changes will be easy and quick to identify, implement and deploy. Since the affected business logic is isolated in one single component, changes will only be necessary in this component and those changes will not force other components to change as well.

Well-structured software requires a lot of clever thinking and analysis and will definitely not emerge by itself. Well renowned software scientist Ralph Johnson once said;

- Software architecture is about the important stuff. Whatever that is.

Following the reasoning above, structure may be the single most important thing in software development and should be a matter for software architects, which it definitely is. This fact illuminates another, potentially devastating, flaw when it comes to agile software development methods. Agile methods, such as scrum, states that there should be no pronounced roles within a development team. I.e. there should be no single person that’s responsible for the software architecture. To be honest and clear I strongly believe that is pure and utter nonsense. Actually, the agile principles don’t state that architecture should be ignored. What they do say is that the responsibility for architecture should be shared over the whole development team. In theory this is a beautiful thought but as we all have learned the hard way, in practise everyone being responsible means no one being responsible. This sloppy attitude invites to an irresponsible approach to architecture, i.e. on the “important stuff”. I would even say that this principle is often grossly misinterpreted in such a way that there are no persons with architecture skills included in the development team at all.

Cold spaghetti

Insufficient focus on software architecture and structure within a project will undoubtedly lead to a phenomenon called accidental architecture. This means that the architecture grows in an uncontrolled way during the project, based solely on technical decisions from individual developers, without a clear, long-term plan or a bird’s-eye view. This is probably the most devastating anti-pattern within software development and the inevitable result is what software developers refer to as spaghetti code. Sorry pasta lovers! 

 

Spaghetti is actually a great metaphor for code being complex, entangled and difficult to sort out. This problem worsens over time as the software is maintained and further developed, trying to build upon and add to already messy code. If you’ve ever tried to tangle out a sticky lump of cold spaghetti you know what I mean. Over time, things will take exponentially longer to change and cause an everlasting flow of unforeseen side effects and bugs. Eventually we reach a point in time where the software simply can’t be maintained and new features can't be added with a reasonable development effort and finally goes where software goes to die.

Any developer can create software that works for the moment but it takes real skills to create software that works in the long run.

Accidental architecture and poorly structured software are sneaky bastards in the sense that everything might look prim and proper initially. The project itself may be considered a huge success since the software is delivered according to the customer needs and requirements and is working like a well oiled machinery. The problem is that structural flaws will not show up initially. Instead they will bide their time just waiting to hit you in the face later on. And believe me, they can deliver quite a punch. Software projects that seem highly successful at first may devolve into a smelly mess after some time of changes and additions. Architecture is a tricky subject for customers since it isn't something they can immediately perceive. Any developer can create software that works for the moment, but it takes real skills to create software that works in the long run. Structure and good, well-thought out architectural design becomes most important when the development project is finished and the software enters the maintenance phase.

The silver bullet

Alright, I think you get what I’m saying by now. Bad structure is bad, good structure is good. But what is good architectural structure and how do we achieve it? The answer is, as so often, that it depends. Above all it depends on the customer and their business. One thing that’s certain is that structure and architecture are the things that are hardest to change later on in the software life cycle. Therefore, the architecture should be based upon something that changes as rarely as possible. Additionally, to be able to evolve together with the customer’s business as frictionless as possible, the architecture should resemble the business structure. This concept is known as software-to-business alignment and may be the closest thing to a silver bullet there is within software architecture. If we manage to create an architecture that mimics some entity in the customer’s business structure that rarely changes we have the best possible opportunity to create a high quality, long-term sustainable software solution. Such a solution can smoothly evolve together with the customer’s business and will allow the customer to be quick, innovative, proactive and agile in their business development.

This all boils down to a final, grand question. What golden entity in the customer’s business should we use as the foundation for the software architecture? Let’s focus on the who, the how and the what, which are different perspectives on business structure. The who is about which people perform important business activities. In other words it’s about organization structure. Basing the high-level architecture on organization structure would absolutely be an option. It wouldn’t be the best option since organizational structure tends to change quite often. As an illustrative example I must share a quote which is often mentioned by Sten Sundblad, one of my sources of inspiration. Sten has been a top-level software architect since way before I even knew what software was. The quote originates from a statement that arose in a dialog Sten had with the chief architect for one of the Swedish authorities. Translated to English it reads:

The current organizational structure is the temporary result of the most recent struggle for power.

This might have been said in a playful tone but thinking twice about it, it definitely holds a fair amount of truth. Stay in the same company for a few years and the chances of you experiencing one or maybe several organizational changes, often related to a new CEO or some other manager entering the company, are quite high. Thus, organizational structure may not be the best candidate for an architectural foundation.

The how business perspective is about flows and methods for performing business activities and how these activities are tied together to form the business. This is called business processes and since they are way more stable than organization structure they form a better basis for software architecture. Although, business processes do actually change every now and then, particularly when the business takes measures to be more effective. As an illustrative example let’s consider the optician industries. A decade ago or so we still went to a physical optician store to buy glasses or lenses. Today you may rather send your optical receipt to an e-commerce merchant to get your vision aids delivered to your home for a fraction of the price compared to a traditional optician. This development has off-course forced optician companies to change their business processes. Thus, business processes are a fairly good option but maybe there’s an even better alternative.

The third and most interesting alternative for the architecture foundation is the what business perspective. This is about what the business needs to be able to do to fulfill its needs, goals and to stay relevant and competitive. The big advantage with this perspective is that what the business has to be able to do seldom changes. If it actually changes it’s almost always because the business must be able to do a completely new thing or that it no longer needs to be able to do something that it used to do. From an architectural perspective, adding new things or removing old things is much easier than changing already existing things. In business terms the what is called business capabilities, i.e. capabilities that the business must possess. Because of its resilience to change, business capabilities is by far the best option for building a stable architecture with a great software-to-business alignment. Pay attention, because I can’t emphasize this enough! If you succeed in creating a software structure based on business capabilities, chances are great that you have created a highly flexible, extendable and maintainable solution that will be able to live for a long, long time. Congratulations! You have just taken a huge leap towards greatness! 

Changing our ways is never easy and building software that aligns with business structure might seem overwhelmingly difficult. Sure, it’s kind of a new mindset and it will probably require some tries before getting it right. The thing is that it doesn’t have to be perfect, not at all. Trying and doing our best will get us a long way and is way better than doing nothing. Above all, by designing a well-thought out architecture, in whatever form it may be, we avoid the misery of accidental architecture with all its catastrophic consequences. 

 

Author

Markus Bergendahl

Solutions Architect

Star Republic