Software vs The Evil Forces of Entropy
Requirements evolve, technology evolves, processes evolve, and users evolve. So why is it that the systems, applications, and their underlying architectures seemingly get built in a way that after a couple iterations of extension, they become brittle and saddled with “technical debt”? More importantly, is there anything that can be done about this, i.e. is there a such thing as “agile architecture”?
To achieve a future proof technology capability, one needs to overcome entropy. Agile architectures are a result of continuous and conscious revision and updating as needed to keep pace with the demands of creating strategic and competitive advantage while remaining maintainable and pliable.
Every several years or so, there is some new architectural pattern that is supposed to save us all from tightly coupled systems, non-extensibility, compounded technical debt, and that will provide a “future proof” architecture. And yet, as time goes on, these systems become more costly to maintain, extend, update, and serve what the business needs are now- just as their predecessors did. These systems get so brittle that making even rather minor changes takes longer, is more expensive, and has unintended ripple effects such that the patience and budgets of the stakeholders are challenged. Eventually it gets to the point where making a material change costs more than outright replacing the system.
The answer often proffered, “modernize”. Adopt the new architectural pattern and reinvest in the current or new set of vendors that sell infrastructure and technology that embodies the new way. Clearly, the world evolves and there comes a time to sunset older things (i.e.: mainframes running scheduled batch processes); and just as clearly, there are poorly architected systems based on relatively modern technologies that never stood a chance of evolvability – that cost their owners as much in frustration as in dollars to keep pace with the business. Also contributing are the laundry list of short-term (or short-sighted) decisions that get made in the interest of getting to done quicker, that actively compromise the maintainability and extensibility of software systems.
Yes, there is a time to modernize, and yes, there are a plethora of systems out there now that investing in a full-blown modernization is a better choice than investing in overcoming the technical debt of the existing system. But once that is done, or if you have a system that is not yet at that point, it’s time to take a hard look at how you invest in your software system and what you can realistically expect from it over time as a result of that investment. Specifically, now is the time to understand how to keep from getting here again (and again, and again).
The reality is that software systems, and their underlying architecture are just as subject to entropy as any physical system in the known Universe. This means that it requires a constant investment of energy to maintain a higher level of order in the system and to prevent the tendency toward disorder. As the system is maintained and evolves over time, one cannot expect to make ad-hoc changes or even a steady-state investment into the system and expect it to remain pliable and responsive to changing needs. You cant expect to win over the forces of entropy without the continuous investment of order creating patterns and organizational energy.
Entropy is an enduring force, that can and will manifest itself in a myriad of ways. From the small: an accumulation of source level warnings here, an unexpected interaction there – to the large: tightly coupled interfaces, hard-coded business rules, repurposing data fields; the mechanisms of entropy are insidious and obstinate. That bolted on feature that seemed like such a great idea at the time will come back to haunt the system in short order. The decision to stay on version N of the (language, library, service, etc…) that seemingly saved a month to market and some thousands of dollars in cost? The outcome of that is decaying even as you read this, the costs and time required to remediate that older version are growing, the number of dependencies involved are growing, the ability to maintain the system with that now out of date dependency is dwindling (and may be non-existent). Entropy is winning, and your software architecture is becoming unstable, brittle, and perhaps even unsupportable.
The only guard to keep your software architecture agile is to understand that it requires continuous investment to overcome the natural process of entropy. This investment takes many forms: micro-modernization, top-down governance on architectural and technology best practices coupled with bottom-up empowerment to do things the right way, iterative refactoring, upgrading skills in the resources, willingness to replace sub-systems which are overcome by events (i.e.: batch for streaming, that upgrade from Java 5- really, it’s time), re-architecting for changes in the business landscape instead of force-fitting the new into the old. This list is by no means exhaustive, but illustrates the breadth of concerns that need to be actively guarded against decay.
In our lifetime so far, there is no silver bullet architectural pattern, agile methodology, or off-the-shelf technology that can magically provide a future-proof, agile platform. I dare say that in our life-time, there won’t be one. Agility connotes nimbleness, an ability to change, and change is a process. Accomplishing an agile architecture is a deliberate, ongoing process, its more of a journey than a destination.
To onboard an agile architecture skill-set and cultural pattern into your product or technology organization, contact Ten Mile Square Technologies at firstname.lastname@example.org. Our team of senior architects and organizational team member level knowledge transfer consultants can rapidly accelerate your team’s success.
Categories: Architecture, Blog, Enterprise, Modernization, Software Development, Technology Strategy
Tags: Agile, Architecture, technology