Collaboration Between Architecture & Project Methodologies

Milan Dabic (P.Eng.), Senior Systems Engineer & Software Architect

May 28, 2018

As architects – whether it is Enterprise, Solution, Domain, Process, Data, Application, Technology, or Software – we work in a jumble of different project and architectural methodologies. In this article, I would like to discuss the challenges and impacts of those different methodologies; both on the architect, on projects, and on the organization.

In the project domain, there are a few dominant methodologies. There is the “Waterfall Methodology”, there is “Agile”, there is “DevOps”, there is “Continuous Delivery” and, more in pure software development, there is “Rapid Application Development.”

The Waterfall Methodology is one that is familiar to every IT professional. The model gets its name from its resemblance to a cascading waterfall. The project team starts with requirements gathering and analysis. Next a design is developed. That is followed by an implementation of the design, and that is subsequently followed by verification that is meant to prove that the requirements have been successfully met. Finally, the product/solution enters long-term maintenance.

Agile methodologies are often presented as an alternative to the Waterfall Methodology. Originally, Agile was presented as a software development methodology, but it has grown to include all projects within IT. Unlike the linear nature of the Waterfall Methodology, the Agile methodology is a loop. The main attraction of Agile methodologies is to produce a working solution early on. This methodology requires that users and developers (in which I include all technical staff) work together over the duration of a project to refine, expand, and evolve the solution until needs are met. It is not until the end of the project that the original problem is fully understood.

DevOps is a methodology that arose from the successes of Agile methodologies. In fact, it was at the 2008 Agile conference in Toronto where the concept that would become DevOps was first presented. The methodology of DevOps is broader than a single project. It represents a combination of development and operations. If the Agile methodology is a loop, DevOps is two loops combined. In this methodology, the “development”, “operations”, and “user” staff work together to reduce the time between committing a high quality change to a system and the change being placed into normal production.

Continuous Delivery and DevOps are similar and have common goals, and both are inspired by Agile methods. However, Continuous Delivery is a software engineering methodology that aims to deliver software to production with great frequency, while DevOps factors in organizational structure.

Rapid Application Development is the last of the common methodologies. It is a purely software development methodology. This methodology was developed in the 1970’s and 1980’s as an alternative to pure Waterfall. It can be considered a predecessor of Agile methods.

Well, there you have it. An admittedly brief description of the project methodologies we as architects take part in.

As architects, we have our own methodologies or disciplines if you prefer. We are Enterprise, Solution and Software architects, among others of course. These just happen to be the architecture roles that I play in. Regardless of the architecture role, we provide architecture guidance on a project and are responsible for architecture deliverables (see Architectural Deliverables) for an example set) supporting a solution.

Furthermore, there are project methodologies and there are architectural methodologies (see for example TOGAF). How do they align and how do they conflict?

Sometimes, as architects, we are in a position to influence the selection of a methodology. We may be a part of a process that analyses the forces that drive the selection of a project methodology. We hope to share an opinion that bears on the flexibility of the team and the availability of resources – be they time, people or money, as well as the commitment of the user community. Unfortunately, we don’t always get that chance. Sometimes we’re working in a Waterfall or Agile or DevOps or Continuous Delivery organization.

At some point we’re in an organization with a methodology and you think, “I’ve got it and I’m ready to go”. But are you? Are you able to produce your best work and drive the success of your project, team, and organization?

At this point I’m wondering, “Where did all of these methodologies come from and what spurred their creation?”

If you Google “waterfall methodology origins”, you’ll quickly discover that the earliest mention of the phases described in it were made by Herbert D. Benington at a symposium in 1956.

Did you know it had been around so long? We have been working and some continue to work with a methodology that was developed in the post World-War II years. That was an era of top-down organization planning and of multi-year and even multi-decade strategic planning. It’s not so surprising that the Waterfall Methodology incorporates this cultural norm.

However, by the 1980s or earlier, culture (including corporate culture) was changing. Things were becoming unpredictable, and by the 1990s, new skills and new products were needed and quick! Agile methods appeared as a response to this changing culture and business landscape. Is it any surprise that development methodologies that don’t try to predict the final outcome arose by that time?

So, if we were in an unpredictable and rapidly changing world, all’s well because we have Agile methodologies. Not so fast. Agile methodologies rely on Agile methodologies to save them from the problems caused by Agile methodologies. Quite a mouthful isn’t it? It even makes me dizzy saying it. In a nutshell, any incremental methodology assumes that its flexibility will allow it to always reach an optimal conclusion. However, what if that isn’t the case? What if relying on an emergent architecture doesn’t lead to success, but to a dead end?

Let’s ask ourselves another question: is traditional architecture a product of a culture that doesn’t exist anymore? The answer I’m going to give is a qualified maybe. However, what is certain is that times, cultures, and organizations have changed.

Even human resources organizations are changing to embrace the thinking behind Agile methods. If HR is changing, is it a surprise to expect that architecture practices should change too? This is a question The Open Group is asking too (see Agile Enterprise Architecture Methodology).

In an earlier article, I described the architect as an “agent of change” (see The Psychology of Change: An Architect’s Perspective) and I am going to use that metaphor again: architects must play a crucial role in projects and organizations as the advance guard that works to create an environment where agile methods can succeed.

Our work cannot become a relic of the Waterfall era. If we are to remain relevant, we must abandon the idea that an architecture, of any kind, can be a fully-formed static structure. We must be “agents of change” and create dynamic artefacts and processes that are not only a part of a project (and organization), but allow and promote other members of the team to be agile today and in the long run.