Compound interest of alignment

If software solidifies organisational structure, then what happens if you re-structure?

Brooks in his “The Mythical Man-month” cited Conway’s seminal paper on the relationship between software and organisational structures – the basic idea is phrased as:

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.

In other words, software solidifies organisational processes and structures.

What happens when you re-organise? And re-organise again? And add more software. And re-organise?

 

The answer is obvious

My “old” research supervisor once told me (slightly paraphrased): “Once you know the right question, then the answer is obvious”. Of course, the real trick is to find the questions that you can answer with very limited resources – without limiting yourself to what you already know; or think you know.

And therein lies the all to common trap – focus only what you know and you’ll ask the wrong questions leading to poor conclusions and rework. The goal of questioning must also be clear.

This is also why issue based consulting methodologies like “The Pyramid Principle” by Barbara Minto become problematic. Under the usual cost and schedule pressures, progress and deliverables are easier to reach through clarifying questions – they get a tick for successful engagement; but miss the bigger, more important questions. They fail their clients by not acting as a change facilitator. As one of my clients once said: “Request a consultant to tell you the time, they’ll grab your wrist and look at your watch”.

References:
[1] Tom Pohlmann & Neethi Mary Thomas, “Relearning the Art of Asking Questions”, https://hbr.org/2015/03/relearning-the-art-of-asking-questions
[2] Barbara Minto, “The Minto Pyramid Principle Concept”, http://www.barbaraminto.com/concept.html

To-be or not to-be; as-is the question

I did a quick survey of the software architecture literature on my book shelf (physical and virtual) this rainy Easter weekend for guidance on how to model a “transition” or “interim” architecture. As there was almost nothing on the topic, the next stop was Google – only to re-affirm my growing suspicion that book authors perceived “software architecture” as something that could only exist before (as-is) and after a project (to-be).As-is and to-be

My second suspicion was that they either ignored or didn’t know about the idea of multiple releases (seemed ridiculous). If they did know then did they view those releases as either not having an architecture (seemed even more ridiculous) or just as multiple iterations of the above diagram (more reasonable).

Reality

However, I think the second diagram looks more like reality. We start with a current (as-is) architecture. We then seek to change it through a number of project releases only to find by release 2 that we need a release 2.1 which splits from release 3. Release 4 is subsequently cancelled, but the political environment demands release 5. Meanwhile the old “as-is” architecture still lingers as a ghost. End result, we never got to the defined “to-be” (utopian) architecture, because the delivered “to-be” architecture is the sum of release 2.1, 5 and the old “as-is”.

This then begs the question: did the “as-is” really exist (as documented), because  our “to-be” is after all the next project’s “as-is”? The most obvious answer is that our beloved architecture never actually is in a known “as-is state” and only exists in a permanent “interim state” haphazardly moving towards an ever-changing target architecture.

And sadly based on my limited literature review, our modelling tools aren’t really designed to support this?

Note: I did find one reference provided by Avolution. They have defined five stages of architecture: 1) Historical, 2) Current, 3) Soon-to-be, 4) Transition or maybe, and 5) target (to-be). But they want you to buy their tool before saying how you’d model the relationships between each of five.

Everyone is (not) an architect!

It continue to amaze me how willingly people offer their advice on software architecture. Most people would struggle to understand the inner-workings of email, but that doesn’t seem to stop them. And if they do have some experience with the topic – regardless of how long ago – some people think they are experts.

I’d never give advice on corporate accounting; nor how to manage your finances or investments – that’d be illegal without the right training and certification (just ask CBA).

By all means ask questions, get your architect to explain their rationale, and hold them accountable. But don’t do their job for them.

Simplicity, Design Elegance and Architecture

Chris Aitken‘s article “Simplicity is the ultimate sophistication” had me intrigued and well worth a read. If nothing else to acquaint yourself with the described architectural principles and the idea of ‘principle assertions‘ – statements that are either ‘true’ or ‘false’ depending on whether the principle is evident or not in a given architecture specification.

wpid-notes-6.png.png

  • The Principle of the Separation of Concerns: A design should be comprised of a set of independent components each of which addresses discrete functionality requirements for each defined concern.
  • The Principle of Optimisation: A design should provide a complete solution without necessary duplication.
  • The Principle of Coherency: All design components should be harmonised.

The architecture design process implicitly defined by the article is clever in its simplicity (no pun intended):

  1. Separate the solution concern into two (or more) logical concerns
  2. “Is the concern grouping and separation coherent?” Test each answer using the principle assertions
  3. “Is the concern grouping and separation optimal?” Test each answer using the principle assertions
  4. To go 1 if answers to 2 or 3 produce false assertions.

Note: concerns are relative to your stakeholders who may be concerned with requirements, systems, models, operational processes, code quality etc.

The main criticism of the article is the exchanging use of the terms “good design”, “simplicity” and “elegant design”. Is design simplicity really the same as design elegance? And how is this different to a good design?

I had a look in the dictionary for guidance.

Good: to be desired or approved of; having the required qualities; of a high standard.

A “good design” infers a design that meets all concerns for all stakeholders.

Simplicity: the quality or condition of being easy to understand or do; plain or uncomplicated in form or design.

“Design simplicity” infers a design easy to understand by its stakeholder.

Elegant: graceful and stylish in appearance or manner; pleasingly ingenious and simple.

An “elegant design” infers a design that causes the architect to glow of pride, as it is realised to be both good and simple to all relevant stakeholders.

Beauty is in the eye of the beholder, which is why an elegant design is so difficult to achieve. But don’t let that stop you.

Bob, iPhones and software vendors

You meet Bob at the local coffee shop.

You don’t really know Bob, but one of your acquaintances suggested Bob after you casually mentioned your need for a new mobile phone.

You checked out Bob’s website and you are fairly sure that Bob didn’t write any of the available material himself.

At the coffee shop, you both introduce yourselves. You both order coffee (you pay for both) and after a few pleasantries, Bob starts to ask a few questions about your potential mobile phone purchase.

“So, I understand that you need a new mobile phone?”

“Yes, potentially…”

“And you’d like to make phone calls?”

“That would be handy…”

“And send messages?”

“Also pretty handy…”

“How many?”

“How many? Eh, not sure… Maybe 50″

Bob shifts chair position.

“In that case, my recommendation is the new iPhone. Most of your colleagues use an iPhone, and it’s the leading messaging phone”. Bob looks confident.

You both get up and walk out. After thanking Bob, you walk down the street to the nearest mobile phone reseller and immediately purchase the most expensive iPhone. Although you are slightly concerned about how Bob knows your colleagues, you do want to be better than them.

You never see Bob again.

Epilogue

If this scenario sounds slightly crazy, then you’d be correct. And of course, this isn’t the way you’d choose a software product.

Names have been changed to protect the innocent.

Conway’s Law and Technical Debt

Everyone wants to be agile. It is the catch cry of the day, along with cloud computing, big data and payments processing.

Agile software development boils down to do more with less, and communicate more with less. Do more with less is about using fewer but better suited tools to improve productivity. Communicate more with less is about increasing the design discussion and feedback loops without the overhead of maintaining large documents – note the difference between “discussion” and “documenting what we built”, something many projects confuse.

But I’d postulate that all software development methodologies still fundamentally view software development as a single team activity; despite our typical work environment consisting of multiple teams; multiple delivery structures.

Yes, every project manager has the list of external dependencies on other teams and other systems, but it is just a milestone tracking list. It doesn’t help us become more agile. Similarly, the Scaled Agile Framework is more about making the project management office or the enterprise release management teams more agile than addressing the multiple team challenge.

So what’s the problem?

The below diagram illustrates three projects tasks to deliver three solutions. The three teams must leverage existing corporate assets (e.g., an inventory system or general ledger system) to avoid causing trouble, but the trade-off is overlaps between the three solution spaces. Secondly, the trade-off agreement between team “A” and team “B” will in all likelihood impact the possible trade-off agreement between team “B” and team “C”.

Solution overlaps

Conway discussed this problem back in 1968 and framed it as the correlation between organisational structures and software structures. Bass and others have found similar evidence of projects getting into trouble due to “architectural misalignment”; that is the difference between the organisational enabled communication structure versus the communication structure needed to solve complex architectural problems. This “structures correlation” problem is commonly referred to as Conway’s Law, although “structure” does need to be understood more in context of Bass’ design problem discussion than Conway’s original and more literal computer code structure context.

While we probably don’t have a good answer for dealing with Conway’s Law, it struck me that Martin Fowler’s Technical Debt Quadrant is useful for illustrating the four phases of (what I have coined) the Conway’s Law Denial:

  1. Your company’s methodology calls for identification of project interdependencies; dutifully noted as part of the initial planning efforts.
  2. Early design work uncovers overlaps with other projects (hopefully on your list). But we are all polite and work together, as schedules haven’t yet been tightened (too much); aka we send a few emails with draft designs attached.
  3. The true complexity of the “structures correlation” problem arrives and full-blown denials starts
  4. Final stage of full-blown denial

Projects and TD

And why does all of this matter; other than stop annoying colleagues?

Teams disconnected by organisation, location and/or time cannot make appropriate trade-offs and the scene is ripe for piling on the technical debt.

References

Mel Conway, “How do committees invent?“, 1968

Bass, M. et al, “Architectural Misalignment: An experience report“, 2006

Kwan et al., “Conway’s Law Revisited: The Evidence For a Task-based Perspective“, 2012

If “Buy Before Build” is the answer then what was the question?

If you made a list of the “Top Five All Time Favourite” principles, then I’m sure “Buy Before Build” would be on the list. It just seems like one of those obvious statements. Why wouldn’t you buy of the shelf – proven – software to reduce delivery risk, outsource (non-core) software development, and gain incremental improvements through upgrades.

Buy before build

Buy before build

That’s all very good in theory, but it only works well if the off-the-shelf software provides a good functional fit in terms of business processes, data and organisation, as well as a good technical fit in terms of non-functional requirements and existing technology environment.

The Buy-Before-Build Anti-pattern

This anti-pattern exists when an organisation deploys off-the-shelf software with a poor fit. The result is a pile of custom developed code to change the off-the-shelf software (as illustrated below).

Buy before build - the reality

Buy before build – the reality?

Why would this happen? A poor fit exists when:

  • The (assumed) business processes built into the off-the-shelf software requires significant business process re-engineering. A system implementation project can accommodate smaller changes, but larger changes are unlikely without a dedicated effort. And that’s of course assuming it is in the organisation’s interest to align business processes. Off-the-shelf implies commoditisation, and no one would like to commodities their business advantage.
  • Integration with existing IT systems will be difficult and expensive, where the off-the-shelf software implicitly assumes itself to be the data and/or process master. Even if the new system has a perfect functional fit, the custom code required to ensure good synchronisation between two masters is the perfect recipe for cost blow out.
  • Operation management of an off-the-shelf software system designed for business hours only operation may prove impossible to do within an 24 by 7 environment. Gaps are likely to occur due to long-ish maintenance windows, lack of true redundancy and insufficient configuration management.

Buy-Before-Build is the answer, if the off-the-shelf software provides a “good enough” functional, operational and technical fit – and if future upgrades remain feasible and without too much complexity. If it isn’t then the organisation might have bigger problems to tackle (technical debt?) or it’s trying to standardise its competitive advantage – In those cases, build-before-buy might be the better option (at least in the short-term).