The dark secrets of enterprise architecture

Want to decrease IT costs without increasing technical debt? Enterprise architecture is a good place to start. Want to improve business/IT integration? Enterprise architecture promises to be the tool of choice. How about establishing a more streamlined and effective business? One guess as to where you should look first.

And therein lies a paradox: Improving architecture can drive significant improvements, but the frameworks and methodologies that are supposed to help you improve your company’s architecture rarely live up to their promises. Instead they turn the EA function into an ivory-tower white-paper factory that emphasizes deep, abstract conceptualizing over practical action.

After 30 years of trying to make this work, why do so many attempts at EA fail? The reasons are dark secrets few are willing to acknowledge. How dark? Being metrics minded, I’ll use Ansel Adams’s zone system which ranges from 0 (black) to XI (white).

A word of warning: The darkest secrets come last.

Secret No 1: EA has no metric for success

Zone System rating (ZSR): IV

TOGAF is the best known and most widely used approach to enterprise architecture, so we’ll use it as our stalking horse. In case you aren’t familiar with it, TOGAF stands for The Open Group Architecture Framework. According to the Open Group, “TOGAF®, an Open Group Standard, is a proven enterprise architecture methodology and framework used by the world’s leading organizations to improve business efficiency.”

Which leads to a question: In what way is TOGAF proven? I Googled “TOGAF SUCCESS RATE” and came up dry. So far as I can tell neither the Open Group nor anyone else has even defined a TOGAF success metric, let alone tracked improvement against a baseline.

Secret No. 2: EA pursues the wrong target

ZSR: III

TOGAF claims improved business efficiency as its benefit. But as anyone with an ounce of metrics sophistication knows, “Efficient” is always a ratio — units of one thing per units of some other thing, like miles per gallon, transactions processed per server dollar, or story points per programmer hour.

“Efficient” is meaningless without knowing both the numerator and denominator, and TOGAF provides neither.

This is no mere semantic quibble. Efficiency matters when you’re in a volume business and tomorrow’s markets will look just like yesterday’s markets.

Most business leaders understand that change is the only constant they have, which means flexibility and adaptability matter much more than efficiency. TOGAF’s waterfall nature (the next dark secret) make it a poor choice for achieving flexibility and adaptability.

Secret No. 3: EA is waterfall revisited

ZSR: II

One reason EA frameworks and methodologies fail so often is that they are, in the end, waterfall in nature. They document the current state (a time-consuming endeavor) design the ideal future state (another time-consuming endeavor) and plot a roadmap for closing the gap between the two.

Then, when the world changes, re-plotting everything is just as time consuming. In practice, it means keeping EA involved in business change doesn’t speed things up.

It slows them down.

Secret No. 4: While important, EA isn’t urgent

ZSR: II

Imagine you’re an enterprise architect. Current state, future state, gap, and roadmap in hand — along with estimates of the money to be saved once the company achieves its ideal future state — you meet with the executive leadership team (ELT for the acronym minded) to get funding.

Good luck with that. Just before your meeting, and again after it, business unit leaders also met with the ELT, seeking funding for their pet projects. The ELT compares your request to the other opportunities in front of it. EA’s benefits arrive in an indefinite future. That’s because they’re realized in other, business-defined projects that can’t take advantage of the improved architecture until it’s been implemented, and won’t deliver their business benefits until they complete.

Or, the business projects can launch right now, and the architecture will just have to wait.

Guess what gets funded — especially when the ELT can easily understand the value of, say, improved CRM but have no idea what you’re talking about when you natter on about architecture development frameworks, boundaryless information flows, the enterprise continuum, and the other hundred or so specialized terms and acronyms EA practitioners add to their vocabularies so they can join the Cool Kids Club.

Secret No. 5: EA frameworks can’t describe real-world architectures

ZSR: 0

TOGAF’s foundation contains a fundamental flaw: It describes architecture in terms of a fixed set of four layers: the business layer, application layer, data layer, and technology layer. This has always been an oversimplification — each of these layers has segments and sub-layers.

This is leads to the two biggest and most important dark secrets:

Secret No. 5.1: Platforms are applications — applications are platforms

ZSR: 0

TOGAF’s layered model is just plain wrong. It’s wrong because more and more, platforms are also applications and applications have become platforms. Take SharePoint. You point your browser to it and you’re managing files in a more sophisticated way than you could with shared folders, and, if you want, you can create blogs, wikis, and other interesting stuff.

You can also use SharePoint as a platform for developing general-purpose applications, complete with data-entry screens, workflows, reports, and such. It’s a platform and an application.

Don’t like that example? How about SAP? It and its fellow ERP systems are applications — very big applications. And they’re built to let you define your own data elements and program your own workflows and transactions using them, without violating their structural integrity in the slightest.

They’re platforms as well as applications.

Think this is a minor point? The entire purpose of enterprise architecture frameworks is that they’re supposed to let you accurately and consistently describe architectures. If they can’t do that — if they can’t describe how SharePoint and your ERP suite fit together with everything else you’re running or will want to run — then what value do they provide?

Secret No. 5.2: The missing layer

ZSR: 0

Platforms are applications. Applications are platforms. Modern IT doesn’t just implement and run them. Modern IT integrates them.

Most organizations integrate them with a messy collection of custom-programmed batch interfaces, which go by such names as “spiderweb,” “spaghetti,” and “hairball” depending just how messy they are.

TOGAF has no place for describing hairballs, or for more architecturally sound alternatives like enterprise service buses (ESBs). Which is unfortunate. It’s unfortunate because cleaning up a hairball integration architecture is often the single biggest benefit to be had from improving architecture.

And it’s unfortunate because increasingly, IT doesn’t build applications using just one underlying application as a platform. IT uses the ESB to create a virtual “source of truth” service out of a collection of “systems of record.”

It builds applications out of these services rather than making direct use of application APIs, but won’t be able to use TOGAF to describe them.

EA’s bright secret: Don’t bring me problems. Bring me solutions!

ZSR: 9

Depressed? Don’t be. Just because the most prevalent architecture frameworks and methodologies are a mess, that doesn’t mean you have to live with bad architecture. Quite the opposite. Here are five quick tips.

Perfect is the enemy of good

Okay, it isn’t original. It wasn’t even original when Voltaire took credit for it. You’ll note that Bright Secret #1 promises a ZSR of IX, not XI. The rule here: Don’t even try to describe the perfect future state. Be happy with better and make it happen.

Ask anyone in IT what the top candidates should be (probably, your hairball). They know the answer, and they’ll be delighted to share it. You can get started improving your architecture without ever once documenting your current state or describing your future state in any level of detail.

Think agile

Agile is more than a family of application development methodologies. It’s also a way of thinking. Of particular relevance here are the closely related ideas of iteration and incrementalism. So as you’re setting a target of ZSR IX, figure out small steps you can take right now that will leave the architecture better, not the complete set of steps that will get you all the way there.

Don’t compete with business projects — integrate architecture into them

Those small steps you can take right now to improve your architecture? Instead of competing with business projects for funding, build architectural improvement into the business projects that would get the funding anyway. This way, every IT-related project leaves the architecture in better shape than it found it, with only a modest increase in investment and scope.

And as project sponsors are unlikely to be willing to spend their own budget on better enterprise architecture, you should present to the ELT. Only now you’re asking for an architecture subsidy provided to every approved project, not separate funding.

Use design principles to define what “better architecture” means

Give up on the idea of a complete reference architecture and you’re freed up to figure out what “better architecture” means. Usually it means conforming to a short list of core principles. The hard part isn’t formulating them. The hard part is to not pretend.

For example, just about everyone would agree that eliminating redundancy — normalization — is the hallmark of a good data architecture. Make this a design principle, though, and you’ll just be pretending, because if you’re like most enterprises, IT will buy when it can and build only when it has to. This is, in fact, one of your likely design principles.

And if you buy when you can and build when you have to you won’t be able to avoid data redundancy. That’s fine. Don’t pretend. In a multi-vendor environment the corresponding principle is to document and synchronize redundancy.

Avoid architects

Okay, that’s a bit strong. What you really want to avoid is an EA function with permanent staffing. Do this and you’ll have a hard time keeping it from becoming an ivory-tower white-paper factory. Making it a rotational assignment instead will keep it grounded, because everyone writing and enforcing the rules while assigned to it will have to live with the consequences when they rotate back out again.

In conclusion

Architecture matters. Especially, technical architecture matters. Businesses with clean architectures are demonstrably more nimble and effective than those that lack them.

Architecture matters too much, in fact, to entrust it with the current crop of frameworks and methodologies. It’s too bad, but right now, not having a methodology to fall back on isn’t a problem.

It’s a liberation.