Agile’s dark secret? IT has little need for the usual methodologies

Agile is supposed to be yesterday’s news. Advanced companies? They’ve jumped to DevOps. Except, a lot of application teams not only haven’t adopted agile techniques, they don’t even see how they could.

You might find this surprising — until you take a closer look at what these teams need to do: There’s very little overlap between their objectives and what agile methodologies deliver.

Agile methodologies, especially Scrum, which many IT managers think is synonymous with agile, are best at what IT does least: develop new applications.

With the exception of the company’s website and mobile apps, one of IT’s core principles is “buy when you can, build when you have to.” IT licenses something like 90 percent of all new functionality in the form of commercial off-the-shelf software (COTS) and software as a service (SaaS), leaving 10 percent for software developed in-house.

Scrum, Kanban, Lean Software Development and most other agile methodologies are designed for the 10 percent, not the 90 percent. On top of which, in a typical shop, about 70 percent of developer hours goes to maintaining and enhancing applications already in production, leaving 30 percent for implementing new ones.

Do the math. Agile is mostly useful for 10 percent of the 30 percent — in other words, it handles a whopping 3 percent of what IT’s application teams are called on to do.

Your mileage may vary. Plug in your own numbers. The result will still be small, but it doesn’t have to be.

Maintenance and enhancements the agile way

Let’s take the easy one first: The maintenance and enhancement requests that constitute so much of what IT does on a day-to-day basis.

You don’t even have to squint at the maintenance and enhancement queue to see how much it looks like an agile backlog. Add a product owner, start phrasing requests as user stories (we’ll get to this in a minute). Apply Kanban: The product owner sets the priorities, and whenever a developer finishes a user story, he or she will pick the next one off the backlog to work on. It’s simple, imposes very little overhead, and gets the job done.

Agile COTS

Applying agile methodologies to COTS implementations isn’t quite as simple as applying them to maintenance and enhancements. But then, you wouldn’t expect it to be: Implementing big systems isn’t as simple as handling maintenance and enhancements, period.

Related video: How the agile methodology really works 

But you can apply agile techniques to COTS, too. You just have to recognize two crucial points.

First: Implementing COTS, whether on-premises or through software as a service (SaaS), isn’t just like developing software from scratch, with maybe a tweak or two needed. It’s different in profound ways that make Scrum and Kanban severely unsuitable for the job.

The second is something we’ve already covered, only more so: There are more agile methodologies than Scrum and Kanban.

App dev vs COTS: It starts with the data

When IT teams develop software, they need to understand what “the business” needs the software to do. Once they’ve reached this understanding they don’t start by developing software that does it. They start by designing data structures. The application comes later.

Except, that is, when they don’t have to design data structures because an existing database already handles what the new application needs.

Buy-when-you-can/build-when-you-have-to is profoundly different.

When IT implements multiple COTS/SaaS packages, existing databases make the job harder, not easier. Each package comes with its own database, and when the software vendors design their databases they don’t take the ones you already have into account because they can’t. They also don’t take the ones other vendors have already designed into account because why would they?

So when IT implements a COTS package it has to track down all the existing databases that store the same data the new application manages too — not to take advantage of it, but to figure out how to keep the overlapping data synchronized.

When it comes to managing data, internal development and package implementations are completely different.

Describing what the software is supposed to do

With the data in hand, application teams have to describe what the business wants the software to do. Most agile variants prescribe the use of “user stories” for this purpose — statements that take the form, “As a I want so that .” For example, “As a telemarketer, I want to store information about customer households so I only call people who might possibly want to buy the next product I’m selling.”

Okay, the example is obviously fiction, but don’t worry about that. Worry about this: If you’re implementing a CRM system you’ll waste everyone’s time and annoy the living daylights out of business managers if you insist on this sort of thing. If you’re implementing a CRM package that doesn’t store information about all forms of “customer” you aren’t implementing a CRM package.

When you implement COTS or SaaS software, the usual argument is whether to implement it “plain vanilla” or to tailor it to support the company’s business processes and practices. These are supremely silly alternatives to argue about.

COTS software comes with overt or implied versions of business processes that in many cases are superior to the ones currently used in the business areas that will use it. In many other cases it isn’t.

Those who argue for customization (because IT has to make its “internal customers” happy) are lobbying for — let’s see now — spending a lot of money implementing a shiny new piece of software so you can go about your business exactly as you’re used to.

Big cost, no business improvement, by design. What?

So plain vanilla it is, because there’s no chance at all a business process is different, not because it happens to be, but because it has to be.

Sometimes — actually, by definition — a company is the best in the business at something or other. Force plain vanilla and IT is insisting on business deterioration as a strategy.

If IT and the business it supports are going to have an argument, plain vanilla vs. customization is a bad argument to have. You can skip it entirely.

Two-stage COTS

When a business (not IT) implements COTS-enabled intentional business change there are two stages to the effort: Making the software competent, and optimizing the business.

You can handle the first stage by using a little-known agile variant called the Conference Room Pilot (CRP). How it works:

First, IT loads master data — customer data, product data, and so on — into the new package’s database. Then the project manager reserves a large conference room loaded with lots of white boards, large monitors, and other cool stuff. Next comes a large stack of the business transactions the new package will have to handle in order to be competent at the business.

These business transactions might be the result of a carefully crafted test plan. Or, they might just be whatever was processed last month or the month before. Either approach works fine.

Then, you bring in several business users and some application gurus, lock the door, and occasionally throw in a few pizzas and assorted beverages.

What happens behind the locked door? A business user picks up a transaction, tries to make it work with the software as it is, and explains to the gurus, “I can’t process this transaction because …” The developer fixes the “because” and it’s on to the next reason. After a while the becauses become harder to find. As they do, Stage 1 fades out and Stage 2 fades in.

The transition is marked by a change in the conversation. Where in Stage 1 business users explained why they couldn’t process a transaction, the Stage 2 conversations sound like, “I could process this transaction more easily if the software would …”

Viola! The conversations aren’t about making the software competent anymore. They’re about making processes more effective – about business optimization.

Oh, and by the way, your developers aren’t passive order-takers in this process. At any time they’re encouraged to say something like, “Making the software do this would be messy and expensive. How about making it do this other thing instead?”

All this without a single argument about whether plain vanilla or chocolate sprinkles is the right way to go. Arguments over sausage and pepperoni or Canadian bacon? Those are healthy discussions.

Agile’s darkest secret

Agile has a secret darker than it being useless for most of what IT does: Agile development is all about product delivery. Whether it’s Scrum, Kanban, Lean, or what-have you, the project is finished when IT delivers a software product.

But, whether you’re using Scrum to develop software from scratch or a package using CRP, product delivery isn’t what anyone needs. The point had better be intentional business change or there’s no point. The application is a lever the business can use to achieve the desired change.

CRP’s whole second stage is about making the business run differently and better. Intentional business change is baked into it.

Using agile techniques to develop an application, on the other hand, is rife with risk that business change will be Someone Else’s Problem, because everything about Scrum, Kanban, and the rest are carefully crafted to make product delivery the goal. If that weren’t the case we wouldn’t have product owners, and user stories wouldn’t be about what the software is supposed to do.

What’s needed are techniques that start with designs for doing things better, turn those designs into user stories, and adding tasks to the backlog that are all about changing how business gets done.

Or, even better than this, techniques for iteratively and incrementally improving how business areas go about their work, accomplishing those iterative increments through application changes and enhancements tied to the ideas for business improvement.

In conclusion

Okay, as scandals go, Agile’s darkest secrets are pretty tame. Nonetheless, ignoring them can lead to serious wastes of effort, pointless arguments, and project successes that look a lot like failures once someone asks whether the company is better off than it used to be.

Related IT strategy articles: