To get a little more detailed, here’s what I’m thinking of for each component.  We’ll start with the Requirements component.

Requirements Component

 

The Best of Intentions

 

Right now, in a company much like yours, Larry, a business analyst, is spending hours writing up a requirements document.  Because Larry has been yelled at so many times by developers about not providing sufficient requirements, he is being is being particularly anal, documenting every feature, expected inputs and outputs, validation conditions, etc.  Once they are done, the document is beautiful.  It is 87 pages of clearly defined requirement bliss.  The problem is that the document is not done, and very soon it is not going to be alone.

Larry sends the document to his supervisor (1 copy), who makes a few rough edits and sends it back (another copy).  Larry then cleans up these edits to produce a revised final version (yet another copy) and sends it back the supervisor, who says to run with it.  In this initial review, we have now seen 3 copies of the document, each of which was a little different, and will live forever in email histories, hard drives, dark corners of SharePoint, or printed out on someone’s desk.

Larry sends the latest copy out to the development team, and schedules a meeting to discuss.  By the time the meeting comes around, he realizes he needs to make a few more changes, and brings a new copy to the meeting.  Meanwhile some developers have absentmindedly filed a copy off to their hard drive, and will forever reference that copy anytime a requirements issue comes up.  The meeting generates a lot of discussion, which causes several more changes.  The result is several more drafts of the document. 

 

The Hydra Emerges

 

All the while, the supervisor has taken his “final” copy and shown it to the CEO “just to given him a rough idea of where we are at”.  The CEO, despite his best intentions, will remember this conversation as “here’s exactly what we’re committing to”.  He especially likes the Universal Data Widget feature, which the development team has just notified the Larry is not possible.

The next time the Larry meets with the supervisor, they both have different copies of the requirements, and so the supervisor doesn’t notice that the Universal Data Widget feature, which the CEO loved, is now missing from the latest document.  A few weeks later, when someone else points it out, and nobody can remember why it was removed, and the supervisor plays the “what the CEO wants” card, and so it is added back in.  Meanwhile, the development team has already started coding, and has no knowledge of the feature being added back into the requirements.

 

Reality Strikes

 

3 months later, after the testing phase, the CEO asks “where’s the Universal Data Widget?” and everyone panics.  The development team has no recollection of the feature, and Larry can’t remember when it as added back in.  In their own defense, everyone produces a different copy of the requirements, each of which supports their own case.

After much finger pointing, the supervisor says, “whatever, just add the damn Widget right now!”  The developer takes a stand, pointing to his copy of requirements in hand, and insists that he’s not allowed to make any changes without a documented change request.  So the supervisor grabs a yellow sticky note, writes “Add the damn Widget” on it, and sticks it on the developer’s copy, and storms away.

Everyone started out with the best of intentions, and now they are completely frustrated with each other.  Everyone wants the product to work well, and yet they no have a list of things to blame on everyone else.  So much useful communication was lost in verbal meetings and hallway conversations, and the product suffered as a result.  In the end, the developers still say that the analysts don’t produce sufficient requirements, the analysts still say that the developers will never be happy with whatever requirements are produced, and the management still seems them all as a bunch of finger-pointing whiners who can’t get a product finished.  And honestly, I can’t really blame any of them.

 

Are Requirements Counter-Productive?

 

I’ve seen one variation of this or another so many times over the years.  The sticky note part is actually true: on one project where the requirements were spinning out of control, the development lead was became fanatical about documenting change requests, and the PM actually resorted to placing sticky notes on his copy of the requirements. (The development lead was the same as this guy, the original less-embellished story is here).  The end result is that people stop writing requirements, because it just makes things worse.

There has been a push over the years to move away from requirements, usually as some sort of eXtreme Programming (XP) method.  Requirements get replaced with looser “user stories.”  In in some requirements are abandoned all together, in favor of having the customer “always available” to answer questions and fill in missing details. 

Given how hard it is to get specifications right when you’re really trying to be thorough, it’s hard to imagine how non-technical customers can be expected to make rational, thoughtful, and cohesive requirement decisions by the seat of their pants.  It is the customer’s job to want the best and coolest thing, without consideration to the overall design, because that’s what normal people do.  Conversely, it’s the job of the analyst/PM/developer/whoever to adjust those requests to reality and ensure that the customer’s seat-of-the-pants vision remains consistent with their business objective.  Otherwise, if the design is based on what the customer wants at a given time, Frankenstein’s monster arises.  All of the parts are individually beautiful, when when assembled, the sum result is grotesque.

A lot of great information has been provided about the importance of requirements.  Joel Spolsky has written a series of posts about why the are necessary, and Karl Weiger has written the definitive book on how to do it well.  Over the years, I’ve realized that a lot of people are not actually arguing that requirements are helpful.  Instead, they are arguing that they are not practical, because everyone is always changing their minds.

Again, this is not surprising, because when you have complex documents written in Word being email around, it is not practical to maintain.  The problem here is not the content of the documents, but the approach to maintaining them.

 

Is There A Solution?

This always seemed like a difficult problem to solve, but not an overly complicated problem.  I’ve written more systems than I can count with far more complicated workflows than this.  In my opinion, the idea that anyone, much less software developers, are using Word documents in this day and age is inexcusable.

I have been waiting for years for a solution to arise to solve this problem.  Many systems have come along, either bug trackers or planning tools, with a requirements component bolted on to the side, usually as a wiki or a document management tool.  There are also tools that focus primarily on requirements, and then have a project planning or bug tracking component bolted on as an afterthought. 

However, none of the systems I’ve seen systems really captured the evolving nature of requirements and tied them to work items and bugs as a core system function. 

 

So What’s The Solution?

 

The solution is that we do what we’re always paid to do.  We build a system that accurately reflects the business model, in a way that makes it easy for people to do their jobs well.  It will be a centralized, intuitive system that easily facilities and records the natural conversations that take place in the development of requirements.  It will be a like Word’s track changes feature, but it will actually be useful.

Here’s the new story:

Larry enters his requirements into the new system in a outline and line-item format.  Because it is naturally geared towards his specific needs of entering requirements, he is able to leverage features like reusing requirement sections and building mockups, and is done is half the time that it would take him to create the same requirements in Word.

Larry then clicks a button to submit the requirements to his supervisor, which sends a notification to the supervisor.  Instead of clicking on a document, the supervisor clicks on the link to the system, and reviews the requirements.  He asks a few questions and enters a few suggestions, each of which is attached to a specific line item in the system.  Once he’s done, he clicks a button to send the requirements back to Larry.

Larry reviews the feedback from his boss, makes the changes he suggested, enters a few clarifications of his own, and then forwards it back to the supervisor for final review.  The supervisor clicks the Approve button, and the system has tracked every step of the way.

The process proceeds the same one the developers get involved.  Everyone is looking at the the same copy on the “document.”  If anyone sees something they don’t understand, they can click a button to request clarification, and the analyst can respond right there in the system, so anyone else with that same question can see the same answers.  6 months later, when developer and QA person are arguing about how a feature should work, they can look back the sole latest version of requirements, see all the discussion that took place, and quickly get an answer; if the answer is not there, they ask it through he system

 

How Do You Get People To Use It?

 

People will need to want to use it, but we’re try to build it so usable that they actually want to use it.  We’ll do everything we can to keep the learning curve of the system as small as possible, so it’s easy for people to look at it and just start using it.  It must be an intuitive, pleasant, frictionless interface, or it will be a failure.

You usually lose people when they look at the system and don’t know where to start, or were to go next.  They’ll complain about the system, and all they get back is “well this is a very powerful system, so you need to spend a lot of time to really learn how to us it.” 

This answer always infuriated me, whether it was Oracle or Linux or TFS.  You can always build a simple that is powerful and easy to use a the same time.  It is not easy, but that is why it is so rare.  It is the last mile of the design, and one of the hardest parts, and it’s what separates powerful products that most people hate from great products that most people love. 

Take a look at stackoverflow.com.  When you get down to it’s guts, it is an extremely complicated system, but it is still and inherently easy system to use, because so much effort was put into making it so.

 

Get On Board

 

Just one more note about getting buy-in.  I pitched this idea to a friend of mine, with whom I worked with for years in the trenches, and he said that the business analysts and project managers would never go for it, because it puts too much accountability on them.  I was taken aback by this, as it had a lot of the pessimistic “us-versus-them” partisanship that I want this tool to solve.  The idea here is that people usually have at least somewhat good intentions in this process, and so far it has been the tools that have failed them. 

However, he was right in some cases, and often times you find yourself working with people who are actively dishonest and try to cover there tracks.  In that case, fire them if you have the authority, because they are a detriment to the company.  Or, if you’re lower on the totem pole, quit if you can, because that is an awful place to work in and there are always better options.  Of course, again this may not be practically in all cases, so I often fall back on calling them out and trying to shine the light of truth whenever possible, but that has usually just gotten me in trouble.  This too can be problematic, because often you can come across as a complainer (After one such incident, one of my recruiters suggested I read Dale Carnegie’s “How To Win Friends & Influence People”, which was actually far more useful than I expected).

 

Here We Go

 

So that’s hopefully what we’re going to build.  At least part of it. 

Am I qualified to define how a requirements system should work?  Good gravy, no.  But I’m going to do it anyway.  I have a rudimentary understanding of some basic principles, but I am by no means an expert in requirements analysis and collection.  I just see a broken system and want to improve it.  It may not have all the bells and whistles that some other requirements systems have, but that’s OK. 

And of course, this is an exercise in developing software, not building a marketable product, and so that grants me the latitude to pursue things that I’m completely unqualified to pursue.  Plus sometimes it’s just fun to let go and try to do something you suck at.