You hear all kinds of questions about what project methodology to use, and in what situation. Should it be Agile, Waterfall? How does one approach the various project roles within each? What about documentation, is there an approved standard? Is there an approved standard for anything?
Experience keeps telling me that the answer to all of these questions come out of the context we are working in. For instance, my organization tends to manage projects via a Waterfall approach, as sort of an umbrella organizing function over projects. This is because we contract for large, often multi-year software implementations that involve custom enhancements to products, as well as a great deal of integration work. We need to get the project pretty well defined upfront for contractual purposes. Then under the umbrella there may be one or more Agile teams doing their work on separate streams, while some business analysts are working off to the side, writing interface specifications, training, and gathering further requirements, handing things off to developers, and so forth. And even under the umbrella of a Waterfall methodology, there are constant iterations to gathering specifications, writing documentation, specification review, development, testing, reviewing the results, and often rewriting specifications based on those results.
There are always conflicting needs. First of all our clients need to know what they are paying for, how much they are paying, before committing to a project, and at the same time there are also many unknowns. You have to get good at what you’re doing by doing it, and often there are going to be mistakes. Part of the reason why is that the it that you have to get good at, in order to have mastery of the entire process, is not a thing, but only an idea. At best, it’s a moving target.
This is why you need to hone certain instincts, so that you become sensitive to the needs, not only of the client, but to the needs of the entire project, or just your little sliver of it. You have to know when to stop going in one direction, and turn a corner, or start over from the beginning.
There’s a popular assumption that failure is due to lack of organization, or failure is due to people not knowing what they’re doing. Both of those assumptions are false. Very often the cause of failure is over-organization and lack of flexibility. And even more often the cause of failure is an inability to admit that one is always working in the dark, that we don’t really know what we are doing until we do it. These two things work hand-in-hand – inflexibility is often due to believing you know what needs to be done and the best way to do it when you don’t, or perhaps faking it too hard.
Pretending one knows what one is doing, and relying too much on the scaffolding (i.e. project management methodologies, and other things they teach you in certification programs) instead of the process of discovering what the requirements are and creating the solutions along the way, is more conducive to producing content for primetime sitcoms than it is to actually getting the work done.
Don’t get me wrong – I’m glad the scaffolding is there. When I’m working in a BA role, I’m often so deep in the weeds, into hair-splitting details of a piece of code I’m trying to design, that it’s good to know there’s someone there looking over the whole thing from above, with her map (project plan) of the entire battlefield, so I don’t have to keep track of the next thing I need to do, and the sequence of things, and who the best people are to talk to for that next particular bit of business. But at the same time, I am giving my feedback, and we are rescheduling things, splitting things into multiple tasks, and recombining things where new synergies have been discovered. A project is a living thing, made up of living parts, and like all living things is full of surprises.
So apart from the requirements one needs to meet, that make up the project – i.e. what gets delivered to the client – there are the meta-requirements of the project itself, its incremental discovering what it is, just like all of us.