Agile Projects are Really Waterfall with Agile
Photo by Jeremy Bishop
Sorry agile die-hards. Most all “agile” projects are really agile with some waterfall and process. This is a good thing.
I know I may ruffle some agile purist feathers, but please give me a chance to explain in this post. I’ll start with some assumptions.
Assumptions
Perpetual projects that never end (ongoing product evolution) do have some sort of planning phase.
The definition of “agile” is an iterative approach allowing for change and deploying on a continuous basis rather than just one big deployment at the end.
The definition of “waterfall” is a sequential list of tasks where most are dependent on previous tasks.
The definition of “process” is a predefined set of tasks or actions to be followed to achieve a desired end.
Why My Agile Project is Really a Hybrid
One of the most common use of agile approaches with scrums and sprints is software development projects, and more specifically those with perpetual products development cycles. My company is a a perpetual products company continuously improving our Pie application. At Pie, we use agile methods with disciplined daily scrum meetings and ongoing deployments that can be as often as a daily. In addition to the scrums, we’re always interacting with our teams on ideas, issues, and clarity. Our main emphasis is providing working software for our customers and we continuously collaborate with those users. We are constantly responding to change as it’s hard to plan in detail more than a few weeks at a time.
Sounds like true agile doesn’t it.
However, I still will call our never-ending development project and hybrid waterfall/agile approach.
Waterfall Part of the Project
I’ll use the example of our big project of re-building Pie from the ground up. Before we started to code, there was a clear set of processes and waterfall tasks I knew we needed to accomplish. Not that I’m brilliant. It’s just stuff I learned that works from my years of project management and start-up experiences.
This waterfall part of the project looked like this:
Define objectives and goal
Determine budget scope
Draft first phase application feature requirements with design examples
Identify the best modern-day technology
Find the right new-technology development team
Set up DevOps environments
Etc.
The above is just a high-level example of my Plan phase. Most of the steps were very linear and a few were done in tandem. Nothing at this state required agile iteration for constant change and unknowns.
Although, there are some startups that just wings it, stumble, try something different, stumbles again during the pre-design/coding phase. You could call that being agile, but I call it just inexperienced.
Agile-Scrum Part of the Project
Then came the Design and Coding phases. This is where agile started. About a few weeks before starting the coding, I narrowed down detailed product design using tools like Sketch and InVision. I broke down my requirements into the smallest components and they became the stories (which we call tickets). We kicked off the scrum meetings and our first sprint. The tickets went through a linear process including such as, not started, in progress, pull requests (peer review), QA testing, and ready for production. Some tickets gets bounced back as rejected, others are leapfrogged. Then the “readies” get deployed on the same or next day.
This agile part of the project looked like this:
Review yesterday’s work, determine today’s targets
Discuss challenges and roadblocks
Bounce ideas with a few customers
Determine focus for next couple of weeks
Design next set of features'
Prioritize
Code, test, deploy, and then play with pies!
Update the next week’s focus based on feedback, enlightenment, and roadmap targets
Etc.
The above is just a small example of our ongoing agile process. Since Pie is an application in a competitive market, we need to continuously make it better. Therefore we’ll be skipping the close phase found in projects that do have an ending.
However, this project does have ready-made recipes as waterfall steps for certain conditions, such as DevOps Amazon Web Services set up or changes, Yellowfin BI tool set up for customers, or certain Stripe transaction processes, all part of the project’s sustainability.
When Projects Are Hybrids
There are some projects that are just fine with 100% waterfall, such as when all knowledge on what to do is predefined, the risk of change is tiny, the project timeline is short, and final deliverable is appetizer size.
What about larger implementation projects that have some unknowns, risks of change is likely, timeline is months, and the final output is a big meal. These are common, especially with consultancies providing systems integration or IT departments building or upgrading something sizable.
Many of these projects have finite budgets and timelines along with specific end deliverables. Of course, projects like these could spur other related projects or themselves go into second phases, again with certain budgets and end dates.
Let’s say your company is implementing such a project and you find you could use agile-scrum methods. You may hire agile experts to help or do your own research. You may find agile evangelists who scream, “waterfall is dead and long live agile!” It will be easy to drink the juice, believing that you must step into the new light. Cool.
Ok, let’s say you do jump it. You make arrangements with your client that delivery will be ongoing throughout the engagement and the big set-in-stone budget may be broken into small budgets based on deliveries with some need for frequent reviews. You then charge your development team to prepare the agile mixing bowl. They choose their own project tools, such as JIRA or Trello and decide the best methods for people collaboration, scrum practices, etc. You’re off to the races.
Are you now fully agile? Hold on partner!
The Agile waterfall Sandwich
Your project has a much larger set of tasks, processes, and phases than just the design and development phases. You or your project manager most likely know how to implement these projects. The first steps include your best practices for project inception and planning. This blueprint is pretty solid.
What about requirements? Most companies should not go too far with trying to define requirements in the pre-agile phases unless these requirements are truly well known and locked down. I suggest you move most of your requirements definition to the scrum phase and tackle them as they become more clear over time with the deployment iterations.
And how about post development? Even though delivery was done in small chunks and you don’t have a big go-live, you still close out the project. There are steps to wind down, get final sign offs, etc. Again, you’re in the business of doing these projects and you know what to do in a very linear process fashion. It’s in your framework.
Then there are some of us who will do iterative sprint releases of development, but not fully deploy them to live production until later at set times. This may be because we don’t have the architecture to support flexible production releases. Forrester analyst Dave West introduced the Water-Scrum-Fall model in 2011. The “Fall” part is for the more sluggish production deployment model. But, for the most part, let’s assume you do have the architecture for sprint releases to production and using the waterfall close phase for the business closure of the project.
See where I’m leading?
You’ve got yourself a hybrid sandwich. You’re working with waterfall at the inception and planning, agile/scrum for design, development, and mini-deployments, and then back to waterfall for closing out.
Full disclosure. My model above is pretty simplistic. In reality your mix of waterfall and agile could be a triple decker or more as you may mix them throughout the engagement as needed. My point is although the development team may feel their project is pure agile and your project manager may see it as waterfall, the overall project from start to finish is really a hybrid of agile and waterfall.
Hmm… should we call this “Wagile”?
Although the planning and closing team have great people-focused collaboration, it’s likely that your your project manager is running these phases with tools like Microsoft Excel or Project. Across the project, you find that teams are using different project tools for different parts of the project. Not ideal for cross collaboration, accountability, and transparency.
Individuals and interactions Vs Process and tools
The Manifesto for Agile Software Development says to value individual and interactions more than processes and tools. In the true world of hybrid projects, I believe you need to value them equally. On this point, see my next blog about the Agile Manifesto and my proposed Hybrid Agile Manifesto.
Finishing off my above hybrid development project story, I had realized last year how common it is for companies trying to implement agile really end up implementing agile along with waterfall. I found that tools on the market are either one or the other for each project. Therefore, I responded to this market need with a new agile board feature that’s designed to make it stupid easy to combine processes of waterfall and agile together in the exact same project using the exact same tool. I hope you check it out.
Thanks for reading.
Paul
Written by Paul Dandurand, PieMatrix Founder