Software Process is great topic for endless conversations (and battles) among IT professionals. Every now and then, a new process is arriving on the market that is introduced as the new marvel of the world, the new silver bullet, thanks to which every software project is going to be a never seen before success. We can name a few:

  • Waterfall
  • V-Model
  • Prince 2 (although this one is not really an IT project process)
  • Unified Process
  • Agile family
    • eXtreme Programing
    • Scrum
    • Cristal
    • DSDM (Dynamic systems development method)
    • Kanban
    • Lean
    • Etc.
    • Etc.

confrontation
During our decades of projects experiences, we have encountered many of these processes, used on various projects and in various circumstances. Today, I would like to talk about a project during which we have used a combination of a “proper” (in the sense of created with this purpose) software development process and a more generic project management approach. The two processes are:

  • The Unified Process (also known as Rational Unified Process, before IBM bought Rational) created initially by Ivar Jacobson, Grady Booch and James Rumbaugh.
  • Critical Chain, created by Eliyahu Goldratt

 

The Unified Process

Unified Process-iterationsThe Unified process is not only a process but more generally it is a framework that has been created for being tailored for each project. It is the first iterative and incremental framework that has been applied widely in the industry.

It is composed globally of four Phases, each Phase being composed of Activities.

One of the most interesting elements is the Phases as they describe periods of the project during which the objectives are different. The first phase, the Inception is about scoping the project, identifying a possible architecture and listing the risks to come. Then comes the Elaboration phase, mostly about validating an architecture, fleshing out the specifications with Use Cases and making an accurate estimate of the project in terms of time and manpower. The third phase, Construction, is about building the application in iterations and then the Transition phase is about deployment to target users and entering into the production mode.

This approach has been used on numerous projects around the world. Unfortunately, in most situations, people did not realize that it was built for being tailored to the project. We have seen many project managements asking their staff to “do UP” by the book. UP being a very complete framework, its implementation became very heavy and against the “Agile spirit” of the framework. (UP was never part of the official Agile methodologies).

 

Critical Chain

Methodology : Critical Chain

Critical Chain has been created by Eliyahu Goldratt and introduced in a book with the same title. The origin is not related to software but to the use of the Theory of Constraints. It is used as an alternative to Critical Path Analysis. I can actually recommend the book itself. It is written as a business novel and is easy and enjoyable to read.

Without going into much detail about it, it is using the principle of resources dependencies. It introduces the concept of buffer and suggests to monitor the consumption of the resources globally rather than at the individual level. It also uses the Student Syndrome (the phenomenon that many people will start to fully apply themselves to a task just at the last possible moment before a deadline) and the Parkinson Law that does not need any further presentation.

In the end, one of the most surprising element of this framework is that when the management is receiving the estimates for each tasks of the project by the resources, rather than applying the usual “+ X%” for contingency, Critical Chain is d

ividing each estimate by two, allocates one half to the task itself and the other half to a buffer available to the whole group of resources. So, the projects workers, when asking for 8 days to complete a task, get only 4 days and the other 4 go to the pool. Based on the Student Syndrom, resources, seeing that they only have a portion of the time they believe they need to complete the job, start working on it at full speed straight away rather than waiting. It is important to notice that there is no need to hide the global principle to the team. It is not a sneaky process. Everyone is aware of the rules. But in all the tasks to complete, some will actually be completed within the half time; some others will need a bit more time. But the time is redistributed to the workers only at needs. And those who actually complete the task in less their initial estimates are rewarded by the fact that the time saved is actually going to the tasks that were in need of more time than initially estimated. They then contribute to the good of the project rather than looking bad because they overestimated the time needed for their task. Those who do need more time will receive it without looking bad as this is expected.

 

A blended software process on a blended project

 

A customer we were working for came to us one day with a tricky problem. They had a system in production for which they had identified the need to make serious improvements. The business needs had not been followed by the system and it was looking worse every day to the users. They had asked their IT department a report on what needed to be done to implement these improvements. The report came back with the following conclusion: it was not possible to improve the system; they needed to re-start from scratch. Estimated cost of the project: about 1 million GBP. The business manager who came to us said he did not understand why the system had to be re-written from scratch. It cost the organization a lot of money to build and they were now told to bin the whole thing. What would Liemur do?

We asked him if they still had the requirements and some technical documentation about the project, like the architecture document and hopefully the tests used to UAT the project. He admitted that they had absolutely none of that and that the team who built the system was gone some time ago during a redundancy plan. Did we have an alternative to build the system from scratch?

After studying the situation further we offered to retro-engineer the application, document the system up to a point where the conclusion could be supported by facts. In short: did they really need to bin the system and start a brand new one or could they build on the existing one. If yes, how?

The cost for retro-engineering an application is not small. It requires very good technicians, with excellent analytical skills. As one can imagine, the application had no comments in the code or in the database. Everything had to be figured out.

The problem for the customer was that they could not spend too much money on that retro-engineering, especially if the conclusion had to be that indeed the system had to be binned. We then agreed on a fixed price job that was acceptable to them and a calculated risk to us. We had to deliver a business conclusion to make the job worth it. Below a certain amount of facts, it was not possible to conclude.

 

BlendingI was the project manager of this job and I needed to optimize the little time we had as much as possible. The customer was fond of Unified Process (they called us in to implement that process in the first place) and more importantly familiar with it. So I decided to follow that process.

 

So, the situation was the following:

  • The IT project was not about development but about understanding what others had done è unusual!
  • We had to adopt Unified Process but in a fixed price context è UP was not meant to do that
  • The deadline for delivering the conclusion was known at the start è against the Unified Process principles
  • The job was to be delivered in phases. If we were to break any pre-defined deadline, we would not be paid for the job of that phase. è High risk for us!

 

Unified Process was fine with me but I felt that I needed something to tighten the whole project in more controllable pieces of time. I then decided to mix Unified Process with the Critical Chain principles. Time was a very strong constraint for that project. I had to take that into account.

 

The result, in terms of process was that I followed the Unified Process but cut every Activity’s allocated time in half. I made the team aware of that management system and formally created a schedule for each delivery based on that principle. I made clear that being early on a task (compared to initial estimate) was not a sign of incompetence but the result of the difficulty of assessing the tasks. For every task finished early, they were building a buffer that would allow them to get less stressed when we would arrive closer to each deadline. We had 3 Phases following the Unified Process principles, in which I removed the Transition Phase as we had nothing to deploy.

The project was not about writing new software, so I had to redefine the objectives of each Phase the following way.

Inception

In this Phase, I had to make sure that the project was making business sense. We had to browse the code and the database to ensure that the content was written in a way that we could make sense of and understand the behavior. We used a Wide and Shallow strategy to investigate the system. i.e. we scanned as much of the system as we could to get the best possible overview of the nature of the code and the global architecture. We could summarize this phase the following way:

  • Get an idea of the global architecture
  • Scope the amount of work to do
  • Identify risks and potential difficulties that could stop us from completing the job

Elaboration

During that phase, I wanted to confirm beyond any doubt that we would be in a position to complete the job and deliver a conclusion valuable to the customer, i.e. can they, yes or no, improve the existing system?

To achieve that we needed to document the system architecture to the point where we can say: “It works this way! Here are the components of the system; here are the connections to the database; here are the architectural patterns in use.”

Construction

Once the previous steps achieved, the rest of the job was to document enough of the system for any team to be able to start expanding it if possible. If the extension was not the way forward, the objective then, was to allow a development team to use the produced documentation to specify the new system. As a matter of fact, if we were delivering enough information about the existing system, a lot of effort could be saved for the next one.

 

Project Outcome

First of all, I should cut the chase and tell you that we had a fantastic success with this project. The customer was delighted and we had a great time with it.

Project_successThis project was also a really interesting one in terms of Project Management. The team I used was made of highly qualified experts and they were not very much used to be told what to do in terms of process. But they understood the critical aspect of timing and the risky consequences for us to not deliver. They therefore gratefully played the game with me. We cut the job in small tasks, we made estimates, we prioritized tasks, we planned, etc. We then cut every estimate in half and put the remaining half in a pool. Everyone was keen to finish the job within the cruelly shorter time than what they though they needed. We could not compromise on quality either because we could not “half understand the system”.

Days after days, the rhythm was in place. Communication inside the team was fast and to the point. We were in the same boat and were rowing at the same pace.

After each phase delivered to the customer we celebrated that we succeeded (and were paid), therefore still in business. One phase after the other the team was even keener to make it. On the last delivery, we made a big presentation to the customer’s team. We explained how their system was working, what the possibilities were and why they did not have to start everything from scratch. No, they did not have to spend a million pound on it. In fact about a fifth of this amount would do the job. Here were the technical steps toward the improvement.
It is pointless to tell you how happy the customer was to know that.

Conclusion

Process is very often affair of dogma. Many IT specialists will swear only by the use of such or such process. At the time of the writing of this article, Scrum is probably the biggest trend in the industry. I am, I shall say “we are”, Scrum certified at Liemur, but from my point of view, a process is about organizing tasks. What one should never loose sight of is the “purpose” of what we do. If I do Scrum out of dogmatic belief, I miss the point. If I reject Scrum out of principle, I miss the point again. If I refuse a fixed price approach to a project because it is “against the process” (in this case UP) I can miss a great opportunity. If I do not use Critical Chain because it is not targeting Software Development projects, I miss a possible evolution to my current knowledge!

 

At Liemur, we have, a long time ago, decided to never be dogmatic about anything related to our job. We take risks, we deliver projects and we make customers happy. Customers do not care much if you have used this or that to reach your goal. Being creative in your approach is a plus in their book. That is what we do and the project presented to you today is a concrete example of this philosophy.