How to estimate testing efforts (6 approaches to get test effort estimate)

Test effort estimation is a skill required of a Test Lead or a Test Manager. However, test effort estimation is not a skill that one can learn quickly. It requires understanding of several key concepts and practice. In this post, I will explain what test effort estimation is, point you to your existing knowledge of estimation and provide you the key concepts that you can use in your estimation.

First of all, we should understand what we mean by software test effort estimation. Test effort estimation is answering two basic questions about testing:

I. What will be done?

II. How much effort would it take?

There are other questions e.g. I. Who will do what? II. When will they do it? III. How will they do it? but these questions are not related to effort estimation but to planning and scheduling.

Even if you have not estimated test effort before (having relied on the effort estimates given by the client or your project manager), keep in mind that you do effort estimation on a regular basis. Let me explain. Do you recognize the following situations?

1. You are appearing in an examination. The duration of the examination is 3 hours and you have to answer n questions. You average the time available for answering one question while leaving out certain time for revision at the end. You look at the questions. Some questions are easy for you but some are not. You reserve less time than average for answering the simple questions and more time than average for the difficult ones.

2. You have to attend a job interview. The interview is at 10 a.m. You estimate the time it would take you to reach the interview venue, say 1 hour. You add some time e.g. 30 minutes for delays like traffic snarls. You estimate some time, say 30 minutes for collecting your documents and some time, say 30 minutes for dressing up. This means that you would need to wake up no later than 7:30 a.m. that morning to reach your interview venue in time.

3. It is the beginning of another day at work. Your manager has given you 20 test cases to execute today. In addition, you need to complete the annual self-appraisal form. You estimate that it would take you 1 hour to complete your appraisal form. Out of 8 hours of your work day, you have 7 hours remaining. You reckon that you need to execute a test case every 21 minutes (7 hours X 60 minutes / 20 test cases).

If the above situations look common to you, it means that you already do effort estimation even if you do not consciously recognize it as such.

Next, let us see the factors that you need to consider before you do test effort estimation:

a. Size of the system

It would take longer to test a larger system. In some projects, it is possible to know about the size of the system in terms of Function Points, Use Case Points or Lines of Code. You should take the size of the system into account when estimating the test effort.

b. Types of testing required

Sometimes, it is important to perform multiple types of testing on the system. For example, other than functional testing, it may be necessary to perform load testing, installation testing, help files testing and so on. You should create the effort estimates for each type of testing separately.

c. Scripted or exploratory testing

It may be feasible to only execute test cases or do exploratory testing or do both. If you intend to do scripted testing and do not have test cases available, you should estimate the time it would take to create the test cases and maintain them. Scripted testing requires test data to be created. If the test data is not available, you should estimate the effort it would take to create and maintain test data.

d. "Non-testing" activities

Apart from creating and executing tests, there are other activities that a tester performs. Examples include creating test logs/ reports, logging defects and entering time in the project management tool.

e. Test cycles

By a test cycle, I mean a complete round of testing (build verification testing followed by attempted execution of all test cases followed by all defects logged in the defect tracking system). In practice, one test cycle is not sufficient. You should estimate the number of test cycles it would take to promote the system to the client or production.

Now, let us understand the various approaches that you can use for test effort estimation. You may choose any of these approaches for your estimation. However, in my opinion, a combination of multiple approaches works best (by best, I mean that the effort estimates are close to the real actual efforts). In any case, you should be aware about the following approaches:

1. Use historical data from other projects

This approach is useful when you have effort data available from earlier projects which are very similar to the current project. For example, this approach is useful in the case of long-running projects where the test effort data from previous releases is readily available.

2. Your organization's approach

Your organization may have their custom approach to estimate test effort in projects.

3. Delphi method

This is useful when you have a number of experts knowledgeable in the testing to be done. The experts estimate separately and then their estimates are consolidated

4. Use your own expert judgment

This approach is useful to arrive at a rough test effort estimate quickly.

5. Software size based approach

If the size of the system is available and the formula to convert software size to test effort is available, this approach may be used.

6. Activities based approach

This approach is useful if you can list the activities required. This approach may be used Top-Down (listing the high level activities and breaking them down to lower level activities) or Bottom-Up (listing the individual activities and combining them to higher level activities). Using this approach in the Top-Down manner is better since you can control the level of detail in your effort estimate. Remember to consider activities for each type of testing, any test cases or test data need to be created, the "non-testing" activities and the multiple test cycles.

As I mentioned before, you may choose any approach to do your test effort estimation. However, using at least two approaches is better. This way, you can compare the test effort estimates and address any obvious problems. Whatever hybrid approach you choose, you should document the assumptions made by you in the estimate.

Once you have arrived at the test effort estimate for your project and have convinced the stakeholders that it is a reasonable estimate, it does not stop there. You should track the actual progress in your project constantly to see if it is in line with your test effort estimate. You may find that some of your assumptions were not correct. You should revise your assumptions and your approach in line with your observations.

Continue to use your refined test effort estimation approach across test cycles and releases. In time, you should have a good estimation approach available with you.