What is Automated Testing and when does it occur in a Scrum project? When changes are made to software that has been previously validated, even the smallest revision has the potential to disrupt the existing code. Software developers implement Regression tests to ensure that modifications or additions to the code didn’t break any of the existing functionality. Another objective is make sure that earlier variances are not re-introduced into the code inadvertently. Regression tests are used to show that product functionality will continue to run after code changes. This practice will ensure that the software has not regressed or been broken. This would mean that software components that were initially error-free have become defective due to unintentional code changes.

With Agile frameworks such as Scrum, the repetitive activity of running a suite of Regression tests can be very complicated and lengthy. Without the utilization of automated testing tools, referred to as Regression Testing tools, agile testing would involve a high level of complexity. After development and when a product increment is added, Integration Regression Testing takes place. The code is integrated with the existing product and verified to determine if the addition did not “break” anything. A Scrum software development best practice is to re-run the appropriate regression tests after every code change to ensure that previous tests continue to pass. 

Regression Testing Strategy

It is believed, before the Agile Values and Principles became so popular, that Regression testing was easier. The complexity comes from Sprints that are often two weeks in duration and even shorter. Every time a software change occurs, the probability for new issues to be introduced into the software increases. Regression testing is the method used to uncover surprises in the code. An effective way to approach regression testing is by means of a “Regression Board”.

When using a Regression Board, each member of the Scrum testing team needs to identify a change that they worked on during a Sprint. Once all changes have been identified, they are arranged in a top down order based on which ones should be tested first. The order can be based on risk levels, customer importance, or any reason applicable to the project. Other team members reorder the items based on technical or value-driven information that is known to them. Different team members add their own distinct project prospective when they rearrange the items on the board.

As the work is completed, pre-release testing commences. The Regression testing team refers to the Regression Board to make educational decisions about removing items from the list, adding additional staff to test low risk items or possibly extending the release itself. As the testing team utilizes a distinct and organized list of regression testing items, it creates a major shift in thinking from “We don’t have enough time” to “We know what’s important and how much time we have”.

Automated Testing Tools

When developers create new features, they can do their testing with automated Testing tools. This practice helps decrease risk when making code changes and results in the creation of a change detection system. Scrum projects have their own specific challenges for automation:

• Project scope not clear
• Multiple Iterations
• Light documentation
• Frequent and early automation requirements
• Stakeholder involvement

With traditional test automation, such as “test at the end” tools, also referred to as record and playback tools, a test team could not perform regression testing until after all software was developed. These types of tools don’t really support Scrum environments because there are better suited to the Agile Values and Principles. Test automation at the beginning of the agile project is extremely difficult however, as the system is completed, it then starts to become more relevant for automation.

The Regression Test Suite – User Stories and Automated Testing

With Scrum, the user stories are selected for each Sprint and they used as the foundation for the test sets. As the Scrum team goes through each iteration, regression testing needs to be conducted to ensure that the existing functionality has not been “broken” by the addition of new functionality in each Sprint. As the test team completes the Sprints, the magnitude of the regression testing increases. This is another instance of where good communication is needed among the Automation Testing and Scrum teams (Scrum Master, Product Owner and Development Team).

The interaction between the client and the delivery team must remain collaborative. With heavy client involvement, comes more changes and more channels are needed for communications. The Test Automation methodology needs to ensure that the entire system, as it is developed, meets the business objectives for which it was intended and is fit for its purpose. Defects are automatically captured when an automated test script fails.

We found these books great for finding out more information on Agile Scrum:

Exploratory Testing

With the Agile Values and Principles, exploratory testing demonstrates that the tester has value as an important component of the test process. In addition, values from the Agile Manifesto are shared:

• Individual and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

With agile projects, automated tests that support code are mainly done by the development team and are executed from a programmer’s viewpoint. Exploratory tests are designed to find defects that are beyond the scope of automated programmer tests. The exploratory tests focus on possible gaps where automation may be weak. These tests are not structured and are considered “freestyle”. In comparison, Automated tests are used more for regression testing and Exploratory tests are used for new features.

Test Driven-Development (TDD)

TDD is a process that begins with creating tests for small pieces of functionality. Tests are developed before the code and it contains the specification and the validation of what the program needs to accomplish. TDD runs automated tests prior to the development of the software. TDD is often referred to as Test First-Development (TFD). Finally, TDD is the practice of modifying code so that a test that was previously designed will pass. The TDD process is as follows:

1. Add a test.
2. Run all tests and see if any new test fails.
3. Write some code.
4. Run tests and Refactor code.
5. Repeat