On Wednesday 29th October, at the offices of Rule Financial in the heart of the City of London, a group of software development professionals along with several executives from the IT world met to discuss the thorny question of how Agile Development is changing things for testers.

It undoubtedly is of course, but the rate of change, and slightly amorphous nature of Agile is leaving some testers catching their breath. During the course of this inaugural meeting we covered a lot of ground, and no doubt at future sessions we will focus on specific areas of pain. But for now I have placed the minutes from this meeting here for you to read and enjoy. Please feel free to comment.

Objectives of the meeting

•    The Agile Challenge -Vendor and Client   
•    Practical ways to test whilst "doing Agile”   
•    What should T-Plan adopt to keep step?   
 
Introductory Presentation
Matthew Furneaux of T-Plan led the meeting with a brief presentation outlining the major challenges of testing when done within an Agile development environment. The key points were as follows:

1.    There are no big documents specifying every detail of the requirements and functionality to test against. Only small pieces of documentation per feature and details captured verbally through collaboration.

2.    The software is tested early and throughout the lifecycle while it is still being developed. In other words it is a moving target.

3.    The idea of writing test cases up-front, before the software is developed, so acceptance tests form part of the requirements analysis.

4.    The idea that some tests will be automated at code level and implemented by developers.

5.    Much greater emphasis on automated regression testing due to the fact that feature-level testing has been completed while the code was still being developed.

---

Inevitably when Agile development is employed, the role of the tester is challenged, pushing testers to think and act in a different way.

1.    With User Stories, there's little difference between a requirements scenario and a test case. Some requirements are implied from the test cases.

2.    If test cases are written up-front, there’s a fine line between requirements analysis and test analysis.

3.    A Business Analyst may have special skills in workshop facilitation, interviewing, gathering business requirements, etc. But, regarding functional analysis, Tester and Analyst roles start to converge.

4.    If unit tests are automated, a tester needs to work with developers to ensure that the tests are complete and appropriate, and that all-important scenarios have been identified.

---

As a result of this dynamic shift, testing behaviour begins to change:

1.    Testers need to avoid duplicating tests that have already been adequately unit tested when they and others come to system test the features later in the lifecycle.

2.    Automated regression tests need to be written and maintained.

3.    The tester's involvement is increasingly important throughout the entire lifecycle of the development, not just in the latter stages of development.

4.    The role of tester is more aptly described as Test Analyst, and is more of a general QA role for the team, not necessarily the person who devises and executes all the tests.

---

But what do real-life testers make of these changes? The following comments provide an insight:

“A Tester's life in traditional development methods was reasonably straight-forward. Give a Tester a spec and a finished piece of software, and they can check it works as specified." How different is it in Agile? I am a Tester and working on Agile. The only difference is I do not get a spec but I too get a piece of software that needs to be tested again and again. I have read that Agile is more challenging for Testers but in what way? I am still not convinced what the challenge is for a Tester in Agile!”

“My feeling is that not only must the tester's role change, but also the testing role of everyone else in the team. In my mind the fundamental change should be in what the tester (System Tester, Test Analyst) focus their testing on. Agile calls for far more discipline from all involved. Surely between BAs and Developers they should know whether specs were implemented correctly. This can be done through unit & integration testing and through prototype reviews. This will leave testers to focus on an often overlooked aspect of testing, the stability of the system.”

---

But what did the working group have to say about these challenges? The discussion was wide-ranging and covered a lot of ground, although not always in a logical order!

The notes that follow aim to capture the key lines of discussion, reference the experiences of the various members of the working group, and capture some helpful advice as to how to overcome the common challenges.

The User Story evolves over time. Management of this artifact and how to incorporate it was discussed.

When considering the three C’s of Agile - Card, Conversation, Confirmation - Simon Parry explained that at RBI the initial approach was to take the entire user story into T-Plan. However, this duplication of information was not seen as a practical way forward. The approach now being taken is to reference the user story as part of the Confirmation and then to find a means of managing change.

Tracy Swift (OCLC) discussed how OCLC use a WIKI approach to document and manage change of their user story. This is a truly collaborative method for sharing knowledge, and is a highly Agile approach.

The configuration and change of key information needed management during a fast moving process. Demonstrating control of this not only provides for an efficient process, but one that is auditable and necessary in a regulatory environment.

---

Roles and Resourcing in Agile. Different approaches and ideas were put forward and discussed.

Tracy Swift discussed the lack of Business Analysts in their organisation and that it was the developers who performed the early Business Analysis. The group discussed this dilemma, which was recognizable to all concerned and made the following suggestions:

•    It was vital that the business analyst role was performed – whoever performed it, given team size etc – otherwise the early stakeholder need would not be correctly gathered.

•    Real business involvement was key to project success – throughout the project, not just at start and finish as in the Waterfall/V model approach.

Also testers were not always invited in at the start of the project activities.

•    It was put forward the role of the tester should start at the very onset of the project as the stakeholder need is established. This allows the tester to develop the most efficient test plans as part of the team.

•    We have to tear down the old fashioned proposal that a tester ‘sits at the right of the V model’ and waits for good documentation to arrive. This does not allow for the evolving User stories over time and alienates the testers.


With limited resources people would probably need to perform multiple roles.

•    Without a defined process and role description it would be easy for people who are not able or qualified to perform these roles, to do so badly. For example a very technical developer may not be capable or qualified to clearly establish the stakeholder needs.

•    It was considered difficult to find good testers for an Agile project. Those were able to work from the onset with evolving requirements and the ability to interact with business & development enjoyed the greatest success.

---

Size of Project and Applicability of Agile. Where does Agile work?

David Little (Rule Financial), described an Agile project for which he was responsible, with 20 developers involved in a Sprint along with three testers. He reported that this was working well, despite the size, but any larger would become unwieldy and less manageable, especially when using Agile methods.

•    The practical limit was discussed and thought to be around a maximum of 20 in project size.

Simon Parry (RBI) described that at RBI, “Sprint-Streams” would work as projects as part of a programme but that the streams needed to come together eventually.

•    How this occurred and the integration testing to ensure correct interfaces was discussed.

•    It was felt that more traditional integration testing approach was appropriate at this point – this area needs further debate and will be returned to in more detail at a future meeting.

---

Role of Tools (including T-Plan), Regression & Test Automation.

Tracy Swift (OCLC) described their environment where they were developing with JIRA as the overarching PM tool, integrated with T-plan for test management and QTP for Test Automation. Once completed it is hoped that we can demonstrate this to the working group.

David Blades re-iterated that using a solid Test Management platform was crucial to ensuring proper management of projects, especially when developed using Agile, as the potential for problems is inherently greater.

Pat Ward (JPMC) described his commitment to tools that can genuinely fit his complex, and changing specification. As the financial world morphs into its next phase, the governance and oversight of regulators will only increase, and as such tools that drive quality, and prevent errors are vital.

David Little raised his desire to see greater convergence of tools in the testing space, where there is more compatibility enabling easier integration of numerous technologies. Agile development should not be hampered by vendor restraints – that is very un-Agile!

Steve Marshall (T-Plan) stressed that T-Plan wished to be at the vanguard of this movement, and is actively positioning its suite to be at the heart of the testing world, supporting testers as they learn to adapt to Agile development, and ensuring that all collaborative and associated solutions can easily interact with T-Plan.

Test Driven Development was briefly discussed but will be held over for a future meeting.

---

At this point, the pub beckoned and the meeting was adjourned until a future date, to be agreed by the members of the working group.

T-Plan wish to extend our appreciation for the kind hospitality shown by the directors of Rule Financial in allowing this inaugural meeting of the Agile Testing Working Group to be help at their offices.

---