Projects conducted using Agile methodologies experience rapid changes A high rate of change means that there is an equally rapid change in test status, test progress, and product quality. Testers must impart that information to the team so that decisions can be made about how to keep to the planned work for successful completion of the iteration. Change can also affect features already implemented in previous iterations, so tests (both manual and automated) need to be updated to help mitigate the risk of regression.

Certain test activities are related to specific development activities. This means that the testing that takes place in different lifecycles may also be different. For testers to be able to work efficiently and effectively It is important for them to understand the differences between testing in traditional, lifecycle models (such as the sequential waterfall and V-model, or the iterative Rational Unified Process) and Agile lifecycles.

Agile methodologies differ in many ways including:

• How testing and development activities are integrated
  
  
• The nature of project work products will generally be different
  
  
• Names given to particular work products may be different
  
  
• The entry and exit criteria used for various levels of testing may be defined differently
  
  
• The type of tools chosen, and how they are used may be different
  
  
• Independent testing may be implemented differently

The way development / testing lifecycles are implemented in different organisations can vary significantly. Not all organisations adhere strictly to the ideals or all the principles of Agile lifecycles. The digression or divergence from absolute adherence to the Agile principles may be a result of appropriate adaptation and customisation.

Adapting and adjusting to the special context or needs of a particular project, is something that developers need to do, and it most likely follows that testing (and testers) will also need to modify their practices as a result. The clue is in the name “Agile”. Testing needs to be agile in being able to change its organisation and processes when necessary.

There is little point in adhering 100% to Agile principles and practices if the product fails or is too late at the end of the day. Developers and testers need to be pragmatic. This “agility”, being able to adapt, is a key success factor for testers.

As mentioned in the ISTQB Foundation Level syllabus, test planning is an on-going activity. This is also the case in Agile lifecycles. For Agile projects, two kinds of planning occur

1. Release planning and
   
   
2. Iteration planning.

Continuous integration is a practice whereby the entire code base is (almost) constantly being rebuilt and retested automatically.

When working on a change to an application, a developer will take a copy of the current code. As other developers upload newly changed code, this copy no longer reflects the repository code. Completely new code can also be added, as well as other resources that create dependencies, and potential conflicts.

If development continues this way without regularly merging back, there is an ever-greater the risk of integration conflicts and failures when the developer code is eventually uploaded and merged. When developers submit code to the repository they must first update their code to reflect the changes in the repository since they took their copy. The more changes the repository contains, the more work developers must do before submitting their own changes.

Eventually, the repository may become so different from the developers’ baselines that the time it takes to integrate exceeds the time it took to make their original changes.

Continuous integration is intended to combat these problems and to achieve two main goals:

1. Minimise the time and effort required for each integration
   
   
2. Be able to deliver a product increment that is suitable for release at very short notice

Delivery of a product increment requires reliable, working, integrated software at the end of every sprint. Continuous integration helps this by merging all changes made to the software and integrating all changed components regularly, at least once a day. Some Agile projects integrate tens of times each day.

These objectives require an integration procedure which is reproducible and automated as much as possible. This is more easily achieved by employing tools for:

• Configuration management
  
• Compilation
  
• Software build
  
• Deployment
  
• Testing and
  
• Version control

There are also tools specifically designed to help achieve continuous integration. Build tools can be linked to automatic deployment tools, which can fetch the appropriate build from the continuous integration server (or build server) and deploy it into one or more development, test, staging, or even production environments. This reduces the costs, errors and delays associated with relying on specialised staff or programmers to release builds to these environments.

Organisations using continuous integration typically use build tools to implement continuous quality control. In addition to running unit and integration tests, such tools can:

• Run additional static and dynamic tests
  
  
• Measure performance
  
  
• Extract and format documentation from the source code and
  
  
• Aid in conducting manual quality assurance
  
  • The continuous application of quality control can improve the quality of the product while concurrently reducing the time taken to deliver it because the traditional practice of applying quality control after completing all development is no longer necessary.

The continuous integration process

This approach is most often carried out as the code is written, debugged and checked into the shared source code repository (source control server). This is then used by the build server (continuous integration server) to compile and execute tests, providing feedback to the team on pass / fail results.

An automated continuous integration build and test process takes place on a daily basis (at least) and consists of the following activities:

• Static code analysis
  
  • Execute static code analysis and report the results
    
• Compiling
  
  • Compile and link the code, generating the executable files
    
• Unit testing
  
  • Execute the unit tests and check code coverage
    
    
  • Report the test results. These should be visible to all team members, especially when automated reports are integrated into the process
    
    
  • Continuous integration allows automated tests to be run regularly (where possible as part of the continuous integration process itself). This enables feedback on the quality of the code to be sent to the team quickly.
    
• Deploying
  
  • Install the build into a test environment
    
• Integration testing
  
  • Execute integration tests and report the test results
    
• Reporting
  
  • Post the status of all the above activities to a publicly visible location (or e- mail the status to the team)

Regression testing

Automated regression testing can be conducted throughout the iteration. Effective automated regression testing covers as much functionality as possible, including user stories delivered in previous iterations.

Where there is wide coverage in the automated regression testing it can help support building (and testing) large integrated systems.

By automating regression testing, Agile testers are more able to concentrate their time on manual testing of new features, change requests and confirmation testing or re-testing of defect fixes.

Benefits

Continuous integration can provide the following benefits:

• It is possible to receive frequent and timely feedback on issues with software code and its integration, so defects are detected more quickly
  
  
• Earlier detection of defects and other issues such as integration problems and conflicting changes allows root cause analysis to be easier
  
  
• Gives the development team regular feedback on whether the code is working
  
  
• Keeps the version of the software being tested within a day of the version being developed
  
  
• Reduces regression risk associated with developer code refactoring due to rapid re-testing of the code base after each small set of changes
  
  
• Provides confidence that development work is always based on a solid foundation
  
  
• Makes progress toward the completion of the product increment visible, encouraging developers and testers
  
  
• Eliminates the risks associated with scheduling a big-bang integration
  
  
• Executable software is available throughout the sprint for the purposes of testing, demonstration or training purposes
  
  
• Reduces repetitive manual testing activities
  
  
• Provides quick feedback on decisions made to improve quality and tests
  
  

Possible problems

Continuous integration is not without some risks and challenges:

• Continuous integration tools have to be introduced and maintained
  
  
• The continuous integration process itself needs to be defined and established
  
  
• Test automation requires additional resources and can be complex to implement
  
  
• Thorough test coverage is essential to achieve the advantages of automated testing
  

• Teams sometimes become over-reliant on unit tests and do not therefore perform enough system or acceptance testing

Agile teams should meet regularly to reflect on significant events and to take decisions aimed at remediation. This known as a ‘retrospective’ (also referred to as a ‘heartbeat retrospective’ or ‘sprint retro’). The meeting is held at the end of each iteration to discuss what was successful, what could be improved, and how to incorporate the improvements and retain the successes in future iterations.

The meeting is typically facilitated and follows a set format. Various formats have been used, depending, at least in part, in large part on the time allowed, usually between one and three hours. A good reason for a facilitator-led meeting is it affords more opportunity for all team members to contribute.

The timing and organisation of the retrospective depends on the particular Agile method followed. Business representatives and the team attend each retrospective as participants while the facilitator organises and runs the meeting. In some cases, the teams may invite other participants to the meeting. All team members, testers and non-testers, can provide input on both testing and non-testing activities.

Testers should play an important role in the retrospectives as they are an integral part of the team and bring their own perspective. Testing takes place in every sprint and is a vital part of achieving a quality product. All team members, testers and non-testers, can provide input on both testing and non-testing activities.

Retrospectives can cover many topics including :

• The development / test process including topics such as
  
  • test-related improvement recommendations
  • test effectiveness
  • test productivity
  • quality of test cases
    
• The testability of
  
  • applications
  • user stories
  • features
  • interfaces
    
• People issues – including team satisfaction
  
  
• Organisational matters
  
  
• Relationships within and external to the team and
  
  
• Tools for development and testing
  

Root cause analysis

Retrospectives can also be the place where a root cause analysis of defects can take place, which can drive testing and development improvements.

In general, teams should implement only a few improvements per iteration as this facilitates a sustainable pace of continuous improvement.

Summary

Retrospective meetings, when held regularly and appropriate follow up activities take place, are critical to self-organisation and to the continual improvement of development and testing.

Retrospectives must take place within a professional environment which engenders mutual trust. They should not be used as a witch-hunting or finger-pointing exercise – all participants should feel they are able to speak up without fear of reprisals.

The attributes of a successful retrospective are the same as those for any other review.

Projects can fail for many reasons, but poorly defined specifications is one of the most quoted as causing major problems in development. Issues with specifications can have many root causes, including

• A user’s lack of understanding into their true needs
  
  
• Absence of a comprehensive view of the system
  
  
• Unnecessary or contradictory features
  
  
• Vagueness
  
  
• Lack of objective criteria
  
  
• Unrealistic targets
  
  
• General miscommunication
  

and many others.

User stories

In Agile projects, requirements are generally captured through user stories. A user story is an informal, natural language description of features of a software system and are the smallest unit of work in an Agile framework. They are written from the perspectives of developers, testers, and business representatives, and describe how a piece of work will deliver a particular value back to the customer. Note that ‘customer’, in this context, is not necessarily an end-user, but may be anyone who depends on the work of the Agile team. Stories may be recorded in any agreed way, but most often on index cards, post-it notes, or digitally in project management software.

User stories are usually expressed in forms similar to:

• ‘As a user I can [do something] so that [something will happen]’ or
  
  
• ‘The system shall [do something in these circumstances]’

In other words, they are written in such a way that a business value can be assigned to them and so that it is possible to estimate their difficulty and how long they will take to develop. User stories are a good idea because they focus on functionality and on business value. It is easier for a product owner to sort the backlog if items are not technical. User stories are a lot easier to order than a use case and much easier to compose.

In development using sequential methodologies, requirements are first written, then formally reviewed by stakeholders. By contrast, in Agile development, the shared vision of requirements is achieved by conducting frequent, informal, reviews in parallel with the requirements being written.

In common with more traditional requirements, user stories must cover both the functional and non-functional needs.

Acceptance criteria

Each story includes acceptance criteria for the defined characteristics. These criteria should be defined in collaboration between business representatives, developers, and testers. They provide developers and testers with an extended vision of the feature that business representatives will validate. An Agile team considers a task finished when a set of acceptance criteria have been satisfied.

Tester input

Typically, the perspective of a tester can help to improve user stories by:

• identifying missing details or non-functional requirements
  
  
• asking business representatives open-ended questions about the user story
  
  
• proposing ways to test the user story, and
  
  
• confirming the acceptance criteria.

Techniques for gathering and creating user stories

User stories can be gathered using a variety of techniques:

• User interviews
  
  • Interviewing users is probably the most obvious means to identify user stories, but it takes skill and experience to do it effectively. Open-ended questions should be asked first, eventually honing in on contextual questions
    
• Questionnaires
  
  • Paper or electronic questionnaires are useful in organisations with a large user population. Having gathered and analysed information from a large user base a more focused set of interviews could be conducted
    
• Observation
  
  • Users would be observed in their daily tasks with the observer noting what the users do and what they are trying to achieve.
    
• Story writing workshops
  
  • These are formal meetings where users, developers and other stakeholders work through scenarios to create stories. Techniques that can be used in these sessions include brainstorming, mind-mapping and production of low-fidelity prototypes.

INVEST

Testers, in particular, may use the INVEST set of criteria, or checklist, to assess the quality of a user story. A good user story should be:

• Independent (of all other stories). User stories should be able to be moved from one iteration to another (maybe due to changing priorities). If a user story is linked to another, its movement by itself may be difficult.
  
  
• Negotiable (in other words, not an unchangeable contract for features). User stories should be able to be rewritten, or even discarded by team members, depending on a variety of business, market, technical or other requirements.
  
  
• Valuable. The story must deliver some value to the stakeholders. One of the Agile principles is to deliver valuable software
  
  
• Estimable (to a good approximation). If a user story cannot be estimated it will never become part of an iteration because it cannot be planned or tasked.
  
  
• Small (so as to fit within an iteration). Larger user stories (such as ‘Epics’ that may take more than one iteration to complete) are more difficult to estimate, so should probably be split into smaller stories anyway.
  
  
• Testable (in principle, even if there isn’t a test for it yet). A user story should only be considered ‘Done’ if it has been successfully tested. If it cannot be tested, it should not be considered a good candidate to be part of an iteration backlog.

If a user story fails to meet one of these criteria, the team may want to reword it, or even consider a rewrite (which often translates into physically tearing up the old story card and writing a new one).

The three Cs

The three C’s is a formula for capturing the make-up of a user story:

1. Card
   
   • This is the physical form on which a user story is captured and recorded – a card or a post-it note or even just a piece of paper.
     
     
   • It identifies the basic requirement, its criticality, estimates of development and test effort and duration, and the associated acceptance criteria.
     
     
   • The description must be accurate, because it will be used in the product backlog.
     
2. Conversation
   
   • Various people (such as users, developers and testers) concerned with a particular function or feature of the application discuss the requirements and how they may be implemented.
     
     
   • Normally, conversations would be verbal (and ideally face-to-face) but there will probably be e-mails or sharing of other documentation involved too.
     
     
   • The conversation explains how the software will be used. Testers, having a different point of view from developers and business representatives bring valuable input to the exchange of thoughts, opinions, and experiences. Conversation begins during the release-planning phase and continues when the story is scheduled.
     
3. Confirmation
   
   • The results of the conversations, decisions and agreements are recorded as having been taken – in other words are ‘confirmed’. The confirmation should be as formal a process as possible.
     
     
   • Acceptance criteria, discussed in the conversation, are used to confirm that the story is done. These acceptance criteria may span multiple user stories.
     
     
   • To ensure the criteria are covered sufficiently, both positive and negative tests should be defined.
     
     
   • A variety of participants play the role of a tester during confirmation, including developers and specialists in the fields of performance, security, interoperability, and other quality characteristics.
     
     
   • To confirm a story as ‘done’, the defined acceptance criteria should be tested and shown to be satisfied.

There are several different approaches in use for Agile development. Common practices across most Agile organisations include:

• Collaborative user story creation
• Retrospectives
• Continuous integration
• Planning for each iteration and
• Planning for overall release.

However, each approach implements the values and principles of the Agile Manifesto in different ways.

The ISTQB syllabus expects you to know about just three types of Agile approach:

1. Extreme Programming (XP)
   
   
2. Scrum, and
   
   
3. Kanban.

Projects employing Agile methodologies have iterations that are generally quite short – a duration of two weeks being common. This enables the project team to receive continuous feedback on product quality throughout the whole of the development lifecycle, including the early stages. One way to provide rapid feedback is by employing continuous integration (which we cover in a later lesson).


With sequential development methodologies, customers often do not see the working product until the project is nearly completed (for example, as part of a user acceptance test). Commonly, at that time it is too close to a planned release date for the development team to address adequately issues that the customer may raise. By enabling customers to see the product at appropriate and regular points helps to ensure that customers can provide pertinent and timely feedback, in parallel with product development, In this way, Agile teams are better able to incorporate changes into the development process.

Feedback that is provided early enough also helps the team to concentrate on the features of the application with the greatest actual business value, and / or those that are most likely to fail in production (in other words, features with the highest associated product risk). These features are developed and delivered to the customer first so that issues can be dealt with soonest.

Frequent feedback also helps in managing the team, because certain factors become more visible to everyone – such as:

• how much work (e.g. story points) can be completed in a single iteration?
  
  
• What could help the team to increase it’s velocity?
  
  
• What is hindering or stopping progress?
  

Benefits of early and frequent feedback

The benefits of early and frequent feedback include:

• Misunderstandings with requirements can be detected sooner than may have been the case without feedback. This is a good thing, since the earlier in the development lifecycle a misunderstanding is detected, the less expensive it is to rectify.
  
  
• When customers request new or amended features, they can be clarified and developed sooner, making them available for customer use earlier than would have otherwise been the case. As a result, the product will better reflect the customer’s needs.
  
  
• Through the application of continuous integration, uncovering and rectifying more quickly problems with product quality.
  
  
• Being able to inform the Agile team about its productivity and its ability to deliver what the customer wants.
  
  
• Promoting a consistent sense of movement and confidence that the project will deliver on time, in other words, project momentum. It has been shown that a lack of momentum slows productivity in IT projects

The essence of the whole-team approach is that developers, testers and business representatives work together in all phases of the development process. The whole team is involved in consultations or meetings in which product features are presented, analysed or estimated.

Testers work closely with both the developers and the business representatives to ensure that the required quality levels are met. This includes:

• supporting and collaborating with business representatives to help them create suitable acceptance tests
  
  
• working with developers to agree on the testing strategy and
  
  
• deciding on test automation approaches.

In this way, testers can extend their testing knowledge to other team members and also influence the development of the product.

The concept of involving testers, developers, and business representatives in all feature discussions is sometimes referred to as “the power of three”.

The whole-team approach is facilitated by daily stand-up meetings which involve all members of the team. In these meetings progress is communicated and any blockers are discussed.

Team composition

The whole-team approach means involving everyone who has the knowledge and skills necessary to ensure the success of a project. The team will not only include developers and testers, but also customer representatives and business stakeholders who will determine the features to be implemented in the product.

Team size

The team should not be too large – in fact it is best if it is on the smaller side – possibly as few as three individuals – and probably not more than nine or ten.

Team location

Ideally, the whole team should share the same workspace, as co-location encourages more communication and interaction and makes it easier. If this is not possible, then other communication facilities should be available – conference calls or MS Teams / Zoom for one-to-one interaction.

Benefits of the whole-team approach

The whole-team approach engenders more effective and efficient team dynamics and is one of the main advantages of Agile development.

Specific benefits of the whole-team approach in Agile projects include:

• Enhancing communication and collaboration within the team and with other stakeholders.
  
  
• Leveraging the various skill sets that the team has to benefit the project.
  
  
• Making the whole team responsible for product quality.

The following twelve statements sum up the principles behind the Agile Manifesto.

1. The highest priority is to satisfy the customer through early and continuous delivery of valuable software.
   
   
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
   
   
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
   
   
4. Business people and developers must work together daily throughout the project.
   
   
5. Build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.
   
   
6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
   
   
7. Working software is the primary measure of progress.
   
   
8. Agile processes promote sustainable development. Sponsors, developers and users should be able to maintain a constant pace indefinitely.
   
   
9. Continuous attention to technical excellence and good design enhances agility.
   
   
10. Simplicity – the art of maximising the amount of work not done – is essential.
    
    
11. The best architectures, requirements and designs emerge from self-organising teams.
    
    
12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.
    

The different Agile methodologies provide practices to put these values and principles into action.

The properties of Agile

• Agile is highly creative – encouraging people working together to produce good quality software as quickly as possible.
  
  
• Agile has a high business value – giving the users something they want at the end of an iteration.
  
  
• Agile engenders teamwork – everyone is encouraged to help each other.
  
  
• Agile is quality-focused – testing is involved from day one.
  
  
• Agile uses iterations of design, build & test – this allows quicker delivery of software to the user / customer.

The value of Agile

• Agile helps to satisfy the customer – they are involved from inception to completion of the iteration.
  
  
• Agile promotes a positive response to change – being proactive rather than reactive.
  
  
• Agile methodologies deliver working software frequently.
  
  
• Agile encourages face-to-face conversation – as much as possible, but other means are used where geographic location is an issue.
  
  
• Agile promotes simplicity – a methodology framework is available which has just a few basic principles.
  
  
• Agile keeps the team motivated and together.
  
  

In February 2001, at a ski resort in Utah, a group of seventeen people representing widely-used software development methodologies (ultimately becoming the “Agile Alliance”) met to try to find common ground in development and testing. The end product was the Agile Software Development Manifesto, containing four value statements shown below:

The Agile Manifesto is stating that although the concepts on the right have value, those on the left have greater value.

Individuals and interactions

It is teams, rather than sole individuals that build software. Teams must communicate to be effective, and continuous communication and interaction, rather than a simple reliance on tools or processes, enables teams to work in the most effective way. Agile development, in particular, is a very people-centered methodology.

Customer Collaboration

Defining specifications for a product is never easy, even for those that are experienced. Customers may know what they need, but can find it difficult to express this in a way suitable for developers. By the Agile team collaborating directly with customers and users the probability that what the customer requires will be more fully understood.

Contractual relationships with customers may well be important, or even mandatory in some organisations, but closely and regularly collaborating with them increases the chance that the project will be successful.

Working software

For customers, working software is much more desirable than unnecessarily detailed documentation. Working software is useful because it gives users a chance to feedback to the development team sooner rather than later.

Even though it may have limited functionality, a working product that is available earlier in the development lifecycle can deliver a significant time-to-market advantage for the business. Because of this, Agile development is particularly suitable for business environments where requirements change rapidly, where solutions are unclear or where the business is moving into uncharted territory, such as new technology or domains in which they have no previous experience.

Responding to Change

Change always happens in software projects. Current legislation is amended or new laws introduced, business competitors come and go, and new technologies are developed. All of these, and other factors, can have repercussions for a project and may affect its objectives. These changes must be allowed for by the development methodology. This is why having flexible and adaptable work practices that are able to embrace change, is more important than just sticking fixedly to a plan.

Sequential lifecycle

In the sequential type of development lifecycle the whole product is developed in one go with development and testing activities performed sequentially and test execution at the end. Examples are the Waterfall model and V-Model, illustrated below.

Iterative lifecycle

In iterative-type lifecycles products are developed in increments or phases. Development and testing activities are repeated for each increment. There are two main types:

1. Pre-planned incremental delivery (where all requirements are known at the outset) or
   
   
2. Evolutionary delivery

An increment is defined as a self-contained functional unit and its supporting material.

Agile is a certain type of iterative development.

Most Agile methods attempt to minimise risk by developing software in short, time-boxed sprints. The Agile methodology encourages communication within the team and working software is the primary measure of progress. There are general principles established in the Agile testing community for companies to sign-up to.

Because of these, and other, differences, testers on Agile projects will operate differently to those working on more traditional projects. There are a certain set of values and principles that underpin Agile projects, which testers must understand and adhere to, and testers are an integral part of a whole-team approach which includes developers and business representatives. All members of an Agile project are expected to communicate with each other in a timely and frequent manner, which leads to early defect removal and, ultimately, a quality product. In the coming lessons we will explain more about this.

Mock TMMi Professional Certificate Examination

In the real TMMi Professional examination you have 60 minutes within which to answer 40 multiple-choice questions. This mock examination gives you up to 60 minutes to answer as many questions as you can. At the end of that time, the test will be terminated and marked automatically.

In each case there is only one correct answer. If you think there are two or more answers which could be true, select the answer which is most correct.

For the remaining lessons and topics of this section we cover the main aspects of how TMMi is applicable in Agile projects. These topics provide some practical examples of what could be expected be demonstrated by an Agile organisation undergoing a TMMi assessment. We will take each Process Area in turn.