Question

Quality goals:

For (any) software development project:

What are the quality goals?

Identify the metrics to be used to measure achievement of the goals.

Define how the metrics will be gathered for the project.

Define the major test phases for the SD project.

Identify test strategy that will be used for each test phase. Identify resources required for the test phases.

Determine how defects will be documented and processed

Answer 1

Software Development Project

**) Quality goals :

• Refer to developed software system’s substantive quality requirements.
• Preferably stated in the form of quantitative as opposed to qualitative measures.
• Examples:- 1. A software system to serve the help desk operations of an electrical appliance manufacture is to be developed. The help desk system (HDS) is intended to operate for 100 hours per week. 2. The software quality assurance team was requested to prepare a list of Quantitative Quality Goals appropriate to certain qualitative requirements.

HDS Qualitative Requirements	Related quantitative Quality goals
The HDS should be user friendly	A new help desk operator should be able to learn the details of the HDS following a course lasting less than 8 hours and to master operation of HDS in less than 5 working days
The HDS should be very reliable	HDS availability should exceed 99.5% (HDS downtime should not exceed 30 min per week)
The HDS should operate continuously	The system’s recovery time should not exceed 10 min in 99% of cases of HDS failure
The HDS should be highly efficient	An HDS operator should be handle at least 100 customer calls per 8-hour shift
The HDS should provide high quality service to the applying customer	Waiting time for operator response should no exceed 30 sec in 99% of the calls

**) Software Metrics to be Used to measure the achievement of Goals:

• A software metric is a measure of software characteristics which are quantifiable or countable. Software metrics are important for many reasons, including:- measuring software performance (User Satisfaction/performance Index, Average Response Time ,Error rate, Count of Application Instances ,Request Rate, Application & Server CPU, Application availability, Garbage Collection.), planning work items, measuring productivity, and many other uses.
• Within the software development process, there are many metrics that are all related to each other. Software metrics are related to the four functions of management: Planning, Organization, Control, or Improvement.
• The goal of tracking and analyzing software metrics is to determine the quality of the current product or process, improve that quality and predict the quality once the software development project is complete. On a more granular level, software development managers are trying to: Increase return on investment (ROI), Identify areas of improvement, Manage workloads, Reduce overtime, Reduce costs.
• Software development teams can use software metrics to communicate the status of software development projects, pinpoint and address issues, and monitor, improve on, and better manage their workflow.
• Software metrics offer an assessment of the impact of decisions made during software development projects. This helps managers assess and prioritize objectives and performance goals.

**) Define how metric will be gathered for project:

• Software metrics are great for management teams because they offer a quick way to track software development, set goals and measure performance.
• Software metrics should have several important characteristics. They should be:

1. Simple & Computable
2. Consistent and unambiguous (objective)
3. Use consistent units of measurement
4. Independent of programming languages
5. Easy to calibrate and adaptable
6. Easy and cost-effective to obtain
7. Able to be validated for accuracy and reliability
8. Relevant to the development of high-quality software products

• Four Topics for use Software Metric will be Gathered for Project:
• Link software metrics to goals.

1. Link software metrics to goal:

Often sets of software metrics are communicated to software development teams as goals. So the focus becomes:

• Reducing the lines of codes
• Reducing the number of bugs reported
• Increasing the number of software iterations
• Speeding up the completion of tasks

Focusing on those metrics as targets help software developers reach more important goals such as improving software usefulness and user experience.

2. Track Trends,not numbers :

Software metrics are very seductive to management because complex processes are represented as simple numbers. And those numbers are easy to compare to other numbers. So when a software metric target is met, it is easy to declare success. Not reaching that number lets software development teams know they need to work more on reaching that target.

   Analysis of why the trend line is moving in a certain direction or at what rate it is moving will say more about the process. Trends also will show what effect any process changes have on progress. a trend line showing progress toward a target offers incentive and insight into how to reach that target.

3. Set shorter measurement periods:

    Software development teams want to spend their time getting the work done not checking if they are reaching management established targets. So a hands-off approach might be to set the target sometime in the future and not bother the software team until it is time to tell them they succeeded or failed to reach the target.

    By breaking the measurement periods into smaller time frames, the software development team can check the software metrics and the trend line to determine how well they are progressing. The shorter periods of measurement offer more data points that can be useful in reaching goals, not just software metric targets.

4. Stop using software metrics that do not lead to change:

      We all know that the process of repeating actions without change with the expectation of different results is the definition of insanity. But repeating the same work without adjustments that do not achieve goals is the definition of managing by metrics.

       Some software metrics have no value when it comes to indicating software quality or team workflow. Management and software development teams need to work on software metrics that drive progress towards goals and provide verifiable, consistent indicators of progress.

**) Major Test Phrase for SD project:

  It consists of a series of activities carried out methodologically to help certify your software product.

Requirement Analysis

Test Planning

Test Case Development

Environment Setup

Test Execution

Test Cycle Closure

1. Requirement Analysis

During this phase, test team studies the requirements from a testing point of view to identify the testable requirements. Requirements could be either Functional (defining what the software must do) or Non Functional (defining system performance /security availability )

Activities

• Identify types of tests to be performed.
• Gather details about testing priorities and focus.
• Prepare Requirement Trace ability Matrix (RTM).
• Identify test environment details where testing is supposed to be carried out.
• Automation feasibility analysis (if required).

Deliverables

• RTM
• Automation feasibility report.

2.Test Planning

    Typically, in this stage, a Senior QA manager will determine effort and cost estimates for the project and would prepare and finalize the Test Plan. In this phase, Test Strategy is also determined.

Activities

• Preparation of test plan/strategy document for various types of testing
• Test tool selection
• Test effort estimation
• Resource planning and determining roles and responsibilities.
• Training requirement

Deliverables

• Test plan /strategy document.
• Effort estimation document.

3. Test Case Development

     This phase involves the creation, verification and rework of test cases & test scripts. Test data, is identified/created and is reviewed and then reworked as well.

Activities

• Create test cases, automation scripts
• Review and baseline test cases and scripts
• Create test data (If Test Environment is available)

Deliverables

• Test cases/scripts
• Test data

4.Test Environment Setup

         Test environment decides the software and hardware conditions under which a work product is tested. Test environment set-up is one of the critical aspects of testing process and can be done in parallel with Test Case Development Stage. Test team may not be involved in this activity if the customer/development team provides the test environment in which case the test team is required to do a readiness check (smoke testing) of the given environment.

Activities

• Understand the required architecture, environment set-up and prepare hardware and software requirement list for the Test Environment.
• Setup test Environment and test data
• Perform smoke test on the build

Deliverables

• Environment ready with test data set up
• Smoke Test Results.

5.Test Execution

During this phase, the testers will carry out the testing based on the test plans and the test cases prepared. Bugs will be reported back to the development team for correction and retesting will be performed.

Activities

• Execute tests as per plan
• Document test results, and log defects for failed cases
• Map defects to test cases in RTM
• Retest the Defect fixes
• Track the defects to closure

Deliverables

• Completed RTM with the execution status
• Test cases updated with results
• Defect reports

6.Test Cycle Closure

Testing team will meet, discuss and analyze testing artifacts to identify strategies that have to be implemented in the future, taking lessons from the current test cycle. The idea is to remove the process bottlenecks for future test cycles and share best practices for any similar projects in the future.

Activities

• Evaluate cycle completion criteria based on Time, Test coverage, Cost,Software, Critical Business Objectives, Quality
• Prepare test metrics based on the above parameters.
• Document the learning out of the project
• Prepare Test closure report
• Qualitative and quantitative reporting of quality of the work product to the customer.
• Test result analysis to find out the defect distribution by type and severity.

Deliverables

• Test Closure report
• Test metrics

**) Test Strategies that will be used for Test Phrases:

Unit testing
Unit testing starts at the center and each unit is implemented in source code.

Integration testing
An integration testing focuses on the construction and design of the software.

Validation testing
Check all the requirements like functional, behavioral and performance requirement are validate against the construction software.

System testing
System testing confirms all system elements and performance are tested entirely.

**) Determine how defects will be documented and processed:

Quality assurance activities are not just targeted at defect identification (which is finding defects after they have happened. This simply is testing or Quality control) but also include defect prevention (making sure the defects do not happen in the first place or the defects are removed/reduced before making their way into the software product).

A simple equation equivalent can be:

QA= QC (defect identification) + Defect prevention

The truth of the matter is, defects found during the testing phase or worse after release are costlier to find & fix and might cause a loss of trust on the brand. Hence, the earlier the prevention measures are taken, the better. Besides, defect prevention also helps companies achieve the highest CMMI (Capability Maturity Model Integration) Level.

Defect Prevention

Defect Prevention is a crucial step or activity in any software development process

the defect prevention responsibilities for testers in each of the below stages:

#1) Requirement Specification Review:

After understanding customer’s requirements prepare your requirement’s gist.

A review is important at this step- the First level of review should be within the team, followed by another level of external review (by a development or BA or client) to make sure that all the perspectives are in sync.

#2) Design Review:

Design stage can be considered a strategy stage of sorts and going through it will ensure that the QA team understands the pros and cons of each strategy.

This kind of critical walkthrough will help unearth any problems with the said strategies and fix them before going further.This can be considered a feasibility study for the strategy (or strategies).

#3) Code Review:

There is not a lot for testers to directly get involved in this phase, but the review does go on here too. Developers carry out code inspections, walkthroughs and reviews before they unit and integration test the application.

Defect Prevention Methods and Techniques

Some traditional and common methods that have been in use since a long time for defect prevention are listed below;

1) Review and Inspection: This method includes the review by an individual team member (self-checking), peer reviews and inspection of all work products.

2) Walkthrough: This is more or less like a review but it’s mostly related to comparing the system to the prototype which will give a better idea regarding the correctness and/or the look-and-feel of the system.

3) Defect Logging and Documentation: This method provides some key information, arguments/parameters that can be used to support analyzing defects.

4) Root Cause Analysis: Root cause analysis includes two major approaches:

I) Pareto Analysis

II) Fishbone Analysis