Logo

Navigation
  • Home
  • Services
    • ERP Solutions
    • Implementation Solutions
    • Support and Maintenance Solutions
    • Custom Solutions
    • Upgrade Solutions
    • Training and Mentoring
    • Web Solutions
    • Production Support
    • Architecture Designing
    • Independent Validation and Testing Services
    • Infrastructure Management
  • Expertise
    • Microsoft Development Expertise
    • Mobile Development
    • SQL Server Database and BI
    • SAP BI, SAP Hana, SAP BO
    • Oracle and BI
    • Oracle RAC
  • Technical Training
    • Learn Data Management
      • Business Intelligence
      • Data Mining
      • Data Modeling
      • Data Warehousing
      • Disaster Recovery
    • Learn Concepts
      • Application Development
      • Client Server
      • Cloud Computing Tutorials
      • Cluster Computing
      • CRM Tutorial
      • EDI Tutorials
      • ERP Tutorials
      • NLP
      • OOPS
      • Concepts
      • SOA Tutorial
      • Supply Chain
      • Technology Trends
      • UML
      • Virtualization
      • Web 2.0
    • Learn Java
      • JavaScript Tutorial
      • JSP Tutorials
      • J2EE
    • Learn Microsoft
      • MSAS
      • ASP.NET
      • ASP.NET 2.0
      • C Sharp
      • MS Project Training
      • Silverlight
      • SQL Server 2005
      • VB.NET 2005
    • Learn Networking
      • Networking
      • Wireless
    • Learn Oracle
      • Oracle 10g
      • PL/SQL
      • Oracle 11g Tutorials
      • Oracle 9i
      • Oracle Apps
    • Learn Programming
      • Ajax Tutorial
      • C Language
      • C++ Tutorials
      • CSS Tutorial
      • CSS3 Tutorial
      • JavaScript Tutorial
      • jQuery Tutorial
      • MainFrame
      • PHP Tutorial
      • VBScript Tutorial
      • XML Tutorial
    • Learn Software Testing
      • Software Testing Types
      • SQA
      • Testing
  • Career Training
    • Career Improvement
      • Career Articles
      • Certification Articles
      • Conflict Management
      • Core Skills
      • Decision Making
      • Entrepreneurship
      • Goal Setting
      • Life Skills
      • Performance Development
      • Personal Excellence
      • Personality Development
      • Problem Solving
      • Relationship Management
      • Self Confidence
      • Self Supervision
      • Social Networking
      • Strategic Planning
      • Time Management
    • Education Help
      • Career Tracks
      • Essay Writing
      • Internship Tips
      • Online Education
      • Scholarships
      • Student Loans
    • Managerial Skills
      • Business Communication
      • Business Networking
      • Facilitator Skills
      • Managing Change
      • Marketing Management
      • Meeting Management
      • Process Management
      • Project Management
      • Project Management Life Cycle
      • Project Management Process
      • Project Risk Management
      • Relationship Management
      • Task Management
      • Team Building
      • Virtual Team Management
    • Essential Life Skills
      • Anger Management
      • Anxiety Management
      • Attitude Development
      • Coaching and Mentoring
      • Emotional Intelligence
      • Stress Management
      • Positive Thinking
    • Communication Skills
      • Conversation Skills
      • Cross Culture Competence
      • English Vocabulary
      • Listening Skills
      • Public Speaking Skills
      • Questioning Skills
    • Soft Skills
      • Assertive Skills
      • Influence Skills
      • Leadership Skills
      • Memory Skills
      • People Skills
      • Presentation Skills
    • Finding a Job
      • Etiquette Tips
      • Group Discussions
      • HR Interviews
      • Interview Notes
      • Job Search Tips
      • Resume Tips
      • Sample Resumes
 

Metrics Used In Testing

By Exforsys | on December 5, 2005 |
Testing

Metrics Used In Testing

In this tutorial you will learn about metrics used in testing, The Product Quality Measures – 1. Customer satisfaction index, 2. Delivered defect quantities, 3. Responsiveness (turnaround time) to users, 4. Product volatility, 5. Defect ratios, 6. Defect removal efficiency, 7. Complexity of delivered product, 8. Test coverage, 9. Cost of defects, 10. Costs of quality activities, 11. Re-work, 12. Reliability and Metrics for Evaluating Application System Testing.

The Product Quality Measures:

1. Customer satisfaction index

This index is surveyed before product delivery and after product delivery
(and on-going on a periodic basis, using standard questionnaires).The following are analyzed:

  • Number of system enhancement requests per year
  • Number of maintenance fix requests per year
  • User friendliness: call volume to customer service hotline
  • User friendliness: training time per new user
  • Number of product recalls or fix releases (software vendors)
  • Number of production re-runs (in-house information systems groups)

2. Delivered defect quantities

They are normalized per function point (or per LOC) at product delivery (first 3 months or first year of operation) or Ongoing (per year of operation) by level of severity, by category or cause, e.g.: requirements defect, design defect, code defect, documentation/on-line help defect, defect introduced by fixes, etc.

3. Responsiveness (turnaround time) to users

  • Turnaround time for defect fixes, by level of severity
  • Time for minor vs. major enhancements; actual vs. planned elapsed time

4. Product volatility

  • Ratio of maintenance fixes (to repair the system & bring it into compliance with specifications), vs. enhancement requests (requests by users to enhance or change functionality)

5. Defect ratios

  • Defects found after product delivery per function point.
  • Defects found after product delivery per LOC
  • Pre-delivery defects: annual post-delivery defects
  • Defects per function point of the system modifications

6. Defect removal efficiency

  • Number of post-release defects (found by clients in field operation), categorized by level of severity
  • Ratio of defects found internally prior to release (via inspections and testing), as a percentage of all defects
  • All defects include defects found internally plus externally (by customers) in the first year after product delivery

7. Complexity of delivered product

  • McCabe’s cyclomatic complexity counts across the system
  • Halstead’s measure
  • Card’s design complexity measures
  • Predicted defects and maintenance costs, based on complexity measures

8. Test coverage

  • Breadth of functional coverage
  • Percentage of paths, branches or conditions that were actually tested
  • Percentage by criticality level: perceived level of risk of paths
  • The ratio of the number of detected faults to the number of predicted faults.

9. Cost of defects

  • Business losses per defect that occurs during operation
  • Business interruption costs; costs of work-arounds
  • Lost sales and lost goodwill
  • Litigation costs resulting from defects
  • Annual maintenance cost (per function point)
  • Annual operating cost (per function point)
  • Measurable damage to your boss’s career

10. Costs of quality activities

  • Costs of reviews, inspections and preventive measures
  • Costs of test planning and preparation
  • Costs of test execution, defect tracking, version and change control
  • Costs of diagnostics, debugging and fixing
  • Costs of tools and tool support
  • Costs of test case library maintenance
  • Costs of testing & QA education associated with the product
  • Costs of monitoring and oversight by the QA organization (if separate from the development and test organizations)

11. Re-work

  • Re-work effort (hours, as a percentage of the original coding hours)
  • Re-worked LOC (source lines of code, as a percentage of the total delivered LOC)
  • Re-worked software components (as a percentage of the total delivered components)

12. Reliability

  • Availability (percentage of time a system is available, versus the time the system is needed to be available)
  • Mean time between failure (MTBF).
  • Man time to repair (MTTR)
  • Reliability ratio (MTBF / MTTR)
  • Number of product recalls or fix releases
  • Number of production re-runs as a ratio of production runs

Metrics for Evaluating Application System Testing:

Metric = Formula

Test Coverage = Number of units (KLOC/FP) tested / total size of the system. (LOC represents Lines of Code)

Number of tests per unit size = Number of test cases per KLOC/FP (LOC represents Lines of Code).

Acceptance criteria tested = Acceptance criteria tested / total acceptance criteria

Defects per size = Defects detected / system size

Test cost (in %) = Cost of testing / total cost *100

Cost to locate defect = Cost of testing / the number of defects located

Achieving Budget = Actual cost of testing / Budgeted cost of testing

Defects detected in testing = Defects detected in testing / total system defects

Defects detected in production = Defects detected in production/system size

Quality of Testing = No of defects found during Testing/(No of defects found during testing + No of acceptance defects found after delivery) *100

Effectiveness of testing to business = Loss due to problems / total resources processed by the system.

System complaints = Number of third party complaints / number of transactions processed

Scale of Ten = Assessment of testing by giving rating in scale of 1 to 10

Source Code Analysis = Number of source code statements changed / total number of tests.

Effort Productivity = Test Planning Productivity = No of Test cases designed / Actual Effort for Design and Documentation

Test Execution Productivity = No of Test cycles executed / Actual Effort for testing

« « Syllabus for Certified Software Tester
J2EE Overview » »

Author Description

Avatar

Editorial Team at Exforsys is a team of IT Consulting and Training team led by Chandra Vennapoosa.

Free Training

RSSSubscribe 394 Followers
  • Popular
  • Recent
  • SQL Unit Testing Tools

    March 18, 2005 - 0 Comment
  • Challenges in Testing Web Based Applications

    June 7, 2006 - 0 Comment
  • Testing : Introduction to CMM

    May 22, 2005 - 0 Comment
  • Software Project Planning

    December 4, 2005 - 0 Comment
  • QuickTest Professional 8.0 CBT Tutorial and Evaluation Copy

    April 12, 2005 - 0 Comment
  • Software Testing Myths

    November 10, 2006 - 0 Comment
  • Why Choose LoadRunner for Performance testing

    May 25, 2005 - 0 Comment
  • Software Quality Management

    December 5, 2005 - 0 Comment
  • SDLC : Software Development Life Cycle

    April 22, 2005 - 0 Comment
  • Bug Reporting – Art and Advocacy

    January 11, 2008 - 0 Comment
  • Software Testing Best Practices

    July 20, 2009 - 0 Comment
  • Bug Reporting – Art and Advocacy

    January 11, 2008 - 0 Comment
  • Software Testing Myths

    November 10, 2006 - 0 Comment
  • Challenges in Testing Web Based Applications

    June 7, 2006 - 0 Comment
  • Auditing Software Testing Process

    June 5, 2006 - 0 Comment
  • Testing for Agile Software Development

    June 4, 2006 - 0 Comment
  • What is User Acceptance Testing?

    January 27, 2006 - 0 Comment
  • System Testing: Why? What? & How?

    January 22, 2006 - 0 Comment
  • What is Regression Testing?

    January 22, 2006 - 0 Comment
  • Integration Testing: Why? What? & How?

    January 22, 2006 - 0 Comment

Exforsys e-Newsletter

ebook
 

Related Articles

  • Software Testing Best Practices
  • Bug Reporting – Art and Advocacy
  • Software Testing Myths
  • Challenges in Testing Web Based Applications
  • Auditing Software Testing Process

Latest Articles

  • Project Management Techniques
  • Product Development Best Practices
  • Importance of Quality Data Management
  • How to Maximize Quality Assurance
  • Utilizing Effective Quality Assurance Strategies
  • Sitemap
  • Privacy Policy
  • DMCA
  • Trademark Information
  • Contact Us
© 2023. All Rights Reserved.IT Training and Consulting
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.AcceptReject Read More
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT