SWEN90016
Software Processes & Project Management
Quality Management
School of Computing and Information Systems
The University of Melbourne
andrew.valentine@unimelb.edu.au
2022 – Semester 2
Week 8
Copyright University of Melbourne 2018-2022
SWEN90016 Software Processes and Project Management Quality Management
Some References Used
[1] R. S. Pressman. Software Engineering: A Practitioner’s Approach. McGraw Hill, seventh
edition, 2009.
• Chapter on Quality Management.
[2] I Somerville. Software Engineering, Addison-Wesley Publishing, tenth edition, 2016.
– Chapter 24: Quality Management.
[3] Schwalbe, K., 2016. Information technology project management. Australia: Cengage
Learning.
• Chapter 8: Quality Management.
• You can access this online as an e-book through the University of Melbourne library.
[4] Marco Palomino, Abraham Dávil, Karin Melendez, Marcelo Pessoa. Agile Practices Adoption
in CMMI Organizations: A Systematic Literature Review. International Conference on Software
Process Improvement, 2016.
-2-
SWEN90016 Software Processes and Project Management Quality Management
Learning Outcomes
Quality Management
1. Understand the fundamentals of quality management
2. Understand the quality management process
3. Understand the following quality management activities:
– Quality Assurance
– Quality Planning
– Quality Control and Monitoring
-3-
SWEN90016 Software Processes and Project Management Quality Management
Project Quality
-4-
Today’s lecture
SWEN90016 Software Processes and Project Management Quality Management-5-
SWEN90016 Software Processes and Project Management Quality Management
Learning Outcomes
1. Understand the fundamentals of quality management
2. Understand the quality management process
3. Understand the following quality management activities:
– Quality Assurance
– Quality Planning
– Quality Control and Monitoring
-6-
SWEN90016 Software Processes and Project Management Quality Management
What is quality?
Quality is not an act, it is a habit — Aristotle
• Evidence shows that we cannot simply fix up our software
post-hoc and add in quality attributes after building the system
• Quality must be built into the software from the beginning
• In this topic you will learn how to built quality into the software
through a range of Quality Management activities
-7-
SWEN90016 Software Processes and Project Management Quality Management
What is software quality?
• We define quality from two broad perspectives:
– End-user’s Perspective:
Typically, end-users judge the quality of a product by their interaction with it.
For users, a system has quality if it is fit for purpose, is reliable, has
reasonable performance, is easy to learn and use, and helps the users in
achieving their goals. Sometimes, if the functionality is hard to learn but is
extremely important and worth the trouble of learning, then users will still
judge the system to have high quality. These are termed external quality
characteristics, because they are typically associated with the external
behaviour of the system.
– Developer’s Perspective:
The developer’s perspective typically also includes the number of faults that
the system has, ease of modifying the system, ease of testing the system, the
ease of understanding the system design, the re-usability of components,
conformance to requirements, resource usage, and performance. These are
mainly internal quality characteristics, because they are concerned with the
quality of the internal structure of the system.
-8-
SWEN90016 Software Processes and Project Management Quality Management
Cost of quality
• Some claim:
Most quality assurance activities are too costly - savings made
from not using resources is greater than the cost incurred in fixing
the faults
• For example, instead of performing formal reviews of requirements
specification documents, it is far better to build the system, ask the
client/user for feedback, and to correct any faults from there.
• Alternatively, one can simply release the system and correct faults
as users report them.
• Empirical studies refute the above claim:
– There are many studies in the area
-9-
SWEN90016 Software Processes and Project Management Quality Management
Cost of quality
-10-
SWEN90016 Software Processes and Project Management Quality Management
Learning Outcomes
1. Understand the fundamentals of quality management
2. Understand the quality management process
3. Understand the following quality management activities:
– Quality Assurance
– Quality Planning
– Quality Control and Monitoring
-11-
SWEN90016 Software Processes and Project Management Quality Management
Learning Outcomes
1. Understand the fundamentals of quality management
2. Understand the quality management process
3. Understand the following quality management activities:
– Quality Assurance
• Verification and Validation
• Standards
• Capability Maturity Model
– Quality Planning
– Quality Control and Monitoring
-12-
SWEN90016 Software Processes and Project Management Quality Management
Quality Management Process
Quality assurance:
The establishment of a framework of organizational procedures and
standards that lead to high-quality software
Quality planning:
The selection of appropriate procedures and standards from the
framework, adopted for the specific project
Quality control:
Ensuring that the software development team has followed the project
quality procedures and standards
-13-
SWEN90016 Software Processes and Project Management Quality Management
Verification vs Validation
• Verification and Validation (V &V) are important aspects of
quality assurance
• Verification:
– Verification is an attempt to ensure that the product is built correctly, in the
sense that the output products of an activity meet the specifications imposed on
them in previous activities.
– Verification normally involves two (sets of) artifacts: req. spec. vs design,
design vs code; this is an internal developer activity.
– Verification is ensuring you are building the system right (the right way).
• Validation:
– Validation is an attempt to ensure that the right product is built—that is, the
product fulfils it specific intended purpose.
– Validation involves going back to the stakeholders to check if the product meets
their requirements; this normally involves something/someone external.
– Validation is ensuring that you are building the right system (to meet
stakeholder needs).
-14-
SWEN90016 Software Processes and Project Management Quality Management
Verification and validation
-15-
[1] p421
SWEN90016 Software Processes and Project Management Quality Management
Quality Assurance
-16-
SWEN90016 Software Processes and Project Management Quality Management
Quality Assurance
• Quality assurance process is primarily concerned with defining
or selecting the quality standards
– A standard might simply be defined as a set of rules for ensuring
quality
– Standards play an important role in the quality management
process
• There are two types of standards:
– Product standards:
• These apply to the product being developed
– Process standards:
• These standards define the processes that should be followed
during software development
-17-
SWEN90016 Software Processes and Project Management Quality Management
Product vs Process Standards
-18-
Product Standards Process Standards
Design review form template Design review conduct
Requirements document structure Design validation process
Documentation standards Version release process
Coding standards to follow Project plan approval process
Project plan format Change control process
Change request from template Test recording process
Product vs process standards
SWEN90016 Software Processes and Project Management Quality Management
Documentation standards
• Why are documentation standards important?
– documents are the tangible manifestation of the software
• Documentation process standards
– How documents should be developed, validated and maintained
• Document standards
– Concerned with document identification, structure, presentation,
changes highlighting, etc.
• Document interchange standards
– How documents are stored and interchanged between different
documentation systems
– XML is an emerging standard for document interchange which will be
widely supported in future
SWEN90016 Software Processes and Project Management Quality Management
Software Standards and Systems
• Advantages of standards
– Provide a framework around which the quality assurance process may
be implemented
– Provide encapsulation of best, or at least most appropriate, practice
– Customers sometimes require a particular quality standard/level when
choosing a software vendor
• Problems with standards
– Not seen as relevant and up-to-date by software engineers
– Involve too much bureaucratic form filling
– Unsupported by software tools so tedious manual work is involved to
maintain standards
Standards should not be avoided, but should be tailored as
needed!
-20-
SWEN90016 Software Processes and Project Management Quality Management
Software Standards and Systems
• Many standards and systems related to software quality exists today
• Some examples of software standards and systems
– ISO 9000 family Quality management
• ISO 9000:2015 Quality management systems
• ISO 9001:2015 Quality management systems
• ISO 9004:2018 Quality management
– Capability Maturity Model
• Described in the following slides.
-21-
SWEN90016 Software Processes and Project Management Quality Management
Capability Maturity Model
• Developed by the Software Engineering Institute (SEI) at
Carnegie Mellon University
• Describes the key elements of an effective software
development process
• Describes an approach for software companies to move from
an ad-hoc, immature process to a mature developed process
• Organizations are characterised being at a Level from 1-5
based on the processes they follow
-22-
SWEN90016 Software Processes and Project Management Quality Management
Capability Maturity Model
-23-
SWEN90016 Software Processes and Project Management Quality Management
Process change management
Technology change management
Defect prevention
Software quality management
Quantitative process management
Peer reviews
Intergroup coordination
Software product engineering
Integrated software management
Training programme
Organization process definition
Organization process focus
Software configuration management
Software quality assurance
Software subcontract management
Software project tracking and oversight
Software project planning
Requirements management
Initial
Repeatable
Defined
Managed
Optimizing
Capability Maturity Model
SWEN90016 Software Processes and Project Management Quality Management
Learning Outcomes
1. Understand the fundamentals of quality management
2. Understand the quality management process
3. Understand the following quality management activities:
– Quality Assurance
– Quality Planning
• Software Quality Plan
• Software Quality Attributes
– Quality Control and Monitoring
-25-
SWEN90016 Software Processes and Project Management Quality Management
Quality Management Process
Quality assurance:
The establishment of a framework of organizational procedures and
standards that lead to high-quality software
Quality planning:
The selection of appropriate procedures and standards from the
framework, adopted for the specific project
Quality control:
Ensuring that the software development team has followed the project
quality procedures and standards
-26-
SWEN90016 Software Processes and Project Management Quality Management
Quality Planning
• The process of selecting those standards and systems that
are appropriate to a particular organization and project
• The outcome of the planning process is a:
– Software Quality Plan (SQP), sometimes called a Software
Quality Assurance Plan (SQAP)
-27-
SWEN90016 Software Processes and Project Management Quality Management
SQP - Template
• Software Quality Assurance Plan
– Product Overview
A description of the product, intended market, and quality expectations
– Product Plan
The critical release dates and responsibilities – could point to the schedule
– Quality Goals
The quality goals and plans for the product, including identification and
justification of critical product quality attributes
– Process Description
The quality assurance processes that should be used for product development
and management (reviews, audits etc)
– Document and Coding Standards
Standards for the documents and coding standards
– Risks and Risk Management
The key risks that might affect product quality and the actions to address these
risks (could provide a link to appropriate risks in the Risk Management Plan)
-28-
SWEN90016 Software Processes and Project Management Quality Management-29-
SWEN90016 Software Processes and Project Management Quality Management
Software Quality Attributes
-30-
Safety Understandability Portability
Security Testability Usability
Reliability Adaptability Reusability
Resilience Modularity Efficiency
Robustness Complexity Learnability
Software quality attributes
• Some of the quality attributes matter only matter to developers
while others matter to end-users
• It is not possible for any system to be optimised for all
attributes – trade-off is necessary to select the most important
ones
SWEN90016 Software Processes and Project Management Quality Management
Learning Outcomes
1. Understand the fundamentals of quality management
2. Understand the quality management process
3. Understand the following quality management activities:
– Quality Assurance
– Quality Planning
– Quality Control and Monitoring
• Technical Reviews
• Business Reviews
• Management Reviews
-31-
SWEN90016 Software Processes and Project Management Quality Management
Quality Control and Monitoring
• Involves monitoring the software development process to
ensure that the quality assurance procedures and standards
specified in the SQP are being followed
-32-
Input Process
Quality
Control
Output
SWEN90016 Software Processes and Project Management Quality Management
Reviews
• Review is a common technique used for verification and
validation
• Artefacts produced during the development process are
reviewed as a way of identifying problems seeking ways to
improve them early
• Three common types of reviews:
– Technical Reviews
– Business Reviews
– Management Reviews
-33-
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
• Reviews of artefacts is performed by peers in the
development team but the author/s are involved
• The aim is uncovering problems in an artefact and seeking
ways to improve the artefact
• Is considered a “soft” method for quality assurance - that is,
nothing is executed
– Some developers greet reviews with scepticism - however,
empirical evidence suggests that such scepticism is unjustified
-34-
SWEN90016 Software Processes and Project Management Quality Management-35-
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
• Advantages of technical reviews:
– Can be performed on any software artefact, whereas many “hard” methods
of quality assurance, such as testing and measurement, can be performed only
on executable artefacts.
– Earlier detection of problems in software artefacts leads to lower costs of
resolution.
– Studies show that roughly 30-70% of all programming faults found in a project
were located using source code reviews, and up to 80% according to studies
performed by IBM. Some studies demonstrated that review techniques found
several types of faults that testing failed to find, and vice-versa.
– Reviews find the actual faults in source code, in contrast to testing, which
merely indicates that there is a fault somewhere in the program. After a fault is
detected with testing, it must then be located.
– Due to internal pressure of getting software releases out the door, programmers
make more mistakes when correcting faults that were found during testing than
they do correcting faults during the review phase
-36-
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
• Disadvantages of technical reviews:
– Could be time and resource consuming
– Should be carefully planned and executed to get the desired outcomes
• Types of technical reviews
– Informal Reviews
– Formal Reviews
– Walk throughs
– Code inspections
– Audits
-37-
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
• Informal Reviews:
– A simple desk check or casual meeting with a colleague which aims to
improve the quality of a document
– No formal guidelines or procedures that are followed
– The effectiveness of informal reviews is considerably less than formal
reviews, because of the lack of diversity found in a group
– Checklists are tools that can help to improve the effectiveness of a
review.
– A checklist is a list of questions that the reviewer must answer about
an artefact, however, the questions are generic questions about that
type of artefact
– Less time and resource consuming than a formal review
-38-
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
-39-
Example checklist
for a Requirements
Specification
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
-40-
• Formal Reviews:
– A meeting with multiple stakeholders such as
developers, testers, client
• The group approach has benefits of bringing out different perspectives
– Meeting should adhere to the following constraints
• The review team should be 3-5 members carefully chosen
• The meeting should last no longer than 90 minutes
• Following are the critical roles
– Review Leader: responsible for organizing the review
– Author: at least one author should be present
– Reviewers: at least two or three non-author stakeholders
– Recorder: responsible for recording all important review comments
– The review meeting could recommend one of the following:
• Accept without further changes
• Accept with proposed changes
• Reject the artefact – this requires a re-review after modifications
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
-41-
• Walkthroughs
– Walkthrough could be for code or a document
– This is a review process where the author (the programmer or designer)
leads a group of reviewers
– Following are the main differences from a formal review:
• Moderator, that leads the review is the author of the artefact being reviewed
• Reviewers do not need preparation
• When defects or inconsistencies are found, possible solutions are
discussed
• Code Inspections
– These are very similar to formal reviews, expect that the focus is on the
code
SWEN90016 Software Processes and Project Management Quality Management
Technical Reviews
• Audits
– Reviews of processes and products to determine if a particular product
or process conforms to standards
– It is a type of technical review where the authors of the artefact being
audited are not involved in the audit process at all – all the other roles
are similar to a formal review
– Audits are typically performed by a team that is completely external to
an organisation
– Two types of audits:
• Product audits: to confirm that the product meets the standards
• Process audits: to ensure that the team follows processes
-42-
SWEN90016 Software Processes and Project Management Quality Management
Business Reviews
• The goal of a business review is to ensure that the IT solution
provides the functionality specified in the project scope and
requirements document
• A business review can include all project deliverables to
ensure that:
– It is complete
– Provides the information needed to move to the next phase or
process
– Meets the standards
-43-
SWEN90016 Software Processes and Project Management Quality Management
Management Reviews
• Compares the project’s actual progress against a baseline
project plan
• Project Manager is responsible for presenting the project
progress and providing a clear picture of the current status
• Issues need to be resolved – e.g. resources reallocated as
needed, change to the project course if needed
• May involve reviewing if the project meets the scope,
schedule, budget and quality objectives
-44-
SWEN90016 Software Processes and Project Management Quality Management
Learning Outcomes
1. Understand the fundamentals of quality management
2. Understand the quality management process
3. Understand the following quality management activities:
– Quality Assurance
– Quality Planning
– Quality Control and Monitoring
-45-