SPiCE und Test - Systematic Testing

SPiCE und Test:
Was hat das denn miteinander zu tun?
TAV Düsseldorf 15./16.2.2007
Arbeitskreis „Test eingebetteter Systeme“
Dr. Uwe Hehn
[email protected]
Gliederung
Reifegradmodelle
Übersicht über SPICE
SPICE und Test
Abschließende Bemerkungen
© 2007 method park 2
Grundidee der Reifegradmodelle
Reifegradmodelle betrachten den gesamten
Softwareentwicklungsprozess als Netz von Prozessen
Diese Prozesse bestehen aus einer Menge von Aktivitäten,
die eingehende Arbeitsprodukte
in ausgehende Arbeitsprodukte
transformieren
Beispiel
Der Prozess Anforderungsermittlung
transformiert
Kundenwünsche in eine Anforderungsspezifikation
© 2007 method park 3
Grundidee der Reifegradmodelle
Ziele
Prozess als Gruppe
von Aktivitäten, die
Eingangsdaten in
Ausgangsdaten
transformieren.
Aktivität
Eingangsdaten
Aktivität
Aktivität
Ausgangsdaten
Aktivität
Aktivität
Ressourcen
Infrastruktur
© 2007 method park 4
Grundidee der Reifegradmodelle
In einem Reifegradmodell wird festgelegt,
wie die Reife eines Prozesses bzw. einer Menge von Prozessen
zu bewerten ist
Aktuelle Reifegradmodelle für Software- oder Systementwicklung:
SPICE sowie CMMI
Ein aktuelles Reifegradmodell für Testprozesse: TPI
r
Hie
© 2007 method park 5
Grundidee der Reifegradmodelle
Assessment
Audit
Improvement
© 2007 method park 6
Gliederung
Reifegradmodelle
Übersicht über SPICE
SPICE und Test
Abschließende Bemerkungen
© 2007 method park 7
Was ist eigentlich SPICE?
Software Process Improvement
and Capability dEtermination
SPICE ist ein Projekt der ISO zur Entwicklung eines
Standards für Software Process Assessments
1992
ISO 15504
Projektstart
1994
/1995
ISO 15504
Draft
1998
/1999
ISO 15504
TR
Bis vor kurzer Zeit: Technical Report von 1998/99
(ISO TR 15504:1998)
2005
Veröffentl.
ISO 15504
Ende 2005: International Standard
(ISO 15504)
© 2007 method park 8
Measurement Framework
Process Reference/Assessment Model
Exemplar PAM
Automotive SPICE PAM
ISO 15504
= „Measurement Framework“
Process Reference Model
(PRM)
Process Assessment Model
(PAM)
Examplar PRM / Automotive SPICE
PRM
© 2007 method park 9
Prozessdimension (ISO 15504)
„Exemplar-PRM“ (ISO 12207)
Process Category
Process Group
Process
Primary Life Cycle Processes
Acquisition
Acquisition Process
Process Group
Group (ACQ)
(ACQ)
ACQ.1
ACQ.1 Acquisition
Acquisition preparation
preparation
ACQ.2
Supplier
ACQ.2 Supplierselection
selection
ACQ.3
ACQ.3 Contract
Contract agreement
agreement
ACQ.4
Supplier
monitoring
ACQ.4 Supplier monitoring
ACQ.5
ACQ.5 Customer
Customeracceptance
acceptance
Supply
Supply Process
Process Group
Group (SPL)
(SPL)
SPL.1
SPL.1 Supplier
Suppliertendering
tendering
SPL.2
SPL.2 Product
Product release
release
SPL.3
SPL.3 Product
Productacceptance
acceptance support
support
Engineering
Engineering Process
Process Group
Group (ENG)
(ENG)
ENG.1
ENG.1 Requirements
Requirementselicitation
elicitation
ENG.2
System
ENG.2 System requirements
requirementsanalysis
analysis
ENG.3
ENG.3 System
System architectural
architecturaldesign
design
ENG.4
ENG.4 Software
Software requirements
requirementsanalysis
analysis
ENG.5
Software
design
ENG.5 Software design
ENG.6
ENG.6 Software
Software construction
construction
ENG.7
ENG.7 Software
Software integration
integration
ENG.8
Software
testing
ENG.8 Software testing
ENG.9
ENG.9 System
System integration
integration
ENG.10
ENG.10 System
System testing
testing
ENG.11
ENG.11 Software
Software installation
installation
ENG.12
Software
and
ENG.12 Software andsystem
system maintenance
maintenance
Operation
Operation Process
Process Group
Group (OPE)
(OPE)
OPE.1
OPE.1 Operational
Operationaluse
use
OPE.2
Customer
OPE.2 Customersupport
support
Supporting Life Cycle Processes
Support
Support Process
Process Group
Group (SUP)
(SUP)
SUP.1 Quality assurance
SUP.2 Verification
SUP.3 Validation
SUP.4 Joint review
SUP.5 Audit
SUP.6 Product evaluation
SUP.7 Documentation
SUP.8 Configuration mgmt
SUP.9 Problem resolution mgmt
SUP.10 Change request mgmt
Organisational Life Cycle Processes
Management
Management Process
Process Group
Group (MAN)
(MAN)
MAN.1
MAN.1 Organizational
Organizationalalignment
alignment
MAN.2
Organizational
management
MAN.2 Organizational management
MAN.3
MAN.3 Project
Projectmanagement
management
MAN.4
Quality
MAN.4 Qualitymanagement
management
MAN.5
MAN.5 Risk
Riskmanagement
management
MAN.6
Measurement
MAN.6 Measurement
Process
Process Improvement
Improvement Process
Process Group
Group (PIM)
(PIM)
PIM.1
PIM.1 Process
Processestablishment
establishment
PIM.2
Process
PIM.2 Processassessment
assessment
PIM.3
PIM.3 Process
Processimprovement
improvement
Resource
Resource and
and Infrastructure
Infrastructure Process
Process Group
Group (RIN)
(RIN)
RIN.1
RIN.1 Human
Human resource
resourcemanagement
management
RIN.2
Training
RIN.2 Training
RIN.3
RIN.3 Knowledge
Knowledgemanagement
management
RIN.4
Infrastructure
RIN.4 Infrastructure
Reuse
Reuse Process
Process Group
Group (REU)
(REU)
REU.1
REU.1 Asset
Assetmanagement
management
REU.2
Reuse
REU.2 Reuse program
programmanagement
management
REU.3
Domain
REU.3 Domainengineering
engineering
© 2007 method park 10
Prozessdimension (ISO 15504)
Automotive SPiCE PRM und HIS-Scope
HIS-Geltungsbereich –
Ausgewählte SPICE-Prozesse
© 2007 method park 11
Reifegraddimension (ISO 15504)
Level
Level 55 Optimizing
Optimizing
PA.5.1
PA.5.1
PA.5.2
PA.5.2
Process
Process Innovation
Innovation
Process
Optimization
Process Optimization
Level
Level 44 Predictable
Predictable
PA.4.1
PA.4.1
PA.4.2
PA.4.2
Process
Process Measurement
Measurement
Process
Process Control
Control
Level
Level 33 Established
Established
PA.3.1
PA.3.1
PA.3.2
PA.3.2
Process
Process Definition
Definition
Process
Deployment
Process Deployment
Level
Level 22 Managed
Managed
PA.2.1
PA.2.1
PA.2.2
PA.2.2
Performance
Performance Management
Management
Work
Work Product
ProductManagement
Management
Level
Level 11 Performed
Performed
PA.1.1
PA.1.1
Process
Process Performance
Performance
Level
Level 00 Incomplete
Incomplete
© 2007 method park 12
Reifegraddimension (ISO 15504)
Level 1: Performed.
Der Prozess führt zum
gewünschten Ergebnis
The Performed process achieves its process purpose
through the performance of necessary actions
and the presence of appropriate input and output work
products
which, collectively, ensure
that the process outcome
is achieved.
Angemessene
Arbeitsprodukte
© 2007 method park 13
Reifegraddimension (ISO 15504)
Level 2: Managed Process.
The Managed process is planned, monitored and adjusted
to meet identified objectives for the performance of the process
and to produce work products that are appropriately
identified, documented and controlled.
Spezifizierte
Vorgehensweisen,
Planung, Verfolgung
QS der Dokumente
(Reviews, Freigaben)
© 2007 method park 14
ISO 15504: Bestimmen des Reifegrads
Bewertung der (frei wählbaren) Prozesse P1, ..., Pn
Reifegrad
optimizing (5)
predictable (4)
generic
practises
(allgemein,
für alle
Prozesse)
established (3)
managed (2)
performed (1)
base practises
(Prozessspezifisch)
incomplete (0)
P1 P2
P3
...
Pn
Prozesse
© 2007 method park 16
ISO 15504: Bestimmen des Reifegrads
Process attribute rating scale calibration
N Not achieved
P Partially achieved
L Largely achieved
F Fully achieved
N
0%
P
L
F
100%
© 2007 method park 17
ISO 15504: Bestimmen des Reifegrads
N / Not achieved:
Little or no evidence of achievement
of the defined attribute …
P / Partially achieved:
There is some evidence …, and some achievement of,
the defined attribute …
Some aspects of achievement may be unpredictable.
© 2007 method park 18
ISO 15504: Bestimmen des Reifegrads
L / Largely achieved:
Evidence of a systematic approach to,
and significant achievement of the defined attribute …
Some weakness … may exist …
F / Fully achieved:
Evidence of a complete and systematic approach to,
and full achievement of, the defined attribute …
No significant weaknesses … exist
© 2007 method park 19
Rating
Result: Capability Profile
Capability level
Process
1
2
3
4
Process attribute PA PA PA PA PA PA PA PA PA
1.1 2.1 2.2 3.1 3.2 4.1 4.2 5.1 5.2
ENG.1
Requirements elicitation
F
L
L
ENG.2
System requirements analysis
F
F
F
L
F
ENG.3
System architectural design
F
F
F
F
L
ENG.4
Software requirements analysis P
ENG.5
Software design
L
ENG.6
Software construction
F
F
L
ENG.7
Software integration
N
MAN.3
Project management
F
N
P
SUP.8
Configuration management
P
SUP.1
Quality assurance
P
ACQ.4
Supplier monitoring
F
L
F
F
= Fully achieved
L
5
= Largely achieved
P
= Partially achieved
N
2
3
3
0
1
2
0
1
0
0
2
= Not achieved
© 2007 method park 20
Gliederung
Reifegradmodelle
Übersicht über SPICE
SPICE und Test
Abschließende Bemerkungen
© 2007 method park 21
SPICE und Test
Vorab
Der aktuelle SPICE-Standard ISO 15504 hat im eigentlichen Sinn
gar nichts mit Testen zu tun!
Denn SPICE ist (nun) nur (noch) ein Bewertungsmodell
ohne eigene Prozesse!
Aber:
De facto verbindet man SPICE natürlich wie bisher
mit SoftwareEngineering-Prozessen,
insbesondere den Prozessen
des Exemplar Process Reference/Assessment Models (ISO 12207)
bzw. des Automotive SPICE Process Reference/Assessment Models
© 2007 method park 22
SPICE und Test
Aus Testsicht besonders wichtig
Engineering-Prozesse
Management-Prozesse
Support-Prozesse
© 2007 method park 23
SPICE und Test
Engineering Process Group (ENG)
ENG.1 Requirements elicitation
Kunde
ENG.1
ENG.2 System requirements analysis
ENG.3 System architectural design
ENG.2
System
ENG.4 Software requirements analysis
ENG.5 Software design
ENG.10
ENG.3
ENG.9
ENG.6 Software construction
ENG.7 Software integration
ENG.8 Software testing
ENG.9 System integration
ENG.10 System testing
ENG.4
SW
ENG.5
ENG.8
ENG.7
ENG.6
© 2007 method park 24
ISO 15504 Process Reference Model
normativ
Beispiel: Software testing process
© 2007 method park 25
ISO 15504 Process Assessment Model
Beispiel: Software testing process
(Anteil „Assessmentmodell“: Base practices)
Empfohlene
Vorgehensweisen
© 2007 method park 26
ISO 15504 Process Assessment Model
Für jeden Prozess
Empfohlene
Vorgehensweisen
Typische Eingangsdokumente
Typische Ausgangsdokumente
© 2007 method park 27
SPICE und Test
Traceability
Kunde
ENG.1
a) Anforderungen
b) Design
c) Test
ENG.2
ENG.10
System
ENG.3
ENG.4
ENG.9
SW
ENG.5
ENG.8
ENG.7
ENG.6
© 2007 method park 28
SPICE und Test
Management Process Group (MAN)
…
MAN.3 Project management
MAN.4 Quality management
MAN.5 Risk management
…
© 2007 method park 29
SPICE und Test
Support Process Group (SUP)
SUP.1 Quality assurance
SUP.2 Verification
SUP.3 Validation
SUP.4 Joint review
…
SUP.8 Configuration management
SUP.9 Problem resolution management
SUP.10 Change request management
© 2007 method park 30
SPICE und Test
Überfachlich (Level 2)
Projektmanagement des Prozesses
Definierte Ziele
Kennzahlen, Trends
Qualitätsmanagement des Prozesses
Definierte Ziele
Reviews und Freigaben von Arbeitsprodukten
© 2007 method park 31
Gliederung
Reifegradmodelle
Übersicht über SPICE
SPICE und Test
Abschließende Bemerkungen
© 2007 method park 32
Alternativen?
SPICE (Exemplar PRM, Automotive SPICE PRM)
erfasst den Software-Testprozess sehr gut
(= an den Entwicklungsphasen orientiert)
CMMI enthält durchaus Elemente eines
Software-Testprozesses;
diese sind jedoch quer durch das CMMI-Dokument verteilt
und nicht sehr einfach zu identifizieren
TPI, TMM, etc.
Reine Testreifegradmodelle, betrachten nicht bzw. nur rudimentär
die Einbettung in einen übergeordneten Entwicklungsprozess
© 2007 method park 33
Fragen, Kommentare, ...
[email protected]
© 2007 method park 34