Performance-Based Certification: How to Design a Valid, Defensible, Cost-Effective Program

Contents of the Website / xiii

Introduction / 1

Chapter 1 The Driver / 11

Why Organizations Certify / 11

Protecting the Public / 12

Reinforcing Professional Stature and Promoting Universal Standards / 13

Preparing People for Jobs Requiring Competence in Multiple Disciplines / 14

Protecting Jobs and Enhancing Professional Stature / 15

Improving Business Processes / 16

Establishing Professional Credibility and Influencing Academic Curricula / 18

Establishing Uniform Performance Standards / 18

Protecting the Brand Name / 19

Raising the Level of Core Competencies / 20

Success Measures / 21

Who to Involve / 21

The Players / 21

Target Audience / 22

Stakeholders / 22

Benefits of Certification / 25

Missteps and Oversights / 26

Tips / 27

Summary / 30

Where to Learn More / 31

Notes / 32

Chapter 2 The Business Case / 33

What Goes into a Business Case / 34

Hypotheses, Premises, and Best Guesses / 36

Metrics or Key Success Indicators / 37

Economic Metrics / 39

Noneconomic Metrics / 42

The Requirements / 43

Missteps and Oversights / 43

Tips / 44

Summary / 48

Where to Learn More / 50

Note / 51

Chapter 3 The Requirements / 53

Distinctions Among Eligibility, Qualification, and Certification / 53

Roles Certification Plays / 54

Gatekeeping or Screening / 54

Recognizing Demonstrated Performance / 57

Recognizing Different Levels of Accomplishment or Different Capabilities / 57

Typical Requirements for Certification / 58

Acceptance of a Code of Conduct / 61

Eligibility / 61

Education, Training, and Development / 65

Endorsements / 67

Experience / 67

External Credentials / 69

Tests / 69

Work Samples / 72

Work or Personnel Records / 72

Maintenance and Recertification / 73

Missteps and Oversights / 74

Tips / 74

Summary / 76

Where to Learn More / 77

Notes / 78

Chapter 4 The Standards / 79

Competencies, Standards, and Criteria / 81

Defi ning the Scope of the Effort / 85

The Job or Task Analysis / 89

Traditional Job/Task Analysis Methodologies / 89

Controlling Bias / 91

Sampling Error / 92

Design Error / 95

Administrative Error / 109

Design of the Credential / 111

Certification vs. Certificate / 111

The Elements / 114

Missteps and Oversights / 115

Tips / 116

Standards / 118

The Design / 119

Summary / 119

Procedures / 121

Focus Groups / 121

The NGT / 122

Outcome-Based Competency Session / 123

Delphi Study / 124

Where to Learn More / 125

Note / 126

Chapter 5 Assessment / 127

Definitions / 127

Rigor and Validity / 130

Sampling Error / 131

Under-Representation / 131

Extraneous Abilities / 132

Test Specifications / 133

Design Errors / 134

Administrative Error / 136

Types of Test Items / 137

Response-Supplied Items / 137

Response-Not-Supplied Items / 138

Computer-Based Testing / 141

Determining the Passing Score / 142

Informed Judgment Method / 143

Contrasting Group Method / 143

Conjectural (Angoff-Nedelsky) Method / 145

Issues in Assessment and Testing / 146

Opportunity to Learn / 146

Adequate Time and Resources / 147

Face Validity / 147

Documentation / 148

Missteps and Oversights / 148

Tips / 149

Summary / 155

Where to Learn More / 161

Chapter 6 Governance and Administration / 163

Responsibilities of the Governance Board / 163

Candidate Rights / 165

Disclosure / 165

Appeals and Exemptions / 167

Preparation and Remediation / 169

Ethics / 170

Fees and Compensation / 170

Test Administration / 174

Responsibilities of the Program Administrator / 175

Establishing Administrative Support Systems / 176

Analyzing Test Items / 177

Missteps and Oversights / 178

Tips / 179

Summary / 187

Where to Learn More / 190

Notes / 191

Chapter 7 Recertification and Maintenance / 193

Stability of Enabling Knowledge / 194

Distinguishing Between Knowledge Atrophy and Knowledge Evolution / 195

Defining the Purpose of Recertification or Maintenance / 198

Redoing the Job/Task Analysis / 199

Timing the Requirement / 200

Missteps and Oversights / 201

Tips / 201

Summary / 203

Where to Learn More / 205

Chapter 8 Implementation / 207

The Marketing Strategy / 208

Defining the Market / 209

Branding the Credential / 210

Launching the Credential / 211

Rolling Out the Credential and Penetrating a Market / 212

Developing and Executing a Communication Plan / 213

Developing Effective Program Materials / 214

Celebrating Success / 215

Missteps and Oversights / 216

Tips / 217

Summary / 220

Where to Learn More / 220

Chapter 9 Going Global / 221

Distinctions Among National, International, Global, and Multi-National Programs / 221

Why Go Global or International? / 224

Determining Reciprocity, Equivalency, and Exemptions / 225

Safeguarding Against Fraud or Misrepresentation / 226

Cultural Issues / 227

Language and Translation / 227

Missteps and Oversights / 229

Tips / 230

Summary / 230

Where to Learn More / 233

Note / 233

Chapter 10 Evaluation / 235

Developing an Evaluation Strategy / 235

Defining Evaluation Processes / 236

Determining What to Measure / 237

Determining Where to Start / 238

Being Clear About Your Purpose / 242

Developing an Agenda / 244

Presenting Your Data / 244

Missteps and Oversights / 246

Tips / 247

Summary / 248

Where to Learn More / 251

Index / 253

About the Author / 269