The confluence of cloud computing, parallelism and advanced machine intelligence approaches has created a world in which the optimum knowledge system will usually be architected from the combination of two or more knowledge-generating systems. There is a need, then, to provide a reusable, broadly-applicable set of design patterns to empower the intelligent system architect to take advantage of this opportunity.

This book explains how to design and build intelligent systems that are optimized for changing system requirements (adaptability), optimized for changing system input (robustness), and optimized for one or more other important system  parameters (e.g., accuracy, efficiency, cost). It provides an overview of traditional parallel processing which is shown to consist primarily of task and component parallelism; before introducing meta-algorithmic parallelism which is based on combining two or more algorithms, classification engines or other systems.

Key features:

• Explains the entire roadmap for the design, testing, development, refinement, deployment and statistics-driven optimization of building systems for intelligence
• Offers an accessible yet thorough overview of machine intelligence, in addition to having a strong image processing focus
• Contains design patterns for parallelism, especially meta-algorithmic parallelism – simply conveyed, reusable and proven effective that can be readily included in the toolbox of experts in analytics, system architecture, big data, security and many other science and engineering disciplines
• Connects algorithms and analytics to parallelism, thereby illustrating a new way of designing intelligent systems compatible with the tremendous changes in the computing world over the past decade
• Discusses application of the approaches to a wide number of fields; primarily, document understanding, image understanding, biometrics and security printing
• Companion website contains sample code and data sets

1 Introduction and Overview 1

1.1 Introduction 1

1.2 Why Is This Book Important? 2

1.3 Organization of the Book 3

1.4 Informatics 4

1.5 Ensemble Learning 6

1.6 Machine Learning/Intelligence 7

1.7 Artificial Intelligence 22

1.8 Data Mining/Knowledge Discovery 31

1.9 Classification 32

1.10 Recognition 38

1.11 System-Based Analysis 39

1.12 Summary 39

References 40

2 Parallel Forms of Parallelism 42

2.1 Introduction 42

2.2 Parallelism by Task 43

2.3 Parallelism by Component 52

2.4 Parallelism by Meta-algorithm 64

2.5 Summary 71

References 72

3 Domain Areas: Where Is This Relevant? 73

3.1 Introduction 73

3.2 Overview of the Domains 74

3.3 Primary Domains 75

3.4 Secondary Domains 86

3.5 Summary 101

References 102

4 Applications of Parallelism by Task 104

4.1 Introduction 104

4.2 Primary Domains 105

4.3 Summary 135

References 136

5 Application of Parallelism by Component 137

5.1 Introduction 137

5.2 Primary Domains 138

5.3 Summary 172

References 173

6 Introduction to Meta-algorithmics 175

6.1 Introduction 175

6.2 First-Order Meta-algorithmics 178

6.3 Second-Order Meta-algorithmics 195

6.4 Third-Order Meta-algorithmics 218

6.5 Summary 240

References 240

7 First-Order Meta-algorithmics and Their Applications 241

7.1 Introduction 241

7.2 First-Order Meta-algorithmics and the “Black Box” 241

7.3 Primary Domains 242

7.4 Secondary Domains 257

7.5 Summary 271

References 271

8 Second-Order Meta-algorithmics and Their Applications 272

8.1 Introduction 272

8.2 Second-Order Meta-algorithmics and Targeting the “Fringes” 273

8.3 Primary Domains 279

8.4 Secondary Domains 304

8.5 Summary 308

References 308

9 Third-Order Meta-algorithmics and Their Applications 310

9.1 Introduction 310

9.2 Third-Order Meta-algorithmic Patterns 311

9.3 Primary Domains 313

9.4 Secondary Domains 328

9.5 Summary 340

References 341

10 Building More Robust Systems 342

10.1 Introduction 342

10.2 Summarization 342

10.3 Cloud Systems 350

10.4 Mobile Systems 353

10.5 Scheduling 355

10.6 Classification 356

10.7 Summary 358

Reference 359

11 The Future 360

11.1 Recapitulation 360

11.2 The Pattern of all Patience 362

11.3 Beyond the Pale 365

11.4 Coming Soon 367

11.5 Summary 368

References 368

Index

Steven J. Simske, Hewlett-Packard Labs, Colorado, USA
Dr Simske is currently Director of the Document Ecosystem Lab, at Hewlett-Packard Labs, Colorado, USA. He has been working in algorithms, imaging, machine learning and classification for the past 20 years. As an engineer at HP Labs, he has designed, developed and shipped products associated with a very broad array of domains—document understanding, image segmentation and understanding, speech recognition, medical signal processing and imaging, biometrics, natural language processing, surveillance, optical character recognition, security analytics and security printing. The advantages of systematic meta-algorithmic approaches to the robustness, accuracy, cost and/or other system features which is the focus of the book has been evident across these domains. Dr. Simske is an HP Fellow, IS&T Fellow and IEEE Senior Member. He has published 300 articles and book chapters; and holds 45 US Patents primarily in the areas of classification, machine learning, and large system design and development.