PREFACE xvii

PREFACE TO THE FIRST EDITION xix

INTRODUCTION xxi

ABOUT THE COMPANION WEBSITE xxxi

**PART I DEFINITIONS, BASIC CONCEPTS, AND MATRIX OPERATIONS 1**

**1 Vector Spaces, Subspaces, and Linear Transformations 3**

1.1 Vector Spaces 3

1.2 Base of a Vector Space 5

1.3 Linear Transformations 7

**2 Matrix Notation and Terminology 11**

2.1 Plotting of a Matrix 14

2.2 Vectors and Scalars 16

2.3 General Notation 16

**3 Determinants 21**

3.1 Expansion by Minors 21

3.2 Formal Definition 25

3.3 Basic Properties 27

3.4 Elementary Row Operations 34

3.5 Examples 37

3.6 Diagonal Expansion 39

3.7 The Laplace Expansion 42

3.8 Sums and Differences of Determinants 44

3.9 A Graphical Representation of a 3 × 3 Determinant 45

**4 Matrix Operations 51**

4.1 The Transpose of a Matrix 51

4.2 Partitioned Matrices 52

4.3 The Trace of a Matrix 55

4.4 Addition 56

4.5 Scalar Multiplication 58

4.6 Equality and the Null Matrix 58

4.7 Multiplication 59

4.8 The Laws of Algebra 74

4.9 Contrasts With Scalar Algebra 76

4.10 Direct Sum of Matrices 77

4.11 Direct Product of Matrices 78

4.12 The Inverse of a Matrix 80

4.13 Rank of a Matrix—Some Preliminary Results 82

4.14 The Number of LIN Rows and Columns in a Matrix 84

4.15 Determination of the Rank of a Matrix 85

4.16 Rank and Inverse Matrices 87

4.17 Permutation Matrices 87

**5 Special Matrices 97**

5.1 Symmetric Matrices 97

5.2 Matrices Having All Elements Equal 102

5.3 Idempotent Matrices 104

5.4 Orthogonal Matrices 106

5.5 Parameterization of Orthogonal Matrices 109

5.6 Quadratic Forms 110

5.7 Positive Definite Matrices 113

**6 Eigenvalues and Eigenvectors 119**

6.1 Derivation of Eigenvalues 119

6.2 Elementary Properties of Eigenvalues 122

6.3 Calculating Eigenvectors 125

6.4 The Similar Canonical Form 128

6.5 Symmetric Matrices 131

6.6 Eigenvalues of Orthogonal and Idempotent Matrices 135

6.7 Eigenvalues of Direct Products and Direct Sums of Matrices 138

6.8 Nonzero Eigenvalues of AB and BA 140

**7 Diagonalization of Matrices 145**

7.1 Proving the Diagonability Theorem 145

7.2 Other Results for Symmetric Matrices 148

7.3 The Cayley–Hamilton Theorem 152

7.4 The Singular-Value Decomposition 153

**8 Generalized Inverses 159**

8.1 The Moore–Penrose Inverse 159

8.2 Generalized Inverses 160

8.3 Other Names and Symbols 164

8.4 Symmetric Matrices 165

**9 Matrix Calculus 171**

9.1 Matrix Functions 171

9.2 Iterative Solution of Nonlinear Equations 174

9.3 Vectors of Differential Operators 175

9.4 Vec and Vech Operators 179

9.5 Other Calculus Results 181

9.6 Matrices with Elements That Are Complex Numbers 188

9.7 Matrix Inequalities 189

**PART II APPLICATIONS OF MATRICES IN STATISTICS 199**

**10 Multivariate Distributions and Quadratic Forms 201**

10.1 Variance-Covariance Matrices 202

10.2 Correlation Matrices 203

10.3 Matrices of Sums of Squares and Cross-Products 204

10.4 The Multivariate Normal Distribution 207

10.5 Quadratic Forms and ��2-Distributions 208

10.6 Computing the Cumulative Distribution Function of a Quadratic Form 213

**11 Matrix Algebra of Full-Rank Linear Models 219**

11.1 Estimation of �� by the Method of Least Squares 220

11.2 Statistical Properties of the Least-Squares Estimator 226

11.3 Multiple Correlation Coefficient 229

11.4 Statistical Properties under the Normality Assumption 231

11.5 Analysis of Variance 233

11.6 The Gauss–Markov Theorem 234

11.7 Testing Linear Hypotheses 237

11.8 Fitting Subsets of the x-Variables 246

11.9 The Use of the R(.|.) Notation in Hypothesis Testing 247

**12 Less-Than-Full-Rank Linear Models 253**

12.1 General Description 253

12.2 The Normal Equations 256

12.3 Solving the Normal Equations 257

12.4 Expected Values and Variances 259

12.5 Predicted y-Values 260

12.6 Estimating the Error Variance 261

12.7 Partitioning the Total Sum of Squares 262

12.8 Analysis of Variance 263

12.9 The R(⋅|⋅) Notation 265

12.10 Estimable Linear Functions 266

12.11 Confidence Intervals 272

12.12 Some Particular Models 272

12.13 The R(⋅|⋅) Notation (Continued) 277

12.14 Reparameterization to a Full-Rank Model 281

**13 Analysis of Balanced Linear Models Using Direct Products of Matrices 287**

13.1 General Notation for Balanced Linear Models 289

13.2 Properties Associated with Balanced Linear Models 293

13.3 Analysis of Balanced Linear Models 298

**14 Multiresponse Models 313**

14.1 Multiresponse Estimation of Parameters 314

14.2 Linear Multiresponse Models 316

14.3 Lack of Fit of a Linear Multiresponse Model 318

**PART III MATRIX COMPUTATIONS AND RELATED SOFTWARE 327**

**15 SAS/IML 329**

15.1 Getting Started 329

15.2 Defining a Matrix 329

15.3 Creating a Matrix 330

15.4 Matrix Operations 331

15.5 Explanations of SAS Statements Used Earlier in the Text 354

**16 Use of MATLAB in Matrix Computations 363**

16.1 Arithmetic Operators 363

16.2 Mathematical Functions 364

16.3 Construction of Matrices 365

16.4 Two- and Three-Dimensional Plots 371

**17 Use of R in Matrix Computations 383**

17.1 Two- and Three-Dimensional Plots 396

Exercises 408

APPENDIX 413

INDEX 475