Skip to main content

Linear Models and Time-Series Analysis: Regression, ANOVA, ARMA and GARCH

E-Book

$100.99

Linear Models and Time-Series Analysis: Regression, ANOVA, ARMA and GARCH

Marc S. Paolella

ISBN: 978-1-119-43198-5 October 2018 896 Pages

Description

A comprehensive and timely edition on an emerging new trend in time series

Linear Models and Time-Series Analysis: Regression, ANOVA, ARMA and GARCH sets a strong foundation, in terms of distribution theory, for the linear model (regression and ANOVA), univariate time series analysis (ARMAX and GARCH), and some multivariate models associated primarily with modeling financial asset returns (copula-based structures and the discrete mixed normal and Laplace). It builds on the author's previous book, Fundamental Statistical Inference: A Computational Approach, which introduced the major concepts of statistical inference. Attention is explicitly paid to application and numeric computation, with examples of Matlab code throughout. The code offers a framework for discussion and illustration of numerics, and shows the mapping from theory to computation. 

The topic of time series analysis is on firm footing, with numerous textbooks and research journals dedicated to it. With respect to the subject/technology, many chapters in Linear Models and Time-Series Analysis cover firmly entrenched topics (regression and ARMA). Several others are dedicated to very modern methods, as used in empirical finance, asset pricing, risk management, and portfolio optimization, in order to address the severe change in performance of many pension funds, and changes in how fund managers work. 

  • Covers traditional time series analysis with new guidelines
  • Provides access to cutting edge topics that are at the forefront of financial econometrics and industry
  • Includes latest developments and topics such as financial returns data, notably also in a multivariate context
  • Written by a leading expert in time series analysis 
  • Extensively classroom tested
  • Includes a tutorial on SAS
  • Supplemented with a companion website containing numerous Matlab programs
  • Solutions to most exercises are provided in the book

Linear Models and Time-Series Analysis: Regression, ANOVA, ARMA and GARCH is suitable for advanced masters students in statistics and quantitative finance, as well as doctoral students in economics and finance. It is also useful for quantitative financial practitioners in large financial institutions and smaller finance outlets.    

Preface xiii

Part I Linear Models: Regression and ANOVA 1

1 The Linear Model 3

1.1 Regression, Correlation, and Causality 3

1.2 Ordinary and Generalized Least Squares 7

1.2.1 Ordinary Least Squares Estimation 7

1.2.2 Further Aspects of Regression and OLS 8

1.2.3 Generalized Least Squares 12

1.3 The Geometric Approach to Least Squares 17

1.3.1 Projection 17

1.3.2 Implementation 22

1.4 Linear Parameter Restrictions 26

1.4.1 Formulation and Estimation 27

1.4.2 Estimability and Identifiability 30

1.4.3 Moments and the Restricted GLS Estimator 32

1.4.4 Testing With h = 0 34

1.4.5 Testing With Nonzero h 37

1.4.6 Examples 37

1.4.7 Confidence Intervals 42

1.5 Alternative Residual Calculation 47

1.6 Further Topics 51

1.7 Problems 56

1.A Appendix: Derivation of the BLUS Residual Vector 60

1.B Appendix: The Recursive Residuals 64

1.C Appendix: Solutions 66

2 Fixed Effects ANOVA Models 77

2.1 Introduction: Fixed, Random, and Mixed Effects Models 77

2.2 Two Sample t-Tests for Differences in Means 78

2.3 The Two Sample t-Test with Ignored Block Effects 84

2.4 One-Way ANOVA with Fixed Effects 87

2.4.1 The Model 87

2.4.2 Estimation and Testing 88

2.4.3 Determination of Sample Size 91

2.4.4 The ANOVA Table 93

2.4.5 Computing Confidence Intervals 97

2.4.6 A Word on Model Assumptions 103

2.5 Two-Way Balanced Fixed Effects ANOVA 107

2.5.1 The Model and Use of the Interaction Terms 107

2.5.2 Sums of Squares Decomposition without Interaction 108

2.5.3 Sums of Squares Decomposition with Interaction 113

2.5.4 Example and Codes 117

3 Introduction to Random and Mixed Effects Models 127

3.1 One-Factor Balanced Random Effects Model 128

3.1.1 Model and Maximum Likelihood Estimation 128

3.1.2 Distribution Theory and ANOVA Table 131

3.1.3 Point Estimation, Interval Estimation, and Significance Testing 137

3.1.4 Satterthwaite’s Method 139

3.1.5 Use of SAS 142

3.1.6 Approximate Inference in the Unbalanced Case 143

3.1.6.1 Point Estimation in the Unbalanced Case 144

3.1.6.2 Interval Estimation in the Unbalanced Case 150

3.2 Crossed Random Effects Models 152

3.2.1 Two Factors 154

3.2.1.1 With Interaction Term 154

3.2.1.2 Without Interaction Term 157

3.2.2 Three Factors 157

3.3 Nested Random Effects Models 162

3.3.1 Two Factors 162

3.3.1.1 Both Effects Random: Model and Parameter Estimation 162

3.3.1.2 Both Effects Random: Exact and Approximate Confidence Intervals 167

3.3.1.3 Mixed Model Case 170

3.3.2 Three Factors 174

3.3.2.1 All Effects Random 174

3.3.2.2 Mixed: Classes Fixed 176

3.3.2.3 Mixed: Classes and Subclasses Fixed 177

3.4 Problems 177

3.A Appendix: Solutions 178

Part II Time-Series Analysis: ARMAX Processes 185

4 The AR(1) Model 187

4.1 Moments and Stationarity 188

4.2 Order of Integration and Long-Run Variance 195

4.3 Least Squares and ML Estimation 196

4.3.1 OLS Estimator of a 196

4.3.2 Likelihood Derivation I 196

4.3.3 Likelihood Derivation II 198

4.3.4 Likelihood Derivation III 198

4.3.5 Asymptotic Distribution 199

4.4 Forecasting 200

4.5 Small Sample Distribution of the OLS and ML Point Estimators 204

4.6 Alternative Point Estimators of a 208

4.6.1 Use of the Jackknife for Bias Reduction 208

4.6.2 Use of the Bootstrap for Bias Reduction 209

4.6.3 Median-Unbiased Estimator 211

4.6.4 Mean-Bias Adjusted Estimator 211

4.6.5 Mode-Adjusted Estimator 212

4.6.6 Comparison 213

4.7 Confidence Intervals for a 215

4.8 Problems 219

5 Regression Extensions: AR(1) Errors and Time-varying Parameters 223

5.1 The AR(1) Regression Model and the Likelihood 223

5.2 OLS Point and Interval Estimation of a 225

5.3 Testing a = 0 in the ARX(1) Model  229

5.3.1 Use of Confidence Intervals 229

5.3.2 The Durbin–Watson Test 229

5.3.3 Other Tests for First-order Autocorrelation 231

5.3.4 Further Details on the Durbin–Watson Test 236

5.3.4.1 The Bounds Test, and Critique of Use of p-Values 236

5.3.4.2 Limiting Power as a → ±1 239

5.4 Bias-Adjusted Point Estimation 243

5.5 Unit Root Testing in the ARX(1) Model 246

5.5.1 Null is a = 1 248

5.5.2 Null is a < 1 256

5.6 Time-Varying Parameter Regression 259

5.6.1 Motivation and Introductory Remarks 260

5.6.2 The Hildreth–Houck Random Coefficient Model 261

5.6.3 The TVP Random Walk Model 269

5.6.3.1 Covariance Structure and Estimation 271

5.6.3.2 Testing for Parameter Constancy 274

5.6.4 Rosenberg Return to Normalcy Model 277

6 Autoregressive and Moving Average Processes 281

6.1 AR(p) Processes 281

6.1.1 Stationarity and Unit Root Processes 282

6.1.2 Moments 284

6.1.3 Estimation 287

6.1.3.1 Without Mean Term 287

6.1.3.2 Starting Values 290

6.1.3.3 With Mean Term 292

6.1.3.4 Approximate Standard Errors 293

6.2 Moving Average Processes 294

6.2.1 MA(1) Process 294

6.2.2 MA(q) Processes 299

6.3 Problems 301

6.A Appendix: Solutions 302

7 ARMA Processes 311

7.1 Basics of ARMA Models 311

7.1.1 The Model 311

7.1.2 Zero Pole Cancellation 312

7.1.3 Simulation 313

7.1.4 The ARIMA(p, d, q) Model 314

7.2 Infinite AR and MA Representations 315

7.3 Initial Parameter Estimation 317

7.3.1 Via the Infinite AR Representation 318

7.3.2 Via Infinite AR and Ordinary Least Squares 318

7.4 Likelihood-Based Estimation 322

7.4.1 Covariance Structure 322

7.4.2 Point Estimation 324

7.4.3 Interval Estimation 328

7.4.4 Model Mis-specification 330

7.5 Forecasting 331

7.5.1 AR(p) Model 331

7.5.2 MA(q) and ARMA(p, q) Models 335

7.5.3 ARIMA(p, d, q) Models 339

7.6 Bias-Adjusted Point Estimation: Extension to the ARMAX(1, q) model 339

7.7 Some ARIMAX Model Extensions 343

7.7.1 Stochastic Unit Root 344

7.7.2 Threshold Autoregressive Models 346

7.7.3 Fractionally Integrated ARMA (ARFIMA) 347

7.8 Problems 349

7.A Appendix: Generalized Least Squares for ARMA Estimation 351

7.B Appendix: Multivariate AR(p) Processes and Stationarity, and General Block Toeplitz Matrix Inversion 357

8 Correlograms 359

8.1 Theoretical and Sample Autocorrelation Function 359

8.1.1 Definitions 359

8.1.2 Marginal Distributions 365

8.1.3 Joint Distribution 371

8.1.3.1 Support  371

8.1.3.2 Asymptotic Distribution 372

8.1.3.3 Small-Sample Joint Distribution Approximation 375

8.1.4 Conditional Distribution Approximation 381

8.2 Theoretical and Sample Partial Autocorrelation Function 384

8.2.1 Partial Correlation 384

8.2.2 Partial Autocorrelation Function 389

8.2.2.1 TPACF: First Definition 389

8.2.2.2 TPACF: Second Definition 390

8.2.2.3 Sample Partial Autocorrelation Function 392

8.3 Problems 396

8.A Appendix: Solutions 397

9 ARMA Model Identification 405

9.1 Introduction 405

9.2 Visual Correlogram Analysis 407

9.3 Significance Tests 412

9.4 Penalty Criteria 417

9.5 Use of the Conditional SACF for Sequential Testing 421

9.6 Use of the Singular Value Decomposition 436

9.7 Further Methods: Pattern Identification 439

Part III Modeling Financial Asset Returns 443

10 Univariate GARCH Modeling 445

10.1 Introduction 445

10.2 Gaussian GARCH and Estimation 450

10.2.1 Basic Properties 451

10.2.2 Integrated GARCH 452

10.2.3 Maximum Likelihood Estimation 453

10.2.4 Variance Targeting Estimator 459

10.3 Non-Gaussian ARMA-APARCH, QMLE, and Forecasting 459

10.3.1 Extending the Volatility, Distribution, and Mean Equations 459

10.3.2 Model Mis-specification and QMLE 464

10.3.3 Forecasting 467

10.4 Near-Instantaneous Estimation of NCT-APARCH(1,1) 468

10.5 S𝛼,𝛽-APARCH and Testing the IID Stable Hypothesis 473

10.6 Mixed Normal GARCH 477

10.6.1 Introduction 477

10.6.2 The MixN(k)-GARCH(r, s) Model 478

10.6.3 Parameter Estimation and Model Features 479

10.6.4 Time-Varying Weights 482

10.6.5 Markov Switching Extension 484

10.6.6 Multivariate Extensions 484

11 Risk Prediction and Portfolio Optimization  487

11.1 Value at Risk and Expected Shortfall Prediction 487

11.2 MGARCH Constructs Via Univariate GARCH 493

11.2.1 Introduction 493

11.2.2 The Gaussian CCC and DCC Models 494

11.2.3 Morana Semi-Parametric DCC Model 497

11.2.4 The COMFORT Class 499

11.2.5 Copula Constructions 503

11.3 Introducing Portfolio Optimization 504

11.3.1 Some Trivial Accounting 504

11.3.2 Markowitz and DCC 510

11.3.3 Portfolio Optimization Using Simulation 513

11.3.4 The Univariate Collapsing Method 516

11.3.5 The ES Span 521

12 Multivariate t Distributions 525

12.1 Multivariate Student’s t 525

12.2 Multivariate Noncentral Student’s t 530

12.3 Jones Multivariate t Distribution 534

12.4 Shaw and Lee Multivariate t Distributions 538

12.5 The Meta-Elliptical t Distribution 540

12.5.1 The FaK Distribution 541

12.5.2 The AFaK Distribution 542

12.5.3 FaK and AFaK Estimation: Direct Likelihood Optimization 546

12.5.4 FaK and AFaK Estimation: Two-Step Estimation 548

12.5.5 Sums of Margins of the AFaK 555

12.6 MEST: Marginally Endowed Student’s t 556

12.6.1 SMESTI Distribution 557

12.6.2 AMESTI Distribution 558

12.6.3 MESTI Estimation 561

12.6.4 AoNm-MEST 564

12.6.5 MEST Distribution 573

12.7 Some Closing Remarks 574

12.A ES of Convolution of AFaK Margins 575

12.B Covariance Matrix for the FaK 581

13 Weighted Likelihood 587

13.1 Concept 587

13.2 Determination of Optimal Weighting 592

13.3 Density Forecasting and Backtest Overfitting 594

13.4 Portfolio Optimization Using (A)FaK 600

14 Multivariate Mixture Distributions 611

14.1 The Mixk Nd Distribution 611

14.1.1 Density and Simulation 612

14.1.2 Motivation for Use of Mixtures 612

14.1.3 Quasi-Bayesian Estimation and Choice of Prior 614

14.1.4 Portfolio Distribution and Expected Shortfall 620

14.2 Model Diagnostics and Forecasting 623

14.2.1 Assessing Presence of a Mixture 623

14.2.2 Component Separation and Univariate Normality 625

14.2.3 Component Separation and Multivariate Normality 629

14.2.4 Mixed Normal Weighted Likelihood and Density Forecasting 631

14.2.5 Density Forecasting: Optimal Shrinkage 633

14.2.6 Moving Averages of 𝜆 640

14.3 MCD for Robustness and Mix2Nd Estimation 645

14.4 Some Thoughts on Model Assumptions and Estimation 647

14.5 The Multivariate Laplace and Mixk Lapd Distributions 649

14.5.1 The Multivariate Laplace and EM Algorithm 650

14.5.2 The Mixk Lapd and EM Algorithm 654

14.5.3 Estimation via MCD Split and Forecasting 658

14.5.4 Estimation of Parameter b 660

14.5.5 Portfolio Distribution and Expected Shortfall 662

14.5.6 Fast Evaluation of the Bessel Function 663

Part IV Appendices 667

Appendix A Distribution of Quadratic Forms 669

A.1 Distribution and Moments 669

A.1.1 Probability Density and Cumulative Distribution Functions 669

A.1.2 Positive Integer Moments 671

A.1.3 Moment Generating Functions 673

A.2 Basic Distributional Results 677

A.3 Ratios of Quadratic Forms in Normal Variables 679

A.3.1 Calculation of the CDF 680

A.3.2 Calculation of the PDF 681

A.3.2.1 Numeric Differentiation 682

A.3.2.2 Use of Geary’s formula 682

A.3.2.3 Use of Pan’s Formula 683

A.3.2.4 Saddlepoint Approximation 685

A.4 Problems 689

A.A Appendix: Solutions 690

Appendix B Moments of Ratios of Quadratic Forms 695

B.1 For X ∼ Nn(0, 2I) and B = I  695

B.2 For X ∼ N(0, Σ) 708

B.3 For X ∼ N(𝜇, I) 713

B.4 For X ∼ N(𝜇, Σ) 720

B.5 Useful Matrix Algebra Results 725

B.6 Saddlepoint Equivalence Result 729

Appendix C Some Useful Multivariate Distribution Theory 733

C.1 Student’s t Characteristic Function 733

C.2 Sphericity and Ellipticity 739

C.2.1 Introduction 739

C.2.2 Sphericity 740

C.2.3 Ellipticity 748

C.2.4 Testing Ellipticity 768

Appendix D Introducing the SAS Programming Language 773

D.1 Introduction to SAS 774

D.1.1 Background 774

D.1.2 Working with SAS on a PC 775

D.1.3 Introduction to the Data Step and the Program Data Vector 777

D.2 Basic Data Handling 783

D.2.1 Method 1 784

D.2.2 Method 2 785

D.2.3 Method 3 786

D.2.4 Creating Data Sets from Existing Data Sets 787

D.2.5 Creating Data Sets from Procedure Output 788

D.3 Advanced Data Handling 790

D.3.1 String Input and Missing Values 790

D.3.2 Using set with first.var and last.var 791

D.3.3 Reading in Text Files 795

D.3.4 Skipping over Headers 796

D.3.5 Variable and Value Labels 796

D.4 Generating Charts, Tables, and Graphs 797

D.4.1 Simple Charting and Tables 798

D.4.2 Date and Time Formats/Informats 801

D.4.3 High Resolution Graphics 803

D.4.3.1 The GPLOT Procedure 803

D.4.3.2 The GCHART Procedure 805

D.4.4 Linear Regression and Time-Series Analysis 806

D.5 The SAS Macro Processor 809

D.5.1 Introduction 809

D.5.2 Macro Variables 810

D.5.3 Macro Programs 812

D.5.4 A Useful Example 814

D.5.4.1 Method 1 814

D.5.4.2 Method 2 816

D.6 Problems 817

D.7 Appendix: Solutions 819

Bibliography 825

Index 875