Matrix Algebra As A Tool

Hadi, A. S. (1996)
Belmont, CA:
Duxbury Press.
ISBN: 0-534-23712-6

Preface

I teach intermediate and advanced applied statistics courses, such as applied regression and multivariate analysis, at both the undergraduate and graduate levels. Although most of my students are not statistics majors, their various fields of study demand many mathematical tools. Matrix algebra in particular is necessary for a thorough understanding of research methods courses in other disciplines.

Although the students in my classes are highly motivated, they often lack the mathematical skills needed to understand the material. To specify matrix algebra as a prerequisite would be an easy solution for me as a professor but an unsatisfactory one as an educator, because it would prevent or discourage many students who should, on the contrary, be encouraged to take such courses. Therefore, rather than specify matrix algebra as a prerequisite, I have tried two alternative solutions. The first was to present a brief review of the necessary matrix algebra tools and refer students to one of the many standard books on the topic. Matrix algebra books, however, are typically reference books written at a higher mathematical level, and are inaccessible for nonmajors. As a result, many students dropped out, and many who stayed were very confused.

Alternatively, I tried devoting a good portion of the course to matrix algebra. The obvious disadvantage of this solution, however, is that it does not leave enough time to cover the statistics topics that are the real point of the course. I finally compromised, devoting a relatively short time to matrix algebra and then referring students to some lecture notes I had specifically written for them. These notes - expanded, revised, and reorganized - evolved into this book.

The main purpose of the book is to present matrix algebra as a tool for students and researchers in various fields who find it necessary for their own work. It is intended to provide a basic understanding in language they can easily understand. The book's informal style has forced me on occasion to be mathematically imprecise, but this seems to me a small price to pay for increased accessibility. Some students will be able to read this book on their own, and much of it will require little instruction; the remaining part, however, may require a moderate amount of instruction. I hope that even students with prior courses in matrix algebra will find this book helpful.

The book can be used as a supplement to many courses, such as applied regression analysis, applied multivariate analysis, econometrics, biometrics, and other research methods courses in various disciplines such as sociology, psychology, economics, accounting, finance, marketing, and so on. The book may also serve as the primary text for a short course in matrix algebra that might be offered concurrently with the above-mentioned courses, for instance, or during the summer semester or other open periods in the academic calendar. The book may also serve as a text in a more rigorous matrix algebra course, although this was not the original intention in writing the book. In this case, the instructor may wish to provide proofs of some results and assign the proofs of other results as exercises.

I assume no previous acquaintance with the subject. Except for a very brief mention of the word derivative in Section 8.3 (which is an application section), calculus is not invoked. Although the book addresses many advanced topics (e.g., Section 6.3 and Chapters 8-11), these topics are covered at an elementary level and are illustrated by numerical examples.

As the title indicates, the emphasis is on practical rather than theoretical considerations, and much of the book is devoted to applications. Indeed, Chapters 8, 10, and 11 are devoted entirely to applications. Matrix notation, vocabulary, and concepts are emphasized. Although some of the results are derived, nearly all are stated without proof. Rather, the emphasis is on the interpretations and implications of the results and, in lieu of proofs, the results are illustrated and verified by examples.

Although there is a great temptation to do so in matrix algebra, I have tried not to emphasize matrix computations. Numerical examples unavoidably involve some calculations, but the examples are primarily designed to illustrate the concepts rather than matrix manipulations. For this reason, methods which are purely computational are relegated to an appendix to the chapter in which they appear. I use elementary methods for calculations, which are simple and easily understood but are not necessarily preferred in terms of computational efficiency or numerical accuracy. In most cases, computations are illustrated using small matrices. Most real applications, however, involve larger matrices, many of which are not practical to compute by hand. Whenever possible, professionally written computer programs should be used for matrix calculations in real-life applications. There are several main frame and personal computer packages aimed specifically at matrix calculations. Even spreadsheet programs now include some basic matrix calculations. I do not refer to any specific software or hardware because a general book like this should not be tied to specific software or hardware. Therefore, instructors may use any software package of their choice.

Geometry and matrix algebra are intimately connected, and matrix concepts can often be easily understood by relating them to their geometric counterparts. I tried to make this connection as early as possible in the book, beginning with Chapter 4.

Exercises are given at the end of each chapter. The exercises are not really optional, as many of them involve concepts supplementary to, but not covered in, the text. Readers should be able to do the computations by hand for all small matrices; otherwise all numerical work should be done by a computer. Also, readers are urged to substitute their own real data for the small matrices in the exercises and perform the calculations using a computer.

Vocabulary and concepts are defined and explained when they are first encountered, and key words are italicized. Some material, marked by an asterisk, is either more advanced, more theoretical, or more difficult to read. These parts can be skimmed or skipped at first reading. Except for Section 8.2 and the parts marked with asterisks, the first nine chapters constitute a minimum background and should not be skipped. The remaining material consists of either applications or advanced topics that can be read when the need arises. Nevertheless, it is recommended that the chapters be read in the order they are presented.

Two other chapters I originally intended to include, on other matrix decompositions (e.g., Cholesky, Q-R, and L-D-U) and on basic matrix differential calculus, were omitted as beyond the nature and scope of this book.

I am grateful to many of my students for the comments they have given me and to William Gould (Stata Corporation) for sharing with me his own handwritten notes on some topics of matrix algebra. Paul Green, Gary McDonald, and Mary Roybal have done a superb job editing the manuscript. I thank the following colleagues at Cornell University for reading and commenting on an early draft of the book: Jody Enk, Robert Hutchens, Matthew Hutcheson, Hyeseon Joo, Shayle Searle, Paul Velleman, and John Walker. I have also benefited from the comments of many colleagues at other universities. I am indebted to Gulhan Alpargu (McGill University), Bruce Barrett (University of Alabama), Samprit Chatterjee (New York University), Daniel Coleman (Carlos III University of Madrid), Mohammed El-Saidi (Ferris State University), Jose Garrido (University of Concordia), J. Brian Gray (University of Alabama), Brian Jersky (Sonoma State University), Robert Ling (Clemson University), Alberto Luceño (Cantabria University, Spain), Hans Nyquist (University of , Sweden), Daniel Peña (Carlos III University of Madrid), Jaime Puig-Pey (Cantabria University, Spain), Karen Shane (New York University), Simo Puntanen (University of Tampere, Finland), Gad Saad (University of Concordia), Gary Simon (New York University), Jeffrey Simonoff (New York University), Mun Son (University of Vermont), Lenn Stefanski (North Carolina State University), George Styan (McGill University), and Mohammed Youssef (Norfolk State University). Parts of the book were written while I was visiting the Department of Applied Mathematics and Computing Sciences of the University of Cantabria. I am grateful to Enrique Castillo, who made such a visit both enjoyable and productive

Ali S. Hadi
Ithaca, New York
June 1995

Back to top of page


Table of Contents

Preface v

1. Introduction 1

2. Some Matrix Calculations 13

3. Linear Dependence and Independence 31

4. Vector Geometry 49

5. Three Matrix Reductions 71

6. Matrix Inversion 81

7. Linear Transformation 101

8. Some Applications 117

9. Eigenvalues and Eigenvectors 147

10. Ellipsoids and Distances 161

11. Additional Applications 185

Bibliography 205

Index 207

Back to top of page


Back Cover

Matrix Algebra As a Tool provides students with a practical, applied approach to matrix algebra. It is intended for use in a one-term course in matrix algebra, or it can provide a matrix algebra background for applied courses such as regression analysis, multivariate analysis, econometrics, and other statistics and research methods courses in business, the social sciences, and other disciplines. Matrix Algebra As a Tool gives instructors a useful, flexible resources that avoids detailed discussions of mathematical proofs. Students will appreciate the book's clear, straightforward approach.

ABOUT THE AUTHOR

Ali S. Hadi (Ph.D., New York University) is Vice Provost and Professor of Mathematical, Statistical, and Computational Sciences at the American University in Cairo, Egypt. He is a Stephen H. Weiss Presidential Fellow and Professor Emeritus at Cornell University. He is a fellow of the American Statistical Association and an elected member of the International Statistical Institute. He is co-author of four other books:

Back to top of page