Comparison of ridge Regression Estimation and Ordinary Least Squares Estimation

Abstract

Regression analysis is one of the most widely used statistical techniques for analyzing multifactor data. It's broad appeal results from the conceptually simple process of using an equation to express the relationship between a set of variables.

In the model fitting process the most frequently applied and most popular estimation procedure is the Ordinary Least Squares Estimation (OLSE). The significant advantage of OLSE is that it provides minimum variance unbiased linear estimates for the parameters in the linear regression modeL

In many situations both experimental and non-experimental, especially in the field of business, economics, social sciences and biological sciences, the independent variables tend to be correlated among themselves, and is referred to as multicollinearity. A Variety of interrelated problems are created when multicolloinearity exists. In the model building process, multicollinearity among the independent variables causes high variance (if OLSE is used) even though the estimators are still the minimum variance unbiased estimators in the class of linear unbiased estimators.

In literature several biased estimation procedures were introduced for solving this problem. Among them, Ridge regression estimation, which was first introduced by Horel (1969), and further developed by Hoerl/Kennard (1970), was one of the frequently used methods. Hence this method was rapidly developed in the recent years.

In this study the theoretical results obtained for comparing the Ordinary Ridge regression estimator and OLSE will be applied for a set of multicollinear data, and the stochastic behavior of the two estimators will be analyzed.