Question

How regression analysis techniques help uncover relationships between variables?

What are the seven (7) steps for avoiding the potential pitfalls of regression analysis?

Answer 1

REGRESSION ANALYSIS :

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors'). More specifically, regression analysis helps one understand how the typical value of the dependent variable (or 'criterion variable') changes when any one of the independent variables is varied, while the other independent variables are held fixed.

Most commonly, regression analysis estimates the conditional expectation of the dependent variable given the independent variables – that is, the average value of the dependent variable when the independent variables are fixed.

7 steps to avoid potential pitfalls :

1. Conduct A Lot of Research Before Starting :

Before you begin the regression analysis, you should review the literature to develop an understanding of the relevant variables, their relationships, and the expected coefficient signs and effect magnitudes. Developing your knowledge base helps you gather the correct data in the first place, and it allows you to specify the best regression equation without resorting to data mining.

2. Use a Simple Model When Possible :

It seems that complex problems should require complicated regression equations. However, studies show that simplification usually produces more precise models.

3. Correlation Does Not Imply Causation, even in Regression :

Analysts can forget this important rule while performing regression analysis. As you build a model that has significant variables and a high R-squared, it’s easy to forget that you might only be revealing correlation. Causation is an entirely different matter. Typically, to establish causation, you need to perform a designed experiment with randomization. If you’re using regression to analyze data that weren’t collected in such an experiment, you can’t be certain about causation.

4. Include Graphs, Confidence, and Prediction Intervals in the Results :

It focuses on the fact that how you present your results can influence how people interpret them. The information can be the same, but the presentation style can prompt different reactions. For instance, confidence intervals and statistical significance provide consistent information. When a p-value is less than the 0.05 significance level, the corresponding 95% confidence interval will always exclude zero.

5. Check Your Residual Plots :

Residuals plots are a quick and easy way to check for problems in your regression model. These graphs can also help you make adjustments. For instance, residual plots display patterns when you fail to model curvature that is present in your data.

6. Uses large quantities of reliable data and a few independent variables with well established relationships.

7. Uses sound reasoning to determine which variables to include in the regression model.