Bias vs Variance tradeoff is always encountered in applying supervised learning algorithms. Least squares regression provides a good fit for the training set but can suffer from high variance which lowers predictive ability. To counter this problem, we can regularize the beta coefficients by employing a penalization term. Ridge regression applies l2 penalty to the […]

## Ridge regression in R solutions

Below are the solutions to these exercises on ridge regression. ############### # # # Exercise 1 # # # ############### library(lars) library(glmnet) ## Warning: package ‘glmnet’ was built under R version 3.3.3 ## Warning: package ‘foreach’ was built under R version 3.3.3 data(diabetes) attach(diabetes) set.seed(1234) par(mfrow=c(2,5)) for(i in 1:10){ plot(x[,i], y) abline(lm(y~x[,i])) } layout(1) model_ols […]

## LASSO regression in R exercises

Least Absolute Shrinkage and Selection Operator (LASSO) performs regularization and variable selection on a given model. Depending on the size of the penalty term, LASSO shrinks less relevant predictors to (possibly) zero. Thus, it enables us to consider a more parsimonious model. In this exercise set we will use the glmnet package (package description: here) […]

## LASSO regression in R solutions

Below are the solutions to these exercises on LASSO regression. ############### # # # Exercise 1 # # # ############### library(lars) library(glmnet) data(diabetes) attach(diabetes) ############### # # # Exercise 2 # # # ############### summary(x) ## age sex bmi ## Min. :-0.107226 Min. :-0.04464 Min. :-0.090275 ## 1st Qu.:-0.037299 1st Qu.:-0.04464 1st Qu.:-0.034229 ## Median […]

## Quantile Regression in R exercises

The standard OLS (Ordinary Least Squares) model explains the relationship between independent variables and the conditional mean of the dependent variable. In contrast, quantile regression models this relationship for different quantiles of the dependent variable. In this exercise set we will use the quantreg package (package description: here) to implement quantile regression in R. Answers […]

## Quantile Regression in R solutions

Below are the solutions to these exercises on Quantile regression. ############### # # # Exercise 1 # # # ############### library(quantreg) ## ## data(barro) summary(barro) ## y.net lgdp2 mse2 fse2 ## Min. :-0.056124 Min. :5.820 Min. :0.0240 Min. :0.0000 ## 1st Qu.: 0.003529 1st Qu.:6.989 1st Qu.:0.3180 1st Qu.:0.1350 ## Median : 0.019648 Median :7.745 […]

## Instrumental Variables in R exercises (Part-3)

This is the third part of the series on Instrumental Variables. For other parts of the series follow the tag instrumental variables. In this exercise set we will use Generalized Method of Moments (GMM) estimation technique using the examples from part-1 and part-2. Recall that GMM estimation relies on the relevant moment conditions. For OLS […]

## Instrumental Variables in R exercises (Part-3) Solutions

Below are the solutions to these exercises on Instrumental variables (Part-3). ############### # # # Exercise 1 # # # ############### library(AER) ## Warning: package ‘lmtest’ was built under R version 3.3.3 ## Warning: package ‘zoo’ was built under R version 3.3.3 library(gmm) ## Warning: package ‘gmm’ was built under R version 3.3.3 data("PSID1976") df […]

## Instrumental Variables in R exercises (Part-2)

This is the second part of the series on Instrumental Variables. For other parts of the series follow the tag instrumental variables. In this exercise set we will build on the example from part-1. We will now consider an over-identified case i.e. we have multiple IVs for an endogenous variable. We will also look at […]

## Instrumental Variables in R exercises (Part-2) Solutions

Below are the solutions to these exercises on Instrumental Variables (Part-2). ############### # # # Exercise 1 # # # ############### library(AER) ## Warning: package ‘lmtest’ was built under R version 3.3.3 ## Warning: package ‘zoo’ was built under R version 3.3.3 data("PSID1976") df <- subset(PSID1976, participation=="yes") ############### # # # Exercise 2 # # […]