Notes on linear regression

NOTES

is a linear model that assumes a linear relationship between the input values and the output.
its representation is in form of a linear function.
each input variable has a coefficient associated with and a separe coefficient is added for addding degrees of freedom.
$y = B_{0} + B_{1} \times x$ <- a single input $x$.
with more dimensions the line becomes a plane or hyper-plane.
$B_{0}$ is the bias.
making predictions is simply solving the equation.
data must have no noise.
may over-fit if the inputs are highly correlated (colinear). Calculate the correlations and remove the highest correlated.
make more reliable predictions if the data follows a gaussian like format. Use transformations like log and BoxCon to shape it like that.
make better predictions when normalized and standardized.

aims to minimize the squared residuals.
calculate the distance from each point to the line, square it and sum all of the squared errors together.
uses linear algebra.
residuals -> is the difference between the real data and the line. ($residual = real(y) - predicted(^{y}$)
to quantify the error we use SSE (sum of squared errors) -> $SSE = \sum{y - ^{y}}$

start with a random configuration
sum the squared errors
uses a learning rate and the coefficients are optimized towards the minimizing point.

aim to minimize the squared error but also reduce the complexity of the model.
Lasso Regression (L1 regularization) -> Ordinary least squares are modified to minimize the absolute sum of the coefficients
Ridge Regression (L2 regularization) -> Ordinary least squares are modified to minimize the squared absolute sum of the coefficients