\begin{eqnarray}
C=\sum_{i=1}^N (a_1 x_1^i + \ldots + a_n x_n^i + b – y^i)^2 + \lambda (a_1^2 + a_2^2 + \ldots + a_n^2 + b^2 )
\end{eqnarray}
Using information from the derivation of linear regression, we arrive at:
\begin{eqnarray}
\begin{bmatrix}
\frac{\partial C}{\partial b} \\[6pt]
\frac{\partial C}{\partial a_1} \\
\vdots \\
\frac{\partial C}{\partial a_n}
\end{bmatrix}
&=&
\mathbb{X}^T ( \mathbb{X} \overrightarrow{a} – \overrightarrow{y} ) + 2\lambda \begin{bmatrix}
b \\[6pt]
a_1 \\
\vdots \\
a_n
\end{bmatrix}
\end{eqnarray}
or,
\begin{eqnarray}
\nabla C = \mathbb{X}^T ( \mathbb{X} \overrightarrow{a} – \overrightarrow{y} ) + 2\lambda \overrightarrow{a}
\end{eqnarray}
we can insert an identity matrix to the rigtmost term:
\begin{eqnarray}
\nabla C = \mathbb{X}^T ( \mathbb{X} \overrightarrow{a} – \overrightarrow{y} ) + 2\lambda \mathbb{I} \overrightarrow{a}
\end{eqnarray}
equating the gradient to zero and solving for $\overrightarrow{a}$, we get
\begin{eqnarray}
\overrightarrow{a} = [(\mathbb{X}^T \mathbb{X})^{-1}+\lambda \mathbb{I}] \mathbb{X}^T \overrightarrow{y}
\end{eqnarray}
Note: The intercept \(b\) is usually excluded from regularization, as it shifts all predictions up or down by the same amount. We usually do not penalize it because the model needs this freedom to match the average level of the observed values