\section{Linear Approximation and Differentiability}
Recall that for a single-variable function $f(x)$ differentiable at $x = a$, we have the \underline{linear approximation}
$$
f(x) \approx L(x) : = f(a) + f' (a) (x - a).
$$
This is a good approximation in the sense that
$$
|f(x) - L(x)| = o (|x-a|),
$$
where the ``little $o$'' notation means
$$
\lim_{x \rightarrow a} \frac{|f(x) - L(x)|}{|x - a|} = 0.
$$
We now develop a similar notion of linearization for multi-variable functions.

\subsection{Linearization for Functions of Two Variables} For a function $f(x,y)$ whose partial derivatives $f_x$ and $f_y$ exist at a point $(a,b)$, we want to approximate the values of $f(x,y)$ near $(x,y) = (a,b)$ by the linear function
\begin{equation}\label{eq:31.1}
L(x,y) : = f(a,b) + f_x (a,b) (x - a) + f_y (a,b) (y - b).
\end{equation}
This is the function from the tangent plane equation: we said earlier that if the graph $z=f(x,y)$ has a non-vertical tangent plane at 
the point $(a,b, f(a,b))$, then the equation of this tangent plane is given by $z=L(x,y)$. We did not say previously what exactly it means that this plane is ``tangent" to the graph, or what conditions we need to assume about $f(x,y)$. This is what we will do now.

\begin{itemize}
\item We \underline{define the linearization} of $f(x,y)$ at $(a,b)$ to be the function $L(x,y)$ as in \eqref{eq:31.1}. For the formula to make sense, we need both partials $f_1$ and $f_2$ to exist at $(a,b)$, but this does not yet mean that $L(x,y)$ is a good approximation to anything.

\item In Definition \ref{31-def1}, we state precisely what it means for $L(x,y)$ to be a good approximation to $f(x,y)$ near $(a,b)$. The term we will use for this is \underline{differentiability}. This is a stronger condition than just the existence of $L(x,y)$.

\item We will then look at conditions that guarantee that $f$ is differentiable at a point. An easy-to-use criterion is that if $f$ is $C^1$ on an open set $S$, it is also differentiable at every point of $S$ (Theorem \ref{thm:31thm1}). For functions that are not $C^1$, we may need to check the differentiability condition directly, and we will see examples of that in the next section.

\item We will also see a few examples of using the linear approximation to find approximate values of functions.
\end{itemize}

\begin{definition}\label{31-def1}
We say that $f(x,y)$ is \textbf{differentiable} at $(a,b)$ if the approximation $L(x,y)$ defined in \eqref{eq:31.1} is good, in the sense that
\begin{equation}\label{eq:31.2}
|f(x,y) - L(x,y)| = o \big(\sqrt{(x-a)^2 + (y-b)^2} \big)
\end{equation}
\end{definition}


\medskip

This time, the little-$o$ notation of \eqref{eq:31.2} means that
$$
\lim_{(x,y) \rightarrow (a,b)} \frac{|f(x,y) - L(x,y)|}{\sqrt{(x-a)^2 + (y-b)^2}} = 0,
$$
or equivalently
$$
\lim_{(\Delta x, \Delta y) \rightarrow (0,0)} \frac{|f(a + \Delta x, b + \Delta y) - L(a + \Delta x, b + \Delta y)|}{\sqrt{(\Delta x)^2 + (\Delta y)^2}} = 0
$$
where $\Delta x = x - a$ and $\Delta y = y - b$.

We used the absolute value in the expression $|f(x,y) - L(x,y)|$ to emphasize that this limit gives a bound on the size of the difference. However, the condition
$$
\lim_{(x,y) \rightarrow (a,b)} \frac{f(x,y) - L(x,y)}{\sqrt{(x-a)^2 + (y-b)^2}} = 0
$$
without absolute values, is equivalent since
$$
\lim_{(x,y) \rightarrow (a,b)} F(x,y) = 0 \ \ \Leftrightarrow \ \  \lim_{(x,y) \rightarrow (a,b)} |F(x,y)| = 0.
$$

How does our definition of differentiability for $f(x,y)$ compare with our $C^1$ condition (that is, that $f$ and all its first-order partial derivatives are continuous on some open set)? This question is answered in the following theorem, stated here without proof.

\begin{theorem}\label{thm:31thm1}
If $f(x,y)$ is $C^1$ on a neighbourhood of $(a,b)$, it is differentiable at $(a,b)$.
\end{theorem}

The next diagram illustrates the relationships between the various properties of $f$ related to continuity and differentiability. The implications cannot be reversed; for example, $f(x,y)$ can be differentiable at $(a,b)$ \textit{without being} $C^1$!

\smallskip


\begin{minipage}{\textwidth}
\begin{center}
\begin{tikzpicture}[node distance=2cm]
\node (node1) [startstop] {$f$, $f_x$, $f_y$ continuous in a neighbourhood of $(a,b)$};
\node (node2) [startstop,below of = node1] {$f$ is differentiable at $(a,b)$};
\node (node3a) [startstop,below of = node2, xshift=-4cm] {$f_x$ and $f_y$ exist at $(a,b)$};
\node (node3b) [startstop,below of = node2, xshift=4cm] {$f$ is continuous at $(a,b)$};
\node (node4) [process,below of = node2, yshift=-2.5cm] {$f(x,b)$ is continuous at $x = a$ and $f(a,y)$ is continuous at $y = b$.};

\draw [->, line width = .5mm] (node1) -- node[anchor=east] {(1)} (node2);
\draw [->, line width = .5mm] (node2) -- node[anchor=west, xshift=-1.5cm, yshift=.2cm] {(2)} (node3a);
\draw [->, line width = .5mm] (node2) -- node[anchor=east, xshift=1.5cm, yshift=.2cm] {(3)}  (node3b);
\draw [->, line width = .5mm] (node3a) -- node[anchor=west, xshift=-1.5cm, yshift=-.2cm] {(4)}  (node4);
\draw [->, line width = .5mm] (node3b) -- node[anchor=east, xshift=1.5cm, yshift=-.2cm] {(5)} (node4);
\end{tikzpicture}
\end{center}
\end{minipage}


\medskip

We list the justifications for the implications in the previous diagram.
\begin{enumerate}[(1)]
\item This implication is Theorem \ref{thm:31thm1}.

\medskip

\item We need the functions $f_x$ and $f_y$ to exist in order to define $L(x,y)$ in the first place. Also, if $f$ is \underline{well-approximated}---in the sense of \eqref{eq:31.2}---by a linear function $L(x,y)$, then $f_x$ and $f_y$ have to exist and $L(x,y)$ has to be given by \eqref{eq:31.1}. We discussed this argument in the section on Tangent Planes. We encourage you to revisit this argument now: if we fix $y=b$ and consider the values of $x$ near $a$, what does the condition (\ref{eq:31.2}) say about $f(x,b)$?
\medskip

\item This is discussed below in Theorem \ref{thm:31thm2}.

\medskip

\item This follows from single variable calculus applied to the (single variable) functions $f(x,b)$ and $f(a,y)$.

\medskip

\item If $\lim_{(x,y) \rightarrow (a,b)} f(x,y) = f(a,b)$ then in particular $\lim_{x  \rightarrow a} f(x,b) = f(a,b)$. The similar statement for $b$ and $y$ is true. 
\end{enumerate}

\begin{theorem}\label{thm:31thm2}
If $f(x,y)$ is differentiable at $(a,b)$, then it is continuous at $(a,b)$.
\end{theorem}


\begin{proof}
We want to show that
$$
\lim_{(x,y) \rightarrow (a,b)} f(x,y) = f(a,b),
$$
under the assumption that $f$ is differentiable at $(a,b)$.  Since the linearization $L(x,y)$ is a ``good approximation'' of $f(x,y)$ at $(a,b)$, in the sense that
$$
f(x,y) = L(x,y) + o(\sqrt{(x-a)^2 + (y-b)^2})
$$
and $L(x,y)$ \underline{is} continuous at $(a,b)$, we expect that the same should be true for $f(x,y)$.
However, we should write a rigorous proof of this fact! We will do that now.

\medskip
Let us define $R(x,y) = f(x,y) - L(x,y)$, the difference between $f$ and its linearization.  Then, since $f$ is differentiable at $(a,b)$, we have
$$
\lim_{(x,y) \rightarrow (a,b)} \frac{R(x,y)}{\sqrt{(x-a)^2 + (y-b)^2}} = 0.
$$
So, let $\epsilon = 1$ (say). Then, there exists some $\delta = \delta (\epsilon) > 0$ such that for all $(x,y)$ with $0 < \sqrt{(x-a)^2 + (y-b)^2} < \delta$ we have
$$
\frac{|R(x,y)|}{\sqrt{(x-a)^2 + (y-b)^2}} < 1,\hbox{ so that } 0 \leq |R(x,y)| < \sqrt{(x-a)^2 + (y-b)^2},
$$
for the same values of $(x,y)$. Since $\lim_{(x,y) \rightarrow (a,b)} \sqrt{(x-a)^2 + (y-b)^2} = 0$, the above inequality (together with the Squeeze Theorem) implies
$$
\lim_{(x,y) \rightarrow (a,b)} R(x,y) = 0.
$$
But, using the definition of $R(x,y)$, this gives
\begin{eqnarray*}
\lim_{(x,y) \rightarrow (a,b)} f(x,y) & = & \lim_{(x,y) \rightarrow (a,b)}  L(x,y) + \lim_{(x,y) \rightarrow (a,b)} R(x,y) \\[1ex]
\quad & = & f(a,b) + 0 \\[1ex]
\quad & = & f(a,b).
\end{eqnarray*}
\end{proof}

\begin{example}
Let 
$$
f(x,y)=\begin{cases}
x^2+ y^2 &\hbox{ if at least one of }x,y\hbox{ is rational,}
\\
0 &\hbox{ otherwise.}
\end{cases}
$$
Then $f(x,y)$ is differentiable at $(0,0)$ and its linear approximation at that point is $L(x,y)=0$. However, $f$ is definitely not $C^1$! In fact, $f$ is neither continuous nor differentiable at any point other than $(0,0)$. Think about how you might try to prove this. We will have a worked example of this type in the next section.
\end{example}



\subsection{Differentials}
Linear approximation can be interpreted in terms of \underline{differentials} similar to those in single-variable calculus. Letting $z = f(x,y)$, we write
$$
dz = df = \frac{\partial z}{\partial x} dx + \frac{\partial z}{\partial y} dy,
$$
so that, for fixed values of $(x,y)$, we interpret this as an expression for $dz$ as a linear function of $dx$ and $dy$. We will use this idea in applications of differentials to linear approximation and error estimates in the Worked Problems Section. We strongly encourage you to read through these examples.

\section*{Worked Problems}
\begin{enumerate}[1.]
\item {\it Find the linear approximation to $f(x,y) = x e^{3x - 2y}$ at $(a,b) = (2,3)$. Use it to evaluate the approximate value of $f(2.05, 3.01)$.}

\bigskip

\noindent
We have,
$$
f_x (x,y) = e^{3x - 2y} + 3x e^{3x - 2y}, \ f_y (x,y) = -2x e^{3x - 2y}
$$
so that,
$$
f(2,3) = 2, \ f_x (2,3) = 7, \ f_y (2,3) = -4,
$$
which gives a linearization for $f(x,y)$ at $(2,3)$ as
$$
L(x,y) = 2 + 7 (x-2) - 4(y-3).
$$
Hence, our approximation gives
$$
f(2.05, 3.01) \approx L(2.05, 3.01) = 2 + 7 \cdot 0.05 - 4 \cdot 0.01 = 2.31.
$$

\bigskip

\item {\it The radius and height of a cylindical container are measured with accuracy $1\%$. What is the maximal percentage error in the calculated (based on the measurements) volume of the container?}

\bigskip

We have $V = \pi r^2 h$.  The associated differential is given by
$$
dV = (2 \pi h r) dr + (\pi r^2) dh.
$$
For our measurement to to have within $1\%$ accuracy, this means that
$$
|dr| \leq \frac{1}{100} r, \, \, |dh| \leq \frac{1}{100} h.
$$
Consequently,
$$
|dV| \leq \big|2 \pi h r \cdot \frac{1}{100} r \big| + \big|\pi r^2 \cdot \frac{1}{100} h \big| = \frac{2 \pi r^2 h}{100} + \frac{\pi r^2 h}{100} = \frac{3}{100} V.
$$
Therefore, the maximal percentage error is $3 \%$. 

\bigskip

\end{enumerate}
\section*{Practice Problems}
\begin{enumerate}[1.]
\item
{Let $f(x,y)=ye^{xy^2}$.

\medskip
\noindent
(a) Find the equation of the tangent plane to the graph of the function
$f(x,y)$ at the point $(x,y)=(0,2)$.}


\medskip\noindent
{(b) Use linear approximation at $(0,2)$ to give an approximate value of
$f(-0.2, 2.01)$.}

\medskip\noindent
{(c) Find $f_{12}(x,y)$.}




\item {Let $z = \frac{\sin (x)}{\cos(y)} = \sin (x) \sec (y)$.

\medskip
\noindent
(a) Find the general expression for the differential of $z (x,y)$ for $x$ and $y$ in the interval $[0, \frac{\pi}{2})$. 
}

{
\medskip
\noindent
(b) Now, utilizing the differential calculated in Part (a),  approximate the value of the function at $(a,b) = (0.1,0.3)$ using a nearby point.}

{
\medskip
\noindent
(c) Rather than utilizing the the linearization formula from Lecture Notes 3.1, utilize first order Taylor approximation for $\sin$ and $\cos$ to give another estimation of the function at $(0.1,0.3)$.  Do your approximations agree?}

\medskip

\item{Suppose that $f(x,y)$ is a differentiable function of $x$ and $y$,  and that $ x = x(s,t)$ and $y = y(s,t)$ are themselves differentiable functions of $s,t$.

\medskip
\noindent
(a) Using the Multivariable Chain Rule (see \textit{Lecture Notes 2.6}), find a general formula for the linearization of $f(x(s,t), y(s,t))$ for $(s,t)$ near a point $(a,b)$ in terms of the first-order partial derivatives of the functions $f$, $x$ and $y$. 
}


\medskip
\begin{definition}
 A function $g (x,y)$ of two-variables is called \textbf{Lipschitz} if there exists a number $C > 0$ such that, given any pair of points $(x_0, y_0)$ and $(x_1,y_1)$ in $\mathbb{R}^2$, one has
$$
\big\vert g(x_1, y_1) - g(x_0, y_0) \big\vert \leq C \sqrt{(x_1 - x_0)^2 + (y_1 - y_0)^2}.
$$
The smallest value of $C > 0$ for which the above inequality holds is known as the \textbf{Lipschitz constant} of $g$.
\end{definition}

{
\medskip
\noindent
(b) Suppose that $f(x,y)$, $x(s,t)$ and $y(s,t)$ are all functions whose first-order partial derivatives are bounded by some number $M > 0$.  By \textbf{bounded}, we mean that:
$$
\big\vert \frac{\partial f}{\partial x} (x,y) \big\vert, \, \big\vert \frac{\partial f}{\partial y} (x,y) \big\vert \leq M, \textrm{ for every pair } (x,y) \in \mathbb{R}^2,
$$
$$
\big\vert \frac{\partial x}{\partial s} (s,t) \big\vert, \, \big\vert \frac{\partial x}{\partial t} (s,t) \big\vert , \,  \leq M, \textrm{ for every pair } (s,t) \in \mathbb{R}^2.
$$
$$
\big\vert \frac{\partial y}{\partial s} (s,t) \big\vert, \, \big\vert \frac{\partial y}{\partial t} (s,t) \big\vert , \,  \leq M, \textrm{ for every pair } (s,t) \in \mathbb{R}^2.
$$
Prove that for any $(a,b) \in \mathbb{R}^2$, the linearization of $f(x(s,t), y(s,t))$ for $(s,t)$ near $(a, b)$ is Lipschitz, with Lipschitz constant $C \leq 4 M^2$.

\medskip
\noindent
Hint: this is tricky! You will need four tools to prove this. They are (in order of likely use): the formula you obtained from Part (a.),  the triangle inequality (used twice), the bound on the first-order partial derivatives,  and the inequality 
$$|U| + |V| \leq 2 \max \{|U|, |V| \} \leq 2 \sqrt{|U|^2 + |V|^2},$$
which is valid for any two real numbers $U,V$ (prove this to yourself!)
}

\end{enumerate}

\begin{remark}
We will discuss proofs involving differentiability in more detail in the following Section. We also postpone Practice Problems asking you to prove the differentiability of given functions $f(x,y)$ to the next part of the notes.
\end{remark}





