\section{Higher Order Derivatives}
If a function $f(x_1,...,x_n)$ has a partial derivative $\frac{\partial f}{\partial x_j} (x_1,...,x_n)$, we can then try to find the partial derivatives of $\frac{\partial f}{\partial x_j}$,  such as $\frac{\partial}{\partial x_i} (\frac{\partial f}{\partial x_j})$ for any $1 \leq i \leq n$.  As a simple example:
$$
 f(x,y) = x^2 y^3 \Rightarrow \frac{\partial f}{\partial x} (x,y) = 2 x y^3 \Rightarrow \frac{\partial}{\partial y} \left( \frac{\partial f}{\partial x} \right) (x,y) = \frac{\partial}{\partial y} (2 x y^3) = 6 x y^2.
$$

\bigskip

\noindent
\underline{Notation}: there are several ways to denote higher order partials. For example, if  $z = f(x,y)$, we can write
$$
\frac{\partial}{\partial x} \left( \frac{\partial z}{\partial y}\right)  = \frac{\partial^2 z}{\partial x \partial y}  = f_{21} (x,y) = f_{yx} (x,y).
$$
Notice how the order of differentiation is indicated in each type of notation.  For example,
$$
\underbrace{\frac{\partial^3 f}{\partial z \partial x \partial y} = \frac{\partial}{\partial z} \left [\frac{\partial}{\partial x} \left( \frac{\partial f}{\partial y} \right) \right]}_{\textrm{differentiate first in } y \textrm{ then } x  \textrm{ then } z   } \quad \quad \quad \underbrace{f_{31} = (f_3)_1}_{\textrm{ differentiate in } x_3 \textrm{ then } x_1}
$$
The symbol $\partial^{n}$ for $n = 1,2,3,...$ appearing in the numerator indicates the \textbf{order} of the derivative--the total number of times we differentiate.
For iterated derivatives with respect to the same variable, we will use 
$$
f_{11}=f_{xx}=\frac{\partial^2 f}{\partial x^2}\, ,\ \ f_{yyyx}=\frac{\partial^4 f}{\partial x\,\partial y^3}\, ,\ \ \hbox{and so on.}
$$

\medskip

These are the conventions that we will use, but other textbooks and resources might reverse the order of subscripts, for example $f_{31} = (f_1)_3$ instead of $f_{31} = (f_3)_1$. In many cases, the order of differentiation does not matter---we discuss this in the following subsection. In those cases where it does matter, it is better to use notation such as $(f_1)_3$ or $\frac{\partial}{\partial x} ( \frac{\partial z}{\partial y})$ that indicates the order of differentiation explicitly.


The following example illustrates a scenario where
$
\frac{\partial^2 f}{\partial x \partial y} = \frac{\partial^2 f}{\partial y \partial x}.
$

\begin{example}
Find all second order partial derivatives of $f(x,y) = x^2 + x \sin y$. 
\end{example}

\bigskip
We have
\begin{eqnarray*}
\frac{\partial f}{\partial x} = 2x + \sin y & \quad \quad & \frac{\partial f}{\partial y} = x \cos y
\end{eqnarray*}
so that
\begin{eqnarray*}
\frac{\partial^2 f}{\partial x^2} = \frac{\partial}{\partial x} \left( \frac{\partial f}{\partial x} \right) = 2 & \quad \quad & \frac{\partial^2 f}{\partial y \partial x} = \frac{\partial}{\partial y} \left( \frac{\partial f}{\partial x} \right) = \cos y \\[1ex]
\frac{\partial^2 f}{\partial x \partial y} = \frac{\partial}{\partial x} \left( \frac{\partial f}{\partial y} \right) = \cos y & \quad \quad & \frac{\partial^2 f}{\partial y^2} = \frac{\partial}{\partial y} \left( \frac{\partial f}{\partial y} \right) - x \sin y
\end{eqnarray*}



\subsection{Equality of Mixed Partials} The previous example illustrates the following more general rule.

\begin{theorem}[Equality of Mixed Partials]
Suppose two mixed $n$-th order partial derivatives of $f(x_1,..,x_n)$ involve the same differentiations but in different order. If:
\begin{itemize}
\item these partials are continuous at a point $P$ in $\mathbb{R}^n$; and,

\smallskip

\item the function $f$ and all partials of $f$ of order \underline{less than} $n$ are continuous on a \underline{neighbourhood} of $P$,
\end{itemize}
then those partial derivatives are equal at $P$.
\end{theorem}


\medskip
Here are a few comments regarding the assumptions of the theorem.

\begin{enumerate}[1.]
\item Typically, we will apply the theorem to functions which are linear combinations or compositions of rational functions, trigonometric functions, exponentials, logarithms, etc. All partial derivatives of such functions are also combinations or compositions of similar functions; therefore, they are continuous at all points where they are defined. In such cases the assumptions of the theorem are satisfied for all $P$ such that the $n$-th order partials of $f$ are defined on some neighbourhood of $P$. For example, if
$$
f(x,y,z) = e^{x/y^2} \sin \bigg(\frac{x^3 + 5 y^2 + z^2}{x - z} \bigg),
$$
we know from the theorem (even without computing any derivatives!) that
$$
f_{xyz} = f_{zyx} = f_{yzx} = \cdots 
$$
at all points $(x,y,z)$ where $y \neq 0$ and $x \neq z$.

\medskip

\item It is possible to set up a function (using multiple formulas) where the assumptions of the theorem fail and the mixed partials are \textit{not} equal. For example, let
$$
f(x,y) =
\begin{cases}
\frac{xy (x^2 - y^2)}{x^2 + y^2}, &\textrm{ if } (x,y) \neq (0,0) \\[1ex]
0 & \textrm{ if } (x,y) = (0,0)
\end{cases}
$$
Then, a calculation shows $f_{12} (0,0) \neq f_{21} (0,0)$. See the practice problems for more details.
\end{enumerate}

\subsection{Idea of the Proof}
Let us consider the special case of functions $f(x,y)$ of two variables and try to show that $f_{xy} = f_{yx}$ under the assumptions of the theorem. Let
$$
Q(x,y, \Delta x, \Delta y) = f(x + \Delta x, y + \Delta y) - f(x, y + \Delta y) - f(x + \Delta x, y) + f(x,y).
$$




\begin{figure}[H]\label{25fig1}
\centering
\includegraphics[width= .6\textwidth]{25fig1.png}
\caption{The function $Q$ adds up the values of $f$ at the four vertices of a rectangle, with alternating $\pm$ signs as indicated.}
\end{figure}



\underline{Assume} (for the moment) that, in addition to the other assumptions of the theorem, the following limit in two variables exists:
\begin{equation}\label{eq:25eq1}
\lim_{(\Delta x, \Delta y) \rightarrow (0,0)} \frac{Q(x, y, \Delta x, \Delta y)}{\Delta x \Delta y} = L, \textrm{ for some real number } L.
\end{equation}
We claim that if \eqref{eq:25eq1} holds, then both $f_{xy} (x,y)$ and $f_{yx} (x,y)$ are equal to $L$.  To prove this, let us show that
\begin{equation}\label{eq:25eq2}
f_{yx} (x,y) = L\, ; 
\end{equation}
the calculation for $f_{xy}$ is similar.

\medskip
First, recall that
$$
f_y (x,y) = \lim_{\Delta y \rightarrow 0} \frac{1}{\Delta y} \big[f(x, y + \Delta y) - f(x,y) \big]
$$
so that
$$
f_y (x + \Delta x, y) = \lim_{\Delta y \rightarrow 0} \frac{1}{\Delta y} \big[ f(x + \Delta x, y + \Delta y) - f(x + \Delta x, y)\big].
$$
Then, 
\begin{eqnarray*}
f_{yx} (x,y) & = & \lim_{\Delta x \rightarrow 0} \frac{1}{\Delta x} \big[f_y (x + \Delta x, y) - f_y(x,y) \big] \\[1ex]
\quad & = & \lim_{\Delta x \rightarrow 0} \frac{1}{\Delta x} \bigg[\lim_{\Delta y \rightarrow 0} \frac{1}{\Delta y} \big[f(x + \Delta x, y + \Delta y) - f(x + \Delta x, y) \big] \\[1ex]
\quad & \quad & \quad \quad \quad \quad \quad - \lim_{\Delta y \rightarrow 0} \frac{1}{\Delta y} \big[f(x, y + \Delta y) - f(x,y) \big] \bigg] \\[1ex]
\quad & = & \lim_{\Delta x \rightarrow 0} \frac{1}{\Delta x} \bigg[ \lim_{\Delta y \rightarrow 0} \frac{1}{\Delta y} Q(x,y, \Delta x, \Delta y) \bigg]
\end{eqnarray*}
Now, if we assume equation \eqref{eq:25eq1} so that the two-variable limit exists, then the last equality implies \eqref{eq:25eq2}.

\medskip
In reality, we do not yet know whether the limit \eqref{eq:25eq1} exists. This is why the above is just \underline{the idea} of the proof and not the complete argument.  However, this can be fixed by using the Mean Value Theorem instead. The assumption that the partials $f_x,f_y$ are continuous on a neighbourhood of $P$ is needed to justify the application of the Mean Value Theorem to these partials, and the assumption that $f_{xy}$ and $f_{yx}$ are continuous at $P$ is needed when we take the limit at $P$. See your textbook for details.




\subsection{Partial differential equations: a quick introduction}
A \textit{Partial Differential Equation} (PDE) is an identity involving the partial derivatives of some function.  This is a large area of study (with separate courses dedicated to it).  In this small subsection, we give two simple examples.

\begin{example}
Verify that $z = \frac{2x-y}{x + y}$ satisfies the PDE $x \frac{\partial z}{\partial x} + y \frac{\partial z}{\partial y} = 0$.
\end{example}

\medskip
A calculation (do it!) shows that,
$$
\frac{\partial z}{\partial x} = \frac{3 y}{(x + y)^2}, \quad \quad \frac{\partial z}{\partial y} = -\frac{3x}{(x + y)^2}.
$$
Thus
$$
x \frac{\partial z}{\partial x} + y \frac{\partial z}{\partial y} = x \bigg(\frac{3y}{(x + y)^2} \bigg) + y \bigg(\frac{-3x}{(x + y)^2} \bigg) = 0.
$$
(As an additional exercise, you can check that $z=\frac{ax+by}{x+y}$ satisfies the same equation for any fixed $a,b\in\mathbb{R}$.)

\begin{example}
If $f$ and $g$ are twice-differentiable functions of one variable, prove that $w(x,t) = f(x + ct) + g(x - ct)$ (where $c \neq 0$ is some constant) satisfies the \textbf{wave equation}
$$
\frac{\partial^2 w}{\partial t^2} = c^2 \frac{\partial^2 w}{\partial x^2}.
$$
\end{example}

\medskip
A calculation (do it!) shows that
$$
w_{xx} = f'' (x + ct) + g''(x - ct), \quad  w_{tt} = c^2 f''(x + ct) + c^2 g'' (x - ct),
$$
which shows that $w_{xx} = c^2 w_{tt}$.

\section*{Practice Problems}

Warning: the calculations in Questions 2 and 3 below are relatively long.

\begin{enumerate}[1.]
\item
Let $f(x,y)=x\sin(xy^2)$.

\medskip

(a) Find all first and second order partial derivatives of $f$.

\medskip

(b) Find the equation of the tangent plane to the surface $z=f(x,y)$ at the point
where $x=\pi$, $y=1$.

\bigskip

\item A function $f(x_1,...,x_n)$ of $n$-many variables is said to be \textbf{harmonic} if it satisfies the equation,
$$
\frac{\partial^2 f}{\partial x_1^2}  + \cdots + \frac{\partial^2 f}{\partial x_n^2} = 0.
$$
Demonstrate that the following functions are harmonic:

\medskip

(a) $f(x,y) = \frac{x}{x^2 + y^2}$ for $(x,y) \neq 0$;

\medskip

(b) $f(x,y,z) = \frac{x}{(x^2 + y^2 + z^2)^{3/2}}$ for $(x,y,z) \neq 0$ (a longer calculation);

\medskip

(c) $f(x_1,...,x_n) = \frac{1}{(x_1^2 + \cdots x_n^2)^{(n/2) - 1}}$ for $(x_1,...,x_n) \neq (0,...,0)$ and $n \geq 3$.


\bigskip




\item 
{\it  Let $$F(x,y)= \left\{\begin{matrix}
\displaystyle{\frac{xy(x^2-y^2)}{x^2+y^2}}&\hbox{ if }(x,y)\neq (0,0)\\
0&\hbox{ if }(x,y)=(0,0)\\
\end{matrix}
\right.
$$
Demonstrate carefully that both $F_{12} (0,0)$ and $F_{21} (0,0)$ exist and \underline{are not equal}.  Why does this not violate Theorem 2.5.2?

\smallskip
\noindent
Hint: we advise that you proceed by obtaining the following in order:
\begin{itemize}
\item $F_1(0,y)$ for all $y\in\rr$ and $F_2(x,0)$ for all $x\in\rr$,
\item $F_{12}(0,0)$ and $F_{21}(0,0)$.
\end{itemize}
If done correctly, you will obtain that $F_{21}(0,0)=1$
and $F_{12}(0,0)=-1$. 
}

\begin{remark}
Problem $3$ is a very common example of a function $F(x,y)$ for which $F_{11} \neq F_{22}$. You will likely see it everywhere if you look online.  As far as we can say, the earliest attribution of this belongs to Italian mathematician Giuseppe Peano (b.~1858).
\end{remark}



\end{enumerate}




