\section{The Method of Lagrange Multipliers}
Our goal in this section is to determine the extreme values of a multivariate function $f({\bf x})$ subject to a constraint $g({\bf x}) = c$.  The intended application is minimizing or maximizing $f$ on a region $D$ whose boundary is defined by this constraint.



For a point $\mathbf{a} \in \mathbb{R}^n$, we say that
$f$ has a \underline{local minimum} at $\mathbf{a}$ subject to the constraint $g({\bf x})=c$ if:
\begin{itemize}
\item $g(\mathbf{a})=c$,
\item there is a neighbourhood $U$ of $\mathbf{a}$ such that $f (\mathbf{x}) \geq f (\mathbf{a})$ for all $\mathbf{x} \in U$ such that ${\bf x}$ belongs to the domain of $f$ and that $g(\mathbf{a})=c$.
\end{itemize}

The definition of a local maximum is similar, but with $f (\mathbf{x}) \leq f (\mathbf{a})$. For a global minimum or maximum of $f$ subject to a constraint $g({\bf x}) = c$, we need the inequality in question to hold for all ${\bf x}$ in the domain of $f$ such that $g(\mathbf{a})=c$, not just in some neighbourhood of the given point.

\subsection{Some diagrams}



\begin{figure}[h]
\centering
\includegraphics[width =.7 \linewidth]{43fig1.png}
\caption{A zoomed-out picture showing the level curve $g(x,y) = C$ and its intersection with the level curves of $f(x,y)$.}
\end{figure}

\begin{figure}[h]
\centering
\includegraphics[width =.5 \linewidth]{43fig2.png}
\caption{We zoom in on the critical point $P$ from the previous figure.  The key observation is that the gradient vector $\nabla f (P)$ lies in the span of the gradient vector $\nabla g (P)$.  This means that $\nabla f (P) = \lambda g (P)$ for some real number $\lambda \neq 0$.}
\end{figure}


\begin{figure}[h]
\centering
\includegraphics[width =.5 \linewidth]{43fig3.png}
\caption{If we instead zoom in on a non-critical point $Q$, we see that the gradient vector $\nabla f (Q)$ no longer lies in the span of the gradient vector  $\nabla g (Q)$.}
\end{figure}



The main idea of our approach is easiest to explain when $f$ and $g$ are functions of two variables. Assume that $g(P)=c$ at some point $P$, that both $f(x,y)$ and $g(x,y)$ are $C^1$ on some neighbourhood of $P$, and that $\nabla g(P)\neq {\bf 0}$. Then, again on some neighbourhood of $P$, the equation $g(x,y) = c$ defines a curve $\mathcal{C}$ that can be parametrized by a vector-valued $C^1$ function $(x(t),y(t))$. (This follows from the Implicit Function Theorem, see Section 3.4.) Since $P$ is a point on $\mathcal{C}$, we have $P=(x(t_0),y(t_0))$ for some $t_0$.


Suppose that $f(x,y)$ restricted to $\mathcal{C}$ has a local maximum or minimum at $P$. Then $f(x(t),y(t))$ has a local minimum or maximum at $t=t_0$, so that
$$
0=\frac{d}{dt}f(x(t),y(t))\Big|_{t=t_0}=f_x(P)x'(t_0)+f_y(P)y'(t_0).
$$
This means that $\nabla f(P)$ should be perpendicular to $\langle x'(t_0),y'(t_0)\rangle$. But $\langle x'(t_0),y'(t_0)\rangle$ is tangent to $\mathcal{C}$, therefore perpendicular to $\nabla g(P)$! Therefore the two gradient vectors should be parallel to each other.



Intuitively, we consider how $\mathcal{C}$ is situated relative to the level curve of $f$ near $P$. At the point $P$ on $\mathcal{C}$ where the function $f(x,y)$ is maximized or minimized, $\mathcal{C}$ should be tangent to the level curve of $f$. (Think about climbing part of the way up a hill, then levelling off before you reach the top, then coming down.) This means that the corresponding normal vectors $\nabla f$ and $\nabla g$ should be parallel at that point. Conversely, if $Q$ is a point with $|nabla g\neq{\bf 0}$ where the level curves of $f$ and $g$ are not tangent to each other, then they have to cross each other, so we have a situation with $f$ taking values both larger than $f(Q)$ and greater than $f(Q)$ in any neighbourhood of $Q$. 



\subsection{Procedure}
We assume that $f$ and $g$ are $C^1$ on an open set containing $P$, and that $\nabla g (P) \neq \mathbf{0}$. Let $\mathcal{C}$ be the curve $g(x,y) = g(P) = c$. Suppose that $f(x,y)$, restricted to $\mathcal{C}$, has a local minimum or maximum at $P$. Then $\nabla f (P) = \lambda g (P)$ for some $\lambda \in \mathbb{R}$. Equivalently, there is a $\lambda \in \mathbb{R}$ such that $(P,\lambda) \in \mathbb{R}^2 \times \mathbb{R}$ is a critical point of the ``Lagrange function''
$$
L(x,y,\lambda) = f(x,y) - \lambda (g(x,y) - c), \textrm{ on } \mathbb{R}^3.
$$

We have the following procedure for minimizing or maximizing the function $f(x,y)$ on the curve $\mathcal{C}: = \{(x,y) : g(x,y) = c\}$.

\medskip
\noindent
\fbox{
\begin{minipage}{\textwidth}

\medskip
\begin{center}
\textbf{Method of Lagrange Multipliers}
\end{center}



\begin{enumerate}
\item Find all points $(x,y)$ on $\mathcal{C}$ where:

\medskip

\begin{itemize}
\item $\nabla f(x,y) = \lambda \nabla g (x,y)$ for some $\lambda \in \mathbb{R}$;

\smallskip

\item $\nabla g(x,y) = 0$;

\smallskip

\item $\nabla f(x,y)$ or $\nabla g(x,y)$ does not exist;

\smallskip

\item $\mathcal{C}$ has an endpoint.
\end{itemize}

\medskip

\item Evaluate $f(x,y)$ at all these ``candidate points''.

\medskip

\item Choose the largest and smallest values of $f(x,y)$ at each of these points.

\end{enumerate}
\end{minipage}
}

\bigskip
We will implement this method in Worked Problems Section.

\subsection{Higher dimensions}
In higher dimensions (for functions of $n>2$ variables), the procedure is still similar, but may be more complicated.  For example, to minimize or maximize a function $f(x,y,z)$ on a surface with boundary given by a curve, we may need separate Lagrange multiplier procedures for both the surface and its boundary curve.


\section*{Worked Problems}
\begin{enumerate}[1.]
\item {\it Find the smallest and largest values of $f(x,y) = xy$ on the ellipse $4x^2 + y^2 = 16$.}

\medskip
We follow the Method of Lagrange Multipliers.
Let $g(x,y) = 4x^2 + y^2$. Then both $f$ and $g$ are $C^1$ and
$$
\nabla f (x,y) = \langle y, x \rangle \quad \quad \nabla g (x,y) = \langle 8x, 2y \rangle.
$$
First, notice that $\nabla f$ and $\nabla g$ exist everywhere, and that the curve $g(x,y) = 4x^2 + y^2 = 16$ is closed (so it has no endpoints).  Further,  if $\nabla g (x,y) = \mathbf{0}$, then $x = 0$ and $y = 0$. But then $4 x^2 + y^2 = 0 \neq 16$.  Therefore no points on our ellipse satisfy the equation $\nabla g(x,y) = 0$.

\medskip

Suppose now that $\nabla f (x,y) = \lambda \nabla g (x,y)$ on for some point $(x,y) \in \mathcal{C}$. This generates the system of equations
\begin{equation}\label{eq:43eq1}
y = \lambda \cdot 8x, \, \, x = \lambda \cdot 2y
\end{equation}
\begin{equation}\label{eq:43eq2}
4x^2 + y^2 = 16
\end{equation}
From \eqref{eq:43eq1} we have
$$
x = 2 \lambda y = 2 \lambda (8 \lambda x) = 16 \lambda^2 x.
$$
So, either $x = 0$ or else $16\lambda^2=1$, $\lambda = \pm \frac{1}{4}$.

\medskip

If $x = 0$, then $y =\pm  \sqrt{16 - 0} = \pm 4$ from \eqref{eq:43eq1}.  At the points $(0,\pm 4)$ we have $\nabla f = \langle \pm 4, 0 \rangle$ while $\nabla g = \langle 0, \pm 8 \rangle$.  These vectors are not parallel; so, in particular, we know that we cannot have a minimum or maximum at $(0,\pm 4)$. (Alternatively, it would also be fine to keep these points as {\it possible} minima or maxima, evaluate  $f$ at these points, and keep these values for the final comparison later on.) 

\medskip
If $\lambda = \pm \frac{1}{4}$, we return to \eqref{eq:43eq1} and find that
$$
y = \pm \frac{1}{4} \cdot 8x = \pm 2x \Rightarrow y^2 = 4 x^2 \Rightarrow 4x^2 + y^2 = 8x^2 = 16.
$$
Then, $x = \pm \sqrt{2}$ and $y = \pm 2 x = \pm 2 \sqrt{2}$. Note that all four combinations of $\pm$ are possible!

\medskip
To conclude, we simply evaluate $f$ at each of these points
$$
f(\sqrt{2}, 2 \sqrt{2}) = f(-\sqrt{2}, -2 \sqrt{2}) = 4 \, \longleftarrow \textrm{(max)}
$$
$$
f (- \sqrt{2}, 2 \sqrt{2}) = f(\sqrt{2}, -2 \sqrt{2}) = -4 \, \longleftarrow \textrm{(min)}.
$$

\bigskip

\item{\it Find the minimum and maximum values of $f(x,y) = xy + 2y$ on the region $D : = \{(x,y) : x^2 + 2y^2 \leq 4 \}$.}

\medskip

We use our earlier procedure for minimizing/maximizing $f$ on a compact domain $D$, with Lagrange multipliers used to find the min/max on the boundary of $D$.

\medskip
The function $f$ is $C^1$ and $\nabla f(x,y) = \langle y, x+2 \rangle$. If $\nabla f = \mathbf{0}$, then $x = -2$ and $y = 0$ so that $f(-2,0) = 0$.  We record this information and move on to the boundary.

\medskip
For the ellipse, we let $g(x,y) = x^2 + 2y^2$, which is $C^1$ with $\nabla g (x,y) = \langle 2x, 4y \rangle$.  Clearly, $\nabla f$ and $\nabla g$ both exist everywhere and the ellipse has no endpoints.  Further, if $\nabla g = 0$ then $x = y = 0$ and $(0,0)$ is not on the elliptic boundary.

\medskip
We set up the equations for $\nabla f = \lambda \nabla g$ and $g(x,y) = 4$:
\begin{equation}\label{eq:43eq3}
y = 2 \lambda x, \, \, x + 2 = 4 \lambda y
\end{equation}
\begin{equation}\label{eq:43eq4}
x^2 + 2y^2 = 4
\end{equation}
We have
$$
x(x+2) = x (4 \lambda y) = 2y (2 \lambda x) = 2 y^2 = 4 - x^2.
$$
This simplifies to
$$
x^2 + 2x = 4 - x^2 \Rightarrow x^2 + x - 2 = 0,
$$
so that $x = 1$ or $x = -2$.

\medskip
If $x = 1$, then $2y^2 = 3$, so that $y = \pm \sqrt{3/2}$. If $x = -2$, then $2y^2 = 0,$ so that $y = 0$. We have
$$
f(1, \sqrt{\frac{3}{2}}) = 3 \sqrt{\frac{3}{2}}, \, \, f(1, - \sqrt{\frac{3}{2}}) = -3 \sqrt{\frac{3}{2}}
$$
$$
f(-2, 0) = 0
$$
The maximum value is $f(1,\sqrt{\frac{3}{2}})=3 \sqrt{\frac{3}{2}}$,
 and the minimum value is $f(1, - \sqrt{\frac{3}{2}}) = -3 \sqrt{\frac{3}{2}}$.

\bigskip

\item {\it Find the minimum and maximum values of $f(x,y,z) = yz + xy$ on the sphere $x^2 + y^2 + z^2 = 4$.}

\medskip

Let $g(x,y,z) = x^2 + y^2 + z^2$, then $f$ and $g$ are $C^1$ everywhere and
$$
\nabla f (x,y,z) = \langle y, x+z, y \rangle , \, \nabla g (x,y,z) = \langle 2x, 2y, 2z \rangle.
$$
If $\nabla g = \mathbf{0}$, then $x = y = z = 0$ which is not on the sphere $x^2 + y^2 + z^2 = 4$. So, we set up our Lagrange multipliers. 

\medskip

For convenience, we take the equation $\nabla f = \frac{1}{2} \lambda \nabla g$--since, if $\lambda \in \mathbb{R}$ is a free parameter then $\frac{1}{2} \lambda \in \mathbb{R}$ is also free. This produces the equations
\begin{equation}\label{eq:43eq5}
y = \lambda x, \quad x+z = \lambda y, \quad y = \lambda z
\end{equation}
\begin{equation}\label{eq:43eq6}
x^2 + y^2 + z^2 = 4
\end{equation}
From \eqref{eq:43eq5}, we see that $\lambda x = y = \lambda z$.  So, either $x = z$ or $\lambda = 0$. 

\medskip

If $\lambda = 0$, then $y = 0$ and $x + z = 0$ so that $x = -z$. From the boundary condition \eqref{eq:43eq6}, we have $x^2 + 0^2 + (-x)^2 = 2x^2 = 4$ so that $x = \pm \sqrt{2} = -z$. We evaluate
$$
f(\sqrt{2}, 0, -\sqrt{2}) = f(-\sqrt{2}, 0, \sqrt{2}) = 0.
$$
If $x = z$, then from \eqref{eq:43eq5}
$$
x+z = 2x = \lambda y = \lambda^2 x \Rightarrow \lambda^2 x = 2x.
$$
So either $x = 0$ or $\lambda = \pm \sqrt{2}$. If $x = 0$ then $z = 0$ and $y = \pm 2$.  Evaluating,
$$
f(0,\pm2, 0) = 0.
$$
If instead $\lambda = \pm \sqrt{2}$ then $y = \pm \sqrt{x}$ and
$$
x^2 + 2x^2 + x^2 = 4x^2 = 4 \Rightarrow x = z = \pm 1 \Rightarrow y = \pm \sqrt{2}.
$$
Evaluating,
$$
f(1, \sqrt{2}, 1) = f(-1, -\sqrt{2}, -1) = 2 \sqrt{2} \, \longleftarrow \textrm{(max)}
$$
$$
f(-1, \sqrt{2}, -1) = f(1, -\sqrt{2}, 1) = - 2 \sqrt{2} \longleftarrow \textrm{(min)}.
$$
\end{enumerate}

\section*{Practice Problems}
\begin{enumerate}[1.]
\item Find the largest and smallest values of the function $f(x,y,z)=x+2y$ in the closed disc $x^2+y^2\leq 25$.

\medskip

\item Find the largest and smallest values of the function $f(x,y)=x^2-3x-\frac{y^2}{2}$ in the closed disc $x^2+y^2\leq 4$.

\medskip

\item Find the largest and smallest values of the function $f(x,y)=\frac{x}{y}$ in the closed region 
$x^2+(y-2)^2 \leq 1$.

\medskip


\item Let $A > 0$ be some real number. 
Find the point(s) on the surface $xyz= A^3$ that is closest to the origin, and calculate the minimum distance of this surface to the origin.  Your answer will depend upon the constant $A$.

\medskip


\item Find the largest and smallest values of $xy^2+yz^2$ in the ball $x^2+y^2+z^2\leq 1$.
\end{enumerate}




\vfill





