Constrained Optimization: Lagrange Multipliers

Finding the global minimum and maximum of a function $f(x,y)$ subject to a constraint $g(x,y)=c$ means finding the largest and smallest values that $f(x,y)$ takes over all points that satisfy this constraint. More specifically:

• Finding the global maximum of $f(x,y)$ subject to the constraint $g(x,y)=c$ means finding the point $P_0$ (if it exists) such that$f(P_{0})\geq f(P)$ for all other points $P$ satisfying $g(P)=c\text{.}$
• Finding the global minimum of $f(x,y)$ subject to the constraint $g(x,y)=c$ means finding the point $P_0$ (if it exists) such that$f(P_{0})\leq f(P)$ for all other points $P$ satisfying $g(P)=c\text{.}$

Note that minimum and maximum values take place where the contours of $f(x,y)$ and the constraint $g(x,y)=c$ are parallel or at the endpoints of the constraint. This is because at interior points on the constraint where the contour of $f$ is not parallel to the constraint, the value of $f$ can be increased or decreased by moving along the constraint. Note also that the contours of $f(x,y)$ are parallel to the constraint $g(x,y)=c$ only if $\nabla f(x,y)$ and $\nabla g(x,y)$ are parallel.

The above observations suggests the following method (called the method of Lagrange Multipliers) for finding global minimal and maxima:

1. Assume the constraint $g(x,y)=c$ is bounded and $g$ is continuous (see 4 below for what to do when it is not).

2. Find each point $P_0$ that satisfies both

   $$\begin{align*} \nabla f(x,y)\amp =\lambda \nabla g (x,y), \text{ and}\\ g(x,y)\amp=c. \end{align*}$$

   The first of these ensures the two gradient vectors are parallel at that point, and the second that the point satisfies the constraint. The number $\lambda$ is called a Lagrange Multiplier.

3. Evaluate $f(x,y)$ at each of these points and at the endpoints of the constraint: the largest will be the maximum and the smallest will be the minimum.
4. If the constraint $g(x,y)=c$ is not bounded (for example $xy=1$), then the end behavior of the values of $f(x,y)$ along the constraint needs to be analyzed and compared with the values of $f(x,y)$ at the points that were found before.

For a constraint given via an inequality $g(x,y)\leq c\text{,}$ we also need to consider the critical points inside the constraint, so the solution requires additional steps. Specifically:

1. Assume the region $g(x,y)\leq c$ is bounded is continuous (see 4 below for what to do when it is not).
2. Find each point $P_0$ that satisfies both $\nabla f(x,y)=\lambda \nabla g (x,y)$ and $g(x,y)=c\text{.}$
3. Find all points strictly inside the region $g(x,y)\leq c$ where $\nabla f$ is undefined or $0$ (the critical points).
4. Evaluate $f(x,y)$ at each of the points in the previous two steps: the largest will be the maximum and the smallest will be the minimum.
5. If the region $g(x,y)\leq c$ is not bounded, (for example $xy\leq 1$), then the end behavior of the values of $f(x,y)$ in the region needs to be analyzed and compared with the values of $f(x,y)$ at the points that were found before.