Math 1C: Calculus III
Part 1: Multidimensional Differential Calculus
Lesson 3: Lagrange Multipliers Introduction (Demonstrating a need for another method to Optimize a Function)
©Jeffrey A. Anderson

Key Terms List: Briggs Section 12.9 ()p. 951
□ Objective Function (p. 951)
□ Constraints of optimization function (p. 951)
□ Theorem 12.15: Parallel Gradients (Ball Park Theorem) (p. 952)
□ Procedure: Method of Lagrange Multipliers in Two Variables (p. 953)
□ Procedure: Method of Lagrange Multipliers in Three Variables (p. 955)

Lagrange Multipliers as a Method to Extrema with Constraints

We will now study a famous method to solve one specific instance of a much larger class of problems known as a constrained optimization problem. When trying to solve constrained optimization problems, we aim to optimize (either maximize or minimize)

 		Objective function: 		f(x,y) = (x^4)+(y^4)-4xy+1
 		
 subject to constraints
 		
 		Constraint function:		g(x,y)=(x^2)+(y^2)-1 = 0
 		
Below, we will motivate the method of Langrange Multipliers by visualizing the foundations of constrained optimization problems as they relate to level curves and gradient vectors. Let’s begin by reminding ourselves about the geometry of level curves and surfaces.

The Plot of a Function in 3D

Let’s consider the function;

fFunc[x_,y_]:=(x^4)+(y^4)-4x y +1;

We can use the Plot3D Command to graph this surface. Below is the code that accomplishes this plot.

f=(x^4)+(y^4)-4x y +1;
lm=3;
plot[1]=Plot3D[
	f,
	{x,-lm,lm},
	{y,-lm,lm},
	PlotRange->{-lm,lm},
	AxesLabel->{X,Y,Z},
	AxesOrigin->{0,0,0},
	AxesStyle->{Red,Green, Blue},
	ColorFunction->Function[z,Hue[0.2z+.3]],
	BoxRatios->{1, 1, 1},
	ClippingStyle->None,
	PlotLabel->Text[
			Style["Surface Plot: \n f(x,y) = (x^4)+(y^4)-4xy+1",20, TextAlignment->Center]
			],(*End: Text Command*)
	ImageSize->Full
](*End: Plot3D Command*)

In this problem, we are asked to find the optimal values on this surface, subject to:

g(x,y)=(x^2)+(y^2)-1

In other words, we want restrict the domain of our function f(x,y) to only the points that (x,y) that satisfy the relation g(x,y). Let’s now visualize the intersection as a path on the surface of f(x,y). We will highlight this path in blue to indicate the set of all point on the surface f(x,y) that satisfy the constraint. In other words, the blue path below is the same as the intersection between our curves graphed above.

g = (x^4)+(y^4)-4x y +1-z;
h =  (x^2)+(y^2)-1;
pathPlot = ContourPlot3D[
	{ g==0, h==0},
	{x,-2,2},
	{y,-2,2},
	{z,-2,5},
	MeshFunctions->{Function[{x,y,z,f},h-g]},
	MeshStyle->{{Thick,Blue}},
	Mesh->{{0}},
	ContourStyle->{Directive[Green,Opacity[0.5],Specularity[White,30]],Directive[Green,Opacity[0.0],Specularity[White,30]]},
	PlotLabel->Text[
Style["Surface plot of path (in blue) that marks the intersection between \n f(x,y) = (x^4)+(y^4)-4xy+1 and g(x, y) = (x^2) + (y^2) - 1",20, TextAlignment->Center]
			],(*End: Text Command*)
	ImageSize->Full
]

Our job in this problem is to find the maximum and minimum output values of the points on the blue curve. However, we can identify these points using our graph. The maximum points are the points at the top of this curve.

Show[
ContourPlot3D[
	{ g==0, h==0},
	{x,-2,2},
	{y,-2,2},
	{z,-2,5},
	MeshFunctions->{Function[{x,y,z,f},h-g]},
	MeshStyle->{{Thick,Blue}},
	Mesh->{{0}},
	ContourStyle->{Directive[Green,Opacity[0.5],Specularity[White,30]],Directive[Green,Opacity[0.0],Specularity[White,30]]}
],
ListPointPlot3D[{{(1/(√2)),(1/(√2)),((-1)/2)}, {((-1)/(√2)),((-1)/(√2)),((-1)/2)}},PlotStyle->Directive[Red,PointSize[0.035]]],
ListPointPlot3D[{{((-1)/(√2)),(1/(√2)),(7/2)}, {(1/(√2)),((-1)/(√2)),(7/2)}},PlotStyle->Directive[Black,PointSize[0.035]]],
Graphics3D[Text[StyleForm["MIN",FontSize->14,FontWeight->"Bold"],{(1/(√2)),(1/(√2)),((-1)/2)},{0,4}]],
Graphics3D[Text[StyleForm["MIN",FontSize->12,FontWeight->"Bold"],{((-1)/(√2)),((-1)/(√2)),((-1)/2)},{0,4}]],
Graphics3D[Text[StyleForm["MAX",FontSize->14,FontWeight->"Bold"],{((-1)/(√2)),(1/(√2)),(7/2)},{0,-4}]],
Graphics3D[Text[StyleForm["MAX",FontSize->12,FontWeight->"Bold"],{(1/(√2)),((-1)/(√2)),(7/2)},{0,-4}]],
ImageSize-> Full,
PlotLabel->Text[
Style["Plot of points that represent our MIN and MAX of \n f(x,y) subject to the constaint g(x,y)",20, TextAlignment->Center]
			](*End: Text Command*)
](*End: Show Command*)

Now that we’ve seen a graphical method to solve this problem (literally just pointing to the minimum and maximum on the graph), let’s create some algebraic techniques to help identify the exact location of these minimum and maximum values. 

We begin this process by graphing the level curves on this surface using the ContourPlot command that we studied in Section 11.1. Below, we command Mathematica to graph 10 separate level curves on the surface. We can see the pattern of the level curves by hovering our mouse over each line in the ContourPlot below. This will illuminate the z-axis values associated with each level curve.

plot[3] = 
ContourPlot[
	f,
	{x,-2,2}, 
	{y,-2,2},
	Contours->{-.99, -.75,-.50, -.25, 0,0.25,0.50,0.75,1.0,1.25,1.5, 1.75,2,2.25,2.5,2.75,3,3.5,4,4.5}, 
	ContourShading->None,
	PlotLabel->Text[
			Style["Level Curves: \n  f(x,y) = (x^4)+(y^4)-4xy+1",20, TextAlignment->Center]
			],(*End: Text Command*)
	ImageSize-> Full
](*End: ContourPlot Command*)

We can also visualize the region in the xy-plane corresponding to our constraint function g(x,y) = 0.

plot[5] = ContourPlot[
			(x^2)+(y^2)==1,
			{x,-1.6,1.6}, 
			{y,-1.6,1.6},
			ContourShading->None,
			ImageSize->Full,
			PlotLabel->Text[
			Style["Constraint Curve: \n  g(x,y) = (x^2)+(y^2)-1=0",20, TextAlignment->Center]
			](*End: Text Command*)
	]

Now, let’s super-impose the constraint curve onto of these level curves. In this case, we will graph the level curves of the function

f(x,y) = (x^4)+(y^4)-4xy+1

in grey (as we did above) and super impose a blue curve corresponding to the constraint

g(x,y)=(x^2)+(y^2)-1

Show[
	ContourPlot[
	f,
	{x,-2,2}, 
	{y,-2,2},
	Contours->{-.99, -.75,-.50, -.25, 0,0.25,0.50,0.75,1.0,1.25,1.5, 1.75,2,2.25,2.5,2.75,3,3.5,4,4.5}, 
	ContourShading->None
	],(*End: ContourPlot Command*) 

ContourPlot[
	(x^2)+(y^2)-1==0,
	{x,-2,2}, 
	{y,-2,2}, 
	ContourShading->None
],(*End: ContourPlot Command*) 

PlotLabel->Text[
			Style["Level Curves of  f(x,y) (in grey) \n AND constraint g(x,y) (in blue)",20, TextAlignment->Center]
			],(*End: Text Command*)
ImageSize-> Full
]

While this is very helpful from the standpoint of visaulizing the surface, we get little intuition about the level-curve behavior on this surface. In order to simultaneously graph the level curves on the surface, we can use the code below.

lm=2;
plot[2]=
Show[
	pathPlot ,
	ContourPlot3D[
	z==f,
	{x,-lm,lm},
	{y,-lm,lm},
	{z,-lm,10},
	AxesLabel->{X,Y,Z},
	AxesOrigin->{0,0,0},
	AxesStyle->{Red,Green, Blue},
	ContourStyle->Directive[Green],
	Mesh->{50}, 
	BoxRatios->{1, 1, 1}
],(*End: ContourPlot3D Command*)
ImageSize-> Full
]

Part 5.e calls for the level curve at k = 0.5. In this work, k = -0.5 is used instead. Show this plane, z = - 0.5 together with plot[2]

Show[
 ContourPlot[(x^2)+(y^2)==1,{x,-1.6,1.6}, {y,-1.6,1.6},ContourShading->None] ,
 ContourPlot[f,{x,-1.6,1.6}, {y,-1.6,1.6}, Contours->{ -0.5},ContourShading->None] ,
ImageSize-> Full
](*End: Show Command*)

Show[
 ContourPlot[(x^2)+(y^2)==1,{x,-2,2}, {y,-2,2},ContourShading->None] ,
 ContourPlot[f,{x,-2,2}, {y,-2,2}, Contours->{ 3.5},ContourShading->None],
ImageSize-> Full
](*End: Show Command*)

Method of Lagrange Multipliers

To find the maximum and minimum values of

f(x,y,z)

subject to the constraints

g(x,y,z) = k

we execute the following two steps

Find all values of x, y, z and λ such that

i. ∇f(x,y,z) = λ ∇g(x,y,z),
ii. g(x,y,z) = k

Evaluate f at all these points from the first step above.

The largest of these is the maximum value of f.

The smallest of these is the minimum value of f.