Math 1C
Introduction to Least-Squares Problems
Lesson 0, Part 1: Introduction to Least-Squares Problems
©Jeffrey A. Anderson

Lesson 0, Part 1: Key Terms List
□ Least-Squares Problem
□ Linear Regression
□ Multivariable optimization problems
□ Keeling Curve

. Why Study Multivariable Calculus?

Example 1: Modeling Part of the Keeling Curve

In this example, we will introduce a multivariable function that we can create from real-world data. In doing so, we will see one of the most powerful and popular applications of the theory of multivariable differentiation (Part 1 of Math 1C) to various STEM fields. The process we highlight below is known as linear regression. Regression problems arise when studying technical problems that involve data collection, data analysis and the desire to create a mathematical function that accurately models the collected data set.

Let’s take a look at an example of linear regression by analyzing a subset of data from the famous Keeling Curve data set. You can find more about this data on the Keeling Curve website. In our case, we see that the table given below lists the average carbon dioxide levels in the atmosphere, measured in parts per million, at Mauna Loa Observatory from 1980 to 2006. We will use this partial data set to create a mathematical model of this data.

We begin working with data by storing our data in our working memory. In this case, we will store our input and output data as a list of ordered pairs. Each ordered pair will be stored as a two-tuple using the notation {x, y}, where x represents the year the data was collected and y represents the measured level of CO2. Below is our list of data:

CO2Data = {       {1980, 338.7},
			{1982, 341.1},
			{1984, 344.4},
			{1986, 347.2},
			{1988, 351.5},
			{1990, 354.2},
			{1992, 356.4},
			{1994, 358.9},
			{1996, 362.6},
			{1998, 366.6},
			{2000, 369.4},
			{2002, 372.9},
			{2004, 377.5},
			{2006, 381.9}};

Next, let’s visualize our data and see if we can identify a pattern. To do so, we will use the Mathematica command for plotting a list of ordered pairs ListPlot:

ListPlot[CO2Data]

In this example, we’ve input a single argument into ListPlot: our CO2Data list of two-tuples. When using the ListPlot command, Mathematica interprets each two-tuple in the list we’ve provided as a point in the coordinate plane and plots these points.

As we’ve seen before, we can specify option settings for many of Mathematica’s commands to modify the output of the command. For example, we can set the Joined option to true in the ListPlot command to “connect the dots” with straight lines.

ListPlot[CO2Data, Joined-> True]

Notice that the default setting for the Joined option is False. Explicitly setting the Joined option to False produces the exact same graph our original example.

ListPlot[CO2Data, Joined-> False]

Now let’s further enhance our visual by joining our data with straight lines between each two data points while showing the original data points explicitly:

ListPlot[CO2Data, Joined-> True, Mesh->All]

Enough about Mathematica for now. Let’s use this tool to do some math. Look at the behavior illustrated in our above plots of the data.

What do we see?

What kind of function might we use to model this data?

What is the general equation of this family of functions?

Let’s use Mathematica’s computational engine to create a model to fit our data. We will assume that our data is fit by a linear model. The LinearModelFit command produces the exact linear model in slope-intercept form that best fits our data. The output from the LinearModelFit command is a symbolic FittedModel object that represents the constructed linear model.

lm= LinearModelFit[CO2Data,x,x]

Plot[lm[x],{x,1980,2006}, PlotRange->{{1980,2006},{330,390}}]

Let’s now graph both the data and the linear model on the same axis. We will do this using the Show command which allows us to combine several graphics in a single axis. In this case, let’s merge the ListPlot graph we made earlier with our Plot command for the linear fit.

Show[
		ListPlot[CO2Data], 
		Plot[
			lm[x],
			{x,1980,2006}, 
			PlotRange->{{1980,2006},{330,390}}
	](*end: Plot command*)
](*end: Show command*)

In case we’d like to increase the size of our combined graph, we can set the ImageSize option to Full, which will fill the entire notebook page when the graphic is produced.

Show[ListPlot[CO2Data], Plot[lm[x],{x,1980,2006}, PlotRange->{{1980,2006},{330,390}}], ImageSize->Full]

We can now extract more information about this model’s fit. In particular, we can see how accurate the model represents the original data by viewing the numerical values of the errors between the model and the collected data.

lm["FitResiduals"]

ListPlot[lm["FitResiduals"], Filling->Axis]

As we will learn in this class, the model that is constructed in the process of Linear Regression, is a “best fit” model, in that this is the unique model of the form y = mx + b that minimize the total squared error between the observed data and the predicted values.