This notebook contains an excerpt from the Python Programming and Numerical Methods - A Guide for Engineers and Scientists, the content is also available at Berkeley Python Numerical Methods.
The copyright of the book belongs to Elsevier. We also have this interactive book online for a better learning experience. The code is released under the MIT license. If you find this content useful, please consider supporting the work on Elsevier or Amazon!
< 15.5 Summary and Problems | Contents | 16.1 Least Squares Regression Problem Statement >
Chapter 16. Least Squares Regression¶
CHAPTER OUTLINE¶
Motivation¶
Often in science and engineering coursework, we are asked to determine the state of a system given the parameters of the system. For example, the relationship between the force exerted by a linear spring, F, and the displacement of the spring from its natural length, x, is usually represented by the model
where k is the spring stiffness. We are then asked to compute the force for a given k and x value.
However in practice, the stiffness and in general, most of the parameters of a system, are not known a priori. Instead, we are usually presented with data points about how the system has behaved in the past. For our spring example, we may be given \((x, F)\) data pairs that have been previously recorded from an experiment. Ideally, all these data points would lie exactly on a line going through the origin (since there is no force at zero displacement). We could then measure the slope of this line and get our stiffness value for k. However, practical data usually has some measurement noise because of sensor inaccuracy, measurement error, or a variety of other reasons. The following figure shows an example of what data might look like for a simple spring experiment.
This chapter teaches methods of finding the “most likely” model parameters given a set of data, e.g., how to find the spring stiffness in our mock experiment. By the end of this chapter you should understand how these methods choose model parameters, the importance of choosing the correct model, and how to implement these methods in.