O.06: Section 4
Section 4: Miscellaneous advanced modeling issues and topics for further study
Unnecessary parameters: The ability to add more parameters by adding basic models together raises the question of where to stop this process. The additional parameters almost always enable the model to come closer to the data points, thus reducing the sum-of-squared-deviations figure—if the number of parameters is increased until there are as many parameters as data points, it is likely that the model can go exactly through every point. However, such a model is usually not better than some simpler one at predicting other output values. The way to assess this is to find the model that produces the minimum standard deviation. Since the standard deviation is computed by an averaging process that increases the result when more parameters are used, using more parameters than are needed to follow the non-random trend of the data will result in a larger standard deviation. A model that has more parameters than are needed is said to be over-fitted. This can be especially misleading for some kinds of modeling formulas whose extrapolation behavior is poor, such as high-order polynomials, because it results in the model replicating some of the noise in a particular data set rather than extracting the repeatable aspects of the process being measured. An example of an over-fitted model is show to the right. Multiple and local solutions: Solver finds solutions by making small changes in each of the indicated parameters and keeping any changes that result in a smaller value for the goodness-of-fit indicator chosen (usually the sum-of-squared-deviations or the standard deviation). This process is continued until none of the changes improves the quality of the fit, even for very small changes. The search thus stops in a “valley” (actually more like the bottom of a bowl), where changing any parameter in any direction will make the fit worse. Usually this process gives good model parameters even if the initial parameter values are quite different from the best-fit values (the search keeps moving downhill until it reaches the bottom of the bowl). But for some functions the fitting process has more than one solution; in such cases, the solution that is found depends on the initial conditions (and to a lesser extent on the details of the search process). If all the solutions are of equal quality, the search is said to have multiple solutions. But it is sometimes possible to have a “local minimum” solution, in which an answer is found that is better than any other nearby parameters settings, but still is significantly larger than some other solution. Splines: A spline is a complex type of composite function that is often used during mechanical design processes to create smoothly-varying curves that do not match a simple mathematical formula. Splines break up the points along the desired path into overlapping groups, then fit modeling formulas to each group, with added provision to ensure that the slope of the curve changes gradually in the overlapping parts. Although splines are now generated by complex software using advanced versions of techniques similar to those introduced in this course, they were originally made by inserting thin flexible pieces of wood between pegs placed along the desired path, with the path of the bent wood measured or traced to produce a natural-looking curve. Stability of solutions: Since best-fit parameter values are often used to explain aspects of the situation that was measured, it is of interest to know how dependable they are. How much will the best-fit parameters change if a new set of measurement (with new noise, as always) is taken? One way to find out is to add a small amount of random noise to the data and then re-fit the model. Doing this several times and observing the effects will provide information about the sensitivity of each parameter to noise. It is possible to have parameters that are quite sensitive to noise even though the predictions of the model are much less sensitive. This can occur when small changes in the data lead the fitting process to change two different parameters in ways that almost cancel out in the result of the prediction formula. This is a milder version of the “confounding” that was discussed in Section 2 of this topic.Licenses & Attributions
CC licensed content, Shared previously
- Mathematics for Modeling. Authored by: Mary Parker and Hunter Ellinger. License: CC BY: Attribution.
