Example of Fitting a Custom Model
To fit a custom model, you must first create a model column with initial parameter estimates. This method does require a few more steps than fitting a built-in model, but it does allow any nonlinear model to be fit. Also, you can provide a custom loss function, and specify several other options for the fitting process.
This section provides an example of creating the formula column for a model, and fitting the model in the Nonlinear platform. The data is in the US Population.jmp data table. The response variable is the population (in millions) of the Unites States and the predictor is the year.
1. Select Help > Sample Data Library and open Nonlinear Examples/US Population.jmp.
2. Create a new column called Model.
3. Right-click the Model column and select Column Properties > Formula.
The Formula Editor appears.
4. Above the list of columns on the left, select Parameters.
5. Select New Parameter.
6. Use the default name of b0.
7. Type 4 for Value. This is the initial estimate of the parameter.
8. Click OK.
9. Select New Parameter.
10. Keep the default name and enter 0.02 for Value.
11. Click OK.
12. Enter the model formula using the Formula Editor functions, the column year, and the parameters. Figure 13.2 shows the completed model.
Figure 13.2 Completed Model Formula
13. Click OK.
14. Select Analyze > Specialized Modeling > Nonlinear.
15. Assign Model to the X, Predictor Formula role.
16. Assign pop to the Y, Response role.
17. Click OK.
18. Click Go on the Control Panel to fit the model.
A portion of the report is shown in Figure 13.3.
Figure 13.3 Plot and Solution Report
The final parameter estimates are shown in the Solution report, along with other fit statistics. The fitted model is shown on the plot.
Parameters for Models with a Grouping Variable
In the formula editor, when you add a parameter, note the check box for Expand Into Categories, selecting column. This option is used to add several parameters (one for each level of a categorical variable for example) at once. When you select this option, a dialog appears that enables you to select a column. After selection, a new parameter appears in the Parameters list with the name D_column, where D is the name that you gave the parameter. When you use this parameter in the formula, a Match expression is inserted, containing a separate parameter for each level of the grouping variable.
Launch the Nonlinear Platform
To launch the Nonlinear platform, select Analyze > Specialized Modeling > Nonlinear. The launch window is shown in Figure 13.4.
Figure 13.4 Nonlinear Platform Launch Window
The Nonlinear platform launch window has the following features:
Y, Response
Select the Y variable.
X, Predictor Formula
Select either the X variable or a column containing the model formula with parameters.
Group
Specify a grouping variable. The fitted model has separate parameters for each level of the grouping variable. This enables you to compare fitted models and estimated parameters across the levels of the grouping variable.
Weight
Specify a variable containing the weights of observations.
Freq
Specify a variable representing the frequency of an observation.
Loss
Specify a formula column giving a loss function.
By
Specify a variable to perform a separate analysis for every level of the variable.
Model Library
Launches the Model Library tool, which helps you choose initial values to create a formula column. See “Create a Formula Using the Model Library”.
Numeric Derivatives Only
Uses only numeric derivatives only. This option is useful when you have a model for which it is too messy to take analytic derivatives. It can also be valuable in obtaining convergence in tough cases. This option is used only when a formula column is provided in the X, Predictor Formula role.
Expand Intermediate Formulas
Tells JMP that if an ingredient column to the model is a column that itself has a formula, to substitute the inner formula, as long as it refers to other columns. To prevent an ingredient column from expanding, use the Other column property with a name of “Expand Formula” and a value of 0. This option is used only when a formula column is provided in the X, Predictor Formula role.
The Nonlinear Fit Report
The initial Nonlinear Fit report includes the following items, shown in Figure 13.5.
Control Panel
Provides options for controlling the fitting process.
Go
Starts the fitting process.
Stop
Stops the fitting process.
Step
Proceeds through the fitting process one iteration at a time.
Reset
Resets the editable values into the formula, resets the iteration values, and calculates the SSE at these new values.
Criterion
Shows iteration measures from the fitting process.
Current
Shows the current value of each Criterion.
Stop Limit
Sets limits on the measures listed under Criterion.
Plot
Shows a plot of the X and Y variables for models with only one X variable. The model based on the current values is shown on the plot. To change the current values of the parameters, use the sliders or edit boxes beneath the plot.
Figure 13.5 Initial Nonlinear Fit Report
After you click Go to fit a model, the report includes the following additional items, shown in Figure 13.6.
Save Estimates
Saves the current parameter values to the parameters in the formula column.
Confidence Limits
Computes confidence intervals for all parameters. The intervals are profile likelihood confidence intervals, and are shown in the Solution report. The confidence limit computations involve a new set of iterations for each limit of each parameter, and the iterations often do not find the limits successfully. The Edit Alpha and Convergence Criterion options are for the confidence interval computations. For details about the Goal SSE for CL, see “Profile Likelihood Confidence Limits”.
Solution
Shows the parameters estimates and other statistics.
SSE
Shows the residual sum of squared errors. SSE is the objective that is to be minimized. If a custom loss function is specified, this is the sum of the loss function.
DFE
Shows the degrees of freedom for error, which is the number of observations used minus the number of parameters fitted.
MSE
Shows the mean squared error. It is the estimate of the variance of the residual error, which is the SSE divided by the DFE.
RMSE
Estimates the standard deviation of the residual error, which is square root of the MSE.
Parameter
Lists the names that you gave the parameters in the fitting formula.
Estimate
Lists the parameter estimates produced. Keep in mind that with nonlinear regression, there might be problems with this estimate even if everything seems to work.
ApproxStdErr
Lists the approximate standard error, which is computed analogously to linear regression. It is formed by the product of the RMSE and the square root of the diagonals of the derivative cross-products matrix inverse.
Lower CL and Upper CL
Shows the confidence limits for the parameters. They are missing until you click the Confidence Limits on the Control Panel. For more details about the confidence intervals, see “Profile Likelihood Confidence Limits”.
Excluded Data
Shows a report showing fit statistics for excluded rows. This is useful for validating the model on observations not used to fit the model. You can use this feature in conjunction with the Remember Solution option to change the exclusions, and get a new report reflecting the different exclusions
Correlation of Estimates
Displays the correlations between the parameter estimates.
Figure 13.6 Fitted Model Report
Nonlinear Platform Options
The Nonlinear Fit red triangle menu has the following options:
Parameter Bounds
Sets bounds on the parameters. When the option is selected, editable boxes appear in the Control Panel. Unbounded parameters are signified by leaving the field blank.
Plot
Shows or hides a plot of the X and Y variables for models with only one X variable. The model shown on the plot is based on the current values of the parameters. To change the current values of the parameters, use the sliders or edit boxes beneath the plot. If you specify a Group variable at launch, then a curve shows for each group.
Iteration Options
Specifies options for the fitting algorithm.
Iteration Log
Records each step of the fitting process in a new window.
Numeric Derivatives Only
Useful when you have a model that is too messy to take analytic derivatives for. It can also be valuable in obtaining convergence in tough cases.
Expand Intermediate Formulas
Tells JMP that if an ingredient column to the formula is a column that itself has a formula, to substitute the inner formula, as long as it refers to other columns. To prevent an ingredient column from expanding, use the Other column property with a name of “Expand Formula” and a value of 0.
Newton
Specifies whether Gauss-Newton (for regular least squares) or Newton-Raphson (for models with loss functions) is the optimization method.
QuasiNewton SR1
Specifies QuasiNewton SR1 as the optimization method.
QuasiNewton BFGS
Specifies QuasiNewton BFGS as the optimization method.
Accept Current Estimates
Tells JMP to produce the solution report with the current estimates, even if the estimates did not converge.
Show Derivatives
Shows the derivatives of the nonlinear formula in the JMP log. See “Notes Concerning Derivatives”, for technical information about derivatives.
Unthreaded
Runs the iterations in the main computational thread. In most cases, JMP does the computations in a separate computational thread. This improves the responsiveness of JMP while doing other things during the nonlinear calculations. However, there are some isolated cases (models that have side effects that call display routines, for example) that should be run in the main thread, so this option should be turned on.
Profilers
Provides various profilers for viewing response surfaces.
Profiler
Shows the Prediction Profiler. The Profiler lets you view vertical slices of the surface across each x-variable in turn, as well as find optimal values of the factors.
Contour Profiler
Shows the Contour Profiler. The Contour profiler lets you see two-dimensional contours as well as three dimensional mesh plots.
Surface Profiler
Creates a three-dimensional surface plot. This option is available only for models with two or more X variables.
Parameter Profiler
Shows the Prediction Profiler and profiles the SSE or loss as a function of the parameters.
Parameter Contour Profiler
Shows the Contour Profiler and contours the SSE or loss as a function of the parameters.
Parameter Surface Profiler
Creates a three-dimensional surface plot and profiles the SSE or loss as a function of the parameters. This option is available only for models with two or more parameters.
SSE Grid
Create a grid of values around the solution estimates and compute the error sum of squares for each value. The solution estimates should have the minimum SSE. When the option is selected, the Specify Grid for Output report is shown with these features:
Parameter
Lists the parameters in the model.
Min
Displays the minimum parameter values used in the grid calculations. By default, Min is the solution estimate minus 2.5 times the ApproxStdErr.
Max
Displays the maximum parameter value used in the grid calculations. By default, Max is the solution estimate plus 2.5 times the ApproxStdErr.
Number of Points
Gives the number of points to create for each parameter. To calculate the total number of points in the new grid table, multiply all the Number of Points values. Initially Number of Points is 11 for the first two parameters and 3 for the rest. If you specify new values, use odd values to ensure that the grid table includes the solution estimates. Setting Number of Points to 0 for any parameter records only the solution estimate in the grid table.
When you click Go, JMP creates the grid of points in a new table. A highlighted row marks the solution estimate row if the solution is in the table.
Revert to Original Parameters
Resets the platform to the original parameter values (the values given in the formula column parameters).
Remember Solution
Creates a report called Remembered Models, which contains the current parameter estimates and summary statistics. Results of multiple models can be remembered and compared. This is useful if you want to compare models based on different parameter restrictions, or models fit using different options. Click on the radio button for a particular model to display that model in the Plot and the parameter estimates in the Control Panel.
Custom Estimate
Gives an estimate of a function of the parameters. You provide an expression involving only parameters. JMP calculates the expression using the current parameter estimates, and also calculates a standard error of the expression using a first-order Taylor series approximation.
Custom Inverse Prediction
Estimates the X value for a given Y value. It also calculates a standard error for the estimated X. JMP must be able to invert the model. The standard error is based on the first-order Taylor series approximation using the inverted expression. The confidence interval uses a t-quantile with the standard error, and is a Wald interval.
Save Pred Confid Limits
Saves asymptotic confidence limits for the model prediction. This is the confidence interval for the average Y at a given X value.
Save Indiv Confid Limits
Saves asymptotic confidence limits for an individual prediction. This is the confidence interval for an individual Y value at a given X value.
Save Formulas
Gives options for saving model results to data table columns:
Save Prediction Formula
Saves the prediction formula with the current parameter estimates.
Save Std Error of Predicted
Saves the standard error for a model prediction. This is the standard error for predicting the average Y for a given X. The formula is of the form Sqrt(VecQuadratic(matrix1,vector1)). matrix1 is the covariance matrix associated with the parameter estimates, and vector1 is a composition of the partial derivatives of the model with respect to each parameter.
Save Std Error of Individual
Saves the standard error for an individual prediction. This is the standard error for predicting an individual Y value for a given X value. The formula is of the form Sqrt(VecQuadratic(matrix1,vector1)+mse). matrix1 is the covariance matrix associated with the parameter estimates, vector1 is a composition of the partial derivatives of the model with respect to each parameter, and mse is the estimate of error variance.
Save Residual Formula
Saves the formula for computing the residuals.
Save Pred Confid Limit Formula
Saves the formula to calculate the confidence interval for a model prediction. This is a confidence interval for the average Y for a given X.
Save Indiv Confid Limit Formula
Saves the formula to calculate the confidence interval for an individual prediction. This is a confidence interval for an individual Y for a given X.
Save Inverse Prediction Formula
Saves formulas for the inverse of the model, the standard error of an inverse prediction, and the standard error of an individual inverse prediction.
Save Specific Solving Formula
Equivalent to Save Inverse Prediction Formula in simple cases. However, this command allows the formula to be a function of several variables and allows expressions to be substituted. This feature works only for solving easily invertible operators and functions that occur just once in the formula.
After selecting this command, a dialog appears that enables you to select the variable to solve for. You can also edit the names of the columns in the resulting table. You can also substitute values for the names in the dialog. In these cases, the formula is solved for those values.
Note: The standard errors, confidence intervals, and hypothesis tests are correct only if least squares estimation is done, or if maximum likelihood estimation is used with a proper negative log-likelihood.
Show Prediction Expression
Shows the prediction model or the loss function at the top of the report.
Create a Formula Using the Model Library
The Model Library can assist you in creating the formula column with parameters and initial values. Click the Model Library button on the Nonlinear launch window to open the library. Select a model in the list to see its formula in the Formula box (Figure 13.7).
Figure 13.7 Nonlinear Model Library Dialog
Click Show Graph to show a 2-D theoretical curve for one-parameter models and a 3-D surface plot for two-parameter models. No graph is available for models with more than two explanatory (X) variables. On the graph window, change the default initial values of parameters using the slider, or clicking and entering values in directly. See Figure 13.8.
Figure 13.8 Example Graph in Model Library
The Reset button sets the initial values of parameters back to their default values.
Click Show Points to overlay the actual data points to the plot. The dialog in Figure 13.9 opens, asking you to assign columns into X and Y roles, and an optional Group role. The Group role allows for fitting the model to every level of a categorical variable. If you specify a Group role here, also specify the Group column on the platform launch window.
Figure 13.9 Select Roles
For most models, the starting values are constants. Showing points enables you to adjust the parameter values to see how well the model fits for different values of the parameters. For the US population example, the points are shown in Figure 13.10.
Figure 13.10 Show Points
Clicking Make Formula at this point (after using Show Points) creates a new data table column named after the model that you chose from the Model Library. This column has the formula as a function of the latest parameter starting values.
Note: If you click Make Formula before using the Show Graph or Show Points buttons, you are asked to provide the X and Y roles, and an optional Group role. See Figure 13.9. After that, you are brought back to the plot so that you have the opportunity to adjust the parameters starting values if desired. At that point click Make Formula again to create the new column.
Once the formula is created in the data table, continue the analysis by assigning the new column as the X, Predictor Formula in the Nonlinear launch dialog.
Customize the Nonlinear Model Library
The Model Library is created by a built-in script named NonlinLib.jsl, located in the Resources/Builtins folder in the folder that contains JMP (Windows) or in the Application Package (Macintosh). You can customize the nonlinear library script by modifying this script.
To add a model, you must add three lines to the list named Listofmodellist#. These three lines are actually a list themselves, which consists of the following three parts.
• Model name, a quoted string
• Model formula, an expression
• Model scale
For example, suppose you want to add a model called “Simple Exponential Growth” that has the form
Add the following lines to the NonlinLib.jsl script
{//Simple Exponential Growth
"Simple Exponential Growth",
Expr(Parameter({b1=2, k=0.5}, b1*exp(k * :X))),
lowx = -1; highx = 2; lowy = 0; highy = 2},
Some things to note:
• The first line is simply an open bracket (starting the list) and an optional comment. The second line is the string that is displayed in the model library window.
• The values of lowx, highx, logy, and highy specify the initial window for the theoretical graph.
• There is a comma as the last character in the example above. If this is the final entry in the Listofmodellist# list, the comma can be omitted.
• If the model uses more than two parameters, replace the last line (containing the graph limits) with the quoted string “String Not Available”.
To delete a model, delete the corresponding three-lined list from the Listofmodellist# list.
Additional Examples
This section provides several examples of the broad usefulness of the Nonlinear platform.
Example of Maximum Likelihood: Logistic Regression
This example shows how to use the Nonlinear platform to minimize a loss function. The loss function is the negative of a log-likelihood function, thus producing maximum likelihood estimates.
The Logistic w Loss.jmp data table in the Nonlinear Examples sample data folder has an example for fitting a logistic regression using a loss function. The Y column contains ones for events and zeros for non-events. The Model Y column has the linear model, and the Loss column has the loss function. In this example, the loss function is the negative log-likelihood for each observation, or the negative log of the probability of getting the observed response.
Run the model by following the steps below:
1. Select Help > Sample Data Library and open Nonlinear Examples/Logistic w Loss.jmp.
2. Select Analyze > Specialized Modeling > Nonlinear.
3. Assign Model Y to the X, Predictor Formula role.
4. Assign Loss to the Loss role.
Figure 13.11 Nonlinear Launch Window
5. Click OK.
The Nonlinear Fit Control Panel appears.
Figure 13.12 Nonlinear Fit Control Panel
6. Click Go.
The parameter estimates are shown in the Solution report. See Figure 13.13.
Figure 13.13 Solution Report
The Loss value in the Solution report is the negative log-likelihood evaluated at the parameter estimates.
Example of a Probit Model with Binomial Errors: Numerical Derivatives
The Ingots2.jmp sample data table includes the numbers of ingots tested for readiness after different treatments of heating and soaking times. The response variable, NReady, is binomial, depending on the number of ingots tested (Ntotal) and the heating and soaking times. Maximum likelihood estimates for parameters from a probit model with binomial errors are obtained using:
• numerical derivatives
• the negative log-likelihood as a loss function
• the Newton-Raphson method.
The average number of ingots ready is the product of the number tested and the probability that an ingot is ready for use given the amount of time it was heated and soaked. Using a probit model, the P column contains the model formula:
Normal Distribution(b0+b1*Heat+b2*Soak)
The argument to the Normal Distribution function is a linear model of the treatments.
To specify binomial errors, the loss function, Loss, has the formula
-(Nready*Log(p) + (Ntotal - Nready)*Log(1 - p))
Follow these steps to fit the model:
1. Select Analyze > Specialized Modeling > Nonlinear.
2. Assign P to the X, Predictor Formula role,
3. Assign Loss to the Loss role.
4. Select the Numeric Derivatives Only option.
5. Click OK.
6. Click Go.
The platform used the Numerical SR1 method to obtain the parameter estimates shown in Figure 13.14.
Figure 13.14 Solution for the Ingots2 Data
Example of a Poisson Loss Function
A Poisson distribution is often used to model count data.
, n = 0, 1, 2, …where μ can be a single parameter, or a linear model with many parameters. Many texts and papers show how the model can be transformed and fit with iteratively reweighted least squares (Nelder and Wedderburn 1972). However, in JMP it is more straightforward to fit the model directly. For example, McCullagh and Nelder (1989) show how to analyze the number of reported damage incidents caused by waves to cargo-carrying vessels.
The data are in the Ship Damage.jmp sample data table. The model formula is in the model column, and the loss function (or negative log-likelihood) is in the Poisson column. To fit the model, follow the steps below:
1. Select Analyze > Specialized Modeling > Nonlinear.
2. Assign model to the X, Predictor Formula role.
3. Assign Poisson to the Loss role.
4. Click OK.
5. Set the Current Value (initial value) for b0 to 1, and the other parameters to 0 (Figure 13.15).
Figure 13.15 Enter New Parameters
6. Click Go.
7. Click the Confidence Limits button.
The Solution report is shown in Figure 13.16. The results include the parameter estimates and confidence intervals, and other summary statistics.
Figure 13.16 Solution Table for the Poisson Loss Example
Note: The standard errors, confidence intervals, and hypothesis tests are correct only if least squares estimation is done, or if maximum likelihood estimation is used with a proper negative log-likelihood.
Example of Setting Parameter Limits
The Fit Curve personality enables you to fit a model and then use the prediction equation in the full personality of the Nonlinear platform. This method requires more steps and user input but allows any nonlinear model to be fit.
Complete “Example Using the Fit Curve Personality” in the “Fit Curve” chapter to fit the model. This example shows how to save the prediction formula from Fit Curve and then set parameter limits in Nonlinear.
1. From the Logistic 4P red triangle menu, select Save Formulas > Save Parametric Prediction Formula.
A new column named Toxicity Predictor appears in the data table.
2. Select Analyze > Specialized Modeling > Nonlinear.
3. Assign Toxicity to the Y, Response role.
4. Assign Toxicity Predictor to the X, Predictor Formula role.
5. Assign Formulation to the Group role.
6. Click OK.
The Nonlinear Fit window appears (Figure 13.17). In the Control Panel, parameter values and locking options are shown. The letters listed before each parameter correspond to variables from the Prediction Model in the Fit Curve function.
Figure 13.17 Nonlinear Fit Control Panel
Tip: You can lock parameters if you know the values from prior information.
7. Select the red triangle menu next to Nonlinear Fit and then select Parameter Bounds.
Options for setting the lower and upper parameters appear next to the parameters.
8. Set the lower bounds for the parameters as shown in Figure 13.18. You know from prior experience that the maximum toxicity of the drug is at least 1.1.
Figure 13.18 Setting Parameter Bounds
9. Click Go.
The final parameter estimates are shown in the Solution report, along with other fit statistics (Figure 13.19). The fitted model is shown on the plot.
Figure 13.19 Nonlinear Fit Plot and Parameter Estimates
Options below the plot allow for adjusting parameter limits and estimates (Figure 13.19).
Statistical Details
This section provides statistical details and other notes concerning the Nonlinear platform.
Profile Likelihood Confidence Limits
The upper and lower confidence limits for the parameters are based on a search for the value of each parameter after minimizing with respect to the other parameters. The search looks for values that produce an SSE greater by a certain amount than the solution’s minimum SSE. The goal of this difference is based on the F-distribution. The intervals are sometimes called likelihood confidence intervals or profile likelihood confidence intervals (Bates and Watts 1988; Ratkowsky 1990).
Profile confidence limits all start with a goal SSE. This is a sum of squared errors (or sum of loss function) that an F test considers significantly different from the solution SSE at the given alpha level. If the loss function is specified to be a negative log-likelihood, then a Chi-square quantile is used instead of an F quantile. For each parameter’s upper confidence limit, the parameter value is increased until the SSE reaches the goal SSE. As the parameter value is moved up, all the other parameters are adjusted to be least squares estimates subject to the change in the profiled parameter. Conceptually, this is a compounded set of nested iterations. Internally there is a way to do this with one set of iterations developed by Johnston and DeLong. See SAS/STAT 9.1 vol. 3 pp. 1666-1667.
Figure 13.20 shows the contour of the goal SSE or negative likelihood, with the least squares (or least loss) solution inside the shaded region:
• The asymptotic standard errors produce confidence intervals that approximate the region with an ellipsoid and take the parameter values at the extremes (at the horizontal and vertical tangents).
• Profile confidence limits find the parameter values at the extremes of the true region, rather than the approximating ellipsoid.
Figure 13.20 Diagram of Confidence Limits for Parameters
Likelihood confidence intervals are more trustworthy than confidence intervals calculated from approximate standard errors. If a particular limit cannot be found, computations begin for the next limit. When you have difficulty obtaining convergence, try the following:
• use a larger alpha, resulting in a shorter interval, more likely to be better behaved
• relax the confidence limit criteria.
How Custom Loss Functions Work
The nonlinear facility can minimize or maximize functions other than the default sum of squares residual. This section shows the mathematics of how it is done.
Suppose that f(β) is the model. Then the Nonlinear platform attempts to minimize the sum of the loss functions defined as follows:
The loss function 
for each row can be a function of other variables in the data table. It must have nonzero first- and second-order derivatives. The default 
function, squared-residuals, is
for each row can be a function of other variables in the data table. It must have nonzero first- and second-order derivatives. The default function, squared-residuals, is
To specify a model with a custom loss function, construct a variable in the data table and build the loss function. After launching the Nonlinear platform, select the column containing the loss function as the loss variable.
The nonlinear minimization formula works by taking the first two derivatives of 
with respect to the model, and forming the gradient and an approximate Hessian as follows:
with respect to the model, and forming the gradient and an approximate Hessian as follows:
If 
is linear in the parameters, the second term in the last equation is zero. If not, you can still hope that its sum is small relative to the first term, and use
is linear in the parameters, the second term in the last equation is zero. If not, you can still hope that its sum is small relative to the first term, and use
The second term is probably small if ρ is the squared residual because the sum of residuals is small. The term is zero if there is an intercept term. For least squares, this is the term that distinguishes Gauss-Newton from Newton-Raphson.
Note: The standard errors, confidence intervals, and hypothesis tests are correct only if least squares estimation is done, or if maximum likelihood estimation is used with a proper negative log-likelihood.
Notes Concerning Derivatives
The nonlinear platform takes symbolic derivatives for formulas with most common operations. This section shows what type of derivative expressions result.
If you open the Negative Exponential.jmp nonlinear sample data example, the actual formula for the Nonlinear column looks something like this:
Parameter({b0=0.5, b1=0.5}, b0*(1-Exp(-b1*X)))
The Parameter block in the formula is hidden if you use the formula editor. That is how it is stored in the column and how it appears in the Nonlinear Launch dialog. Two parameters named b0 and b1 are given initial values and used in the formula to be fit.
The Nonlinear platform makes a separate copy of the formula, and edits it to extract the parameters from the expression. Then it maps the references to them to the place where they are estimated. Nonlinear takes the analytic derivatives of the prediction formula with respect to the parameters. If you use the Show Derivatives command, you get the resulting formulas listed in the log, like this:
Prediction Model:
b0 * First(T#1=1-(T#2=Exp(-b1*X)), T#3=-(-1*T#2*X))
The Derivative of Model with respect to the parameters is:
{T#1, T#3*b0}
The derivative facility works like this:
• In order to avoid calculating subexpressions repeatedly, the prediction model is threaded with assignments to store the values of subexpressions that it needs for derivative calculations. The assignments are made to names like T#1, T#2, and so on.
• When the prediction model needs additional subexpressions evaluated, it uses the First function, which returns the value of the first argument expression, and also evaluates the other arguments. In this case additional assignments are needed for derivatives.
• The derivative table itself is a list of expressions, one expression for each parameter to be fit. For example, the derivative of the model with respect to b0 is T#1; its thread in the prediction model is 1–(Exp(-b1*X)). The derivative with respect to b1 is T#3*b0, which is –(–1*Exp(-b1*X)*X)*b0 if you substitute in the assignments above. Although many optimizations are made, it does not always combine the operations optimally. You can see this by the expression for T#3, which does not remove a double negation.
If you specify a loss function, then the formula editor takes derivatives with respect to parameters, if it has any. And it takes first and second derivatives with respect to the model, if there is one.
If the derivative mechanism does not know how to take the analytic derivative of a function, then it takes numerical derivatives, using the NumDeriv function. If this occurs, the platform shows the delta that it used to evaluate the change in the function with respect to a delta change in the arguments. You might need to experiment with different delta settings to obtain good numerical derivatives.
Tips
There are always many ways to represent a given model, and some ways behave much better than other forms. Ratkowsky (1990) covers alternative forms in his text.
If you have repeated subexpressions that occur several places in a formula, then it is better to make an assignment to a temporary variable. Then refer to it later in the formula. For example, one of the model formulas above was this:
If(Y==0, Log(1/(1+Exp(model))), Log(1 - 1/(1 + Exp(model))));
This could be simplified by factoring out an expression and assigning it to a local variable:
temp=1/(1+Exp(model));
If(Y==0, Log(temp), Log(1-temp));
The derivative facility can track derivatives across assignments and conditionals.
Notes on Effective Nonlinear Modeling
We strongly encourage you to center polynomials.
Anywhere you have a complete polynomial term that is linear in the parameters, it is always good to center the polynomials. This improves the condition of the numerical surface for optimization. For example, if you have an expression like
a1 + b1x + c1x2
you should transform it to
The two models are equivalent, apart from a transformation of the parameters, but the second model is far easier to fit if the model is nonlinear.
The transformation of the parameters is easy to solve.
If the number of iterations still goes to the maximum, increase the maximum number of iterations or relax one of the convergence criteria.
There is really no one omnibus optimization method that works well on all problems. JMP has options like Newton, QuasiNewton BFGS, QuasiNewton SR1, and Numeric Derivatives Only to expand the range of problems that are solvable by the Nonlinear Platform.
If the default settings are unable to converge to the solution for a particular problem, using various combinations of these settings to increase the odds of obtaining convergence.
Some models are very sensitive to starting values of the parameters. Working on new starting values is often effective. Edit the starting values and click Reset to see the effect. The plot often helps. Use the sliders to visually modify the curve to fit better. The parameter profilers can help, but might be too slow for anything but small data sets.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.