{"id":176,"date":"2009-04-05T13:16:18","date_gmt":"2009-04-05T20:16:18","guid":{"rendered":"http:\/\/inkofpark.wordpress.com\/?p=176"},"modified":"2024-09-01T20:36:53","modified_gmt":"2024-09-02T02:36:53","slug":"coffee-model-fitting-goodness","status":"publish","type":"post","link":"https:\/\/www.inkofpark.com\/?p=176","title":{"rendered":"Coffee: Model Fitting Goodness"},"content":{"rendered":"

You have seen the response function contours through the stationary point, and you have seen the 3D response function visualizations. You may recall that I have two competing models, one is derived from the experimental data design for the response surface, and the other is all that data and the screening results too. Is either model any good? Which one is better?<\/p>\n

First, a quick review of the model:<\/p>\n

q<\/em> = b<\/em>0<\/sub>T<\/em>2<\/sup> + b<\/em>1<\/sub>t<\/em>2<\/sup> + b<\/em>2<\/sub>r<\/em>2<\/sup> + b<\/em>3<\/sub>Tt<\/em> + b<\/em>4<\/sub>tr<\/em> + b<\/em>5<\/sub>Tr<\/em> + b<\/em>6<\/sub>T<\/em> + b<\/em>7<\/sub>t<\/em> + b<\/em>8<\/sub>r<\/em> + b<\/em>9
\n<\/sub><\/p>\n

Fitting produces numbers for all those coefficients, b<\/em>0<\/sub>, b<\/em>1<\/sub>, etc. Look at the goodness of individual coefficients in the following table. The more asterisks in the significance column, the better the coefficient corresponds to the data. Notice that the “all data” model has an overall terrible goodness, with the residual standard error of 0.7, compared to the residual error of 0.358 for the RSO data only.<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
<\/td>\n\n

All Data<\/strong><\/span><\/p>\n<\/td>\n

\n

RSO Data Only<\/strong><\/span><\/p>\n<\/td>\n<\/tr>\n

Std Error<\/strong><\/span><\/td>\n\n

0.749<\/strong><\/span><\/p>\n<\/td>\n

\n

0.358<\/strong><\/span><\/p>\n<\/td>\n<\/tr>\n

<\/td>\nEstimate<\/span><\/td>\nStd. Error<\/span><\/td>\nSignif.<\/span><\/td>\nEstimate<\/span><\/td>\nStd. Error<\/span><\/td>\nSignif.<\/span><\/td>\n<\/tr>\n
(Intercept)<\/strong><\/span><\/td>\n\n

3.344<\/span><\/p>\n<\/td>\n

\n

0.3963<\/span><\/p>\n<\/td>\n

\n

***<\/span><\/p>\n<\/td>\n

\n

3.6667<\/span><\/p>\n<\/td>\n

\n

0.2064<\/span><\/p>\n<\/td>\n

\n

***<\/span><\/p>\n<\/td>\n<\/tr>\n

temp<\/strong><\/span><\/td>\n\n

-0.355<\/span><\/p>\n<\/td>\n

\n

0.2369<\/span><\/p>\n<\/td>\n

<\/td>\n\n

-0.4375<\/span><\/p>\n<\/td>\n

\n

0.1264<\/span><\/p>\n<\/td>\n

\n

*<\/span><\/p>\n<\/td>\n<\/tr>\n

time<\/strong><\/span><\/td>\n\n

0.235<\/span><\/p>\n<\/td>\n

\n

0.2369<\/span><\/p>\n<\/td>\n

<\/td>\n\n

0.0375<\/span><\/p>\n<\/td>\n

\n

0.1264<\/span><\/p>\n<\/td>\n

<\/td>\n<\/tr>\n
cwrat<\/strong><\/span><\/td>\n\n

0.4<\/span><\/p>\n<\/td>\n

\n

0.2369<\/span><\/p>\n<\/td>\n

<\/td>\n\n

0.275<\/span><\/p>\n<\/td>\n

\n

0.1264<\/span><\/p>\n<\/td>\n

\n

.<\/span><\/p>\n<\/td>\n<\/tr>\n

temp:time<\/strong><\/span><\/td>\n\n

-0.5<\/span><\/p>\n<\/td>\n

\n

0.2901<\/span><\/p>\n<\/td>\n

<\/td>\n\n

-0.75<\/span><\/p>\n<\/td>\n

\n

0.1788<\/span><\/p>\n<\/td>\n

\n

**<\/span><\/p>\n<\/td>\n<\/tr>\n

temp:cwrat<\/strong><\/span><\/td>\n\n

0.02<\/span><\/p>\n<\/td>\n

\n

0.2901<\/span><\/p>\n<\/td>\n

<\/td>\n\n

-0.375<\/span><\/p>\n<\/td>\n

\n

0.1788<\/span><\/p>\n<\/td>\n

\n

.<\/span><\/p>\n<\/td>\n<\/tr>\n

time:cwrat<\/strong><\/span><\/td>\n\n

0.09<\/span><\/p>\n<\/td>\n

\n

0.2901<\/span><\/p>\n<\/td>\n

<\/td>\n\n

-0.075<\/span><\/p>\n<\/td>\n

\n

0.1788<\/span><\/p>\n<\/td>\n

<\/td>\n<\/tr>\n
temp^2<\/strong><\/span><\/td>\n\n

-0.401<\/span><\/p>\n<\/td>\n

\n

0.3592<\/span><\/p>\n<\/td>\n

<\/td>\n\n

-0.6833<\/span><\/p>\n<\/td>\n

\n

0.1861<\/span><\/p>\n<\/td>\n

\n

*<\/span><\/p>\n<\/td>\n<\/tr>\n

time^2<\/strong><\/span><\/td>\n\n

-0.201<\/span><\/p>\n<\/td>\n

\n

0.3592<\/span><\/p>\n<\/td>\n

<\/td>\n\n

-0.4833<\/span><\/p>\n<\/td>\n

\n

0.1861<\/span><\/p>\n<\/td>\n

\n

*<\/span><\/p>\n<\/td>\n<\/tr>\n

cwrat^2<\/strong><\/span><\/td>\n\n

0.174<\/span><\/p>\n<\/td>\n

\n

0.3592<\/span><\/p>\n<\/td>\n

<\/td>\n\n

-0.1083<\/span><\/p>\n<\/td>\n

\n

0.1861<\/span><\/p>\n<\/td>\n

<\/td>\n<\/tr>\n
Significance codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1<\/span><\/td>\n<\/tr>\n
<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The two graphs below show the quantile normal plots for the residuals from each model. The all-data model looks more systematically erroneous than the RSM data only. The systematic error suggests that the choice of model is not particularly good. Furthermore, it suggests that the estimates of F-statistic should be considered hesitantly, as F-statistics are calculated by assuming normally distributed independent errors.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

With hesitation then, let’s examine the F-statistic for these models. The RSO data model produces an F-statistic of 6.4, well above the Box, Hunter, and Hunter criterion that the F-statistic exceeds 4. On the other hand, the all data model’s F-statistic is a miserly value of 1.<\/p>\n\n\n\n\n\n\n\n\n\n
<\/td>\nAll Data<\/span><\/td>\nRSO Data Only<\/span><\/td>\n<\/tr>\n
Degrees of Freedom<\/span><\/td>\n9<\/span><\/td>\n5<\/span><\/td>\n<\/tr>\n
Residual Standard Error<\/span><\/td>\n0.749<\/span><\/td>\n0.358<\/span><\/td>\n<\/tr>\n
F<\/em>-statistic<\/span><\/td>\n1.06<\/span><\/td>\n6.43<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

In conclusion, I get a better fit\u2014to the indicated model\u2014using only some of the data. This is, frankly, an indictment of the model. Anyone can carefully select data to produce a good fit to a model, and the fact that I have done so may be considered a round condemnation of my objectivity. I may reconsider the model, and attempt to drop the least important terms while adding either a three-way interaction or two-way involving squares. Perhaps more creative math will be required.<\/p>\n","protected":false},"excerpt":{"rendered":"

You have seen the response function contours through the stationary point, and you have seen the 3D response function visualizations. You may recall that I have two competing models, one is derived from the experimental data design for the response surface, and the other is all that data and the screening results too. Is either […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[23,31,48],"class_list":["post-176","post","type-post","status-publish","format-standard","hentry","category-coffee","tag-coffee","tag-food-cooking","tag-statistics"],"_links":{"self":[{"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=\/wp\/v2\/posts\/176","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=176"}],"version-history":[{"count":1,"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=\/wp\/v2\/posts\/176\/revisions"}],"predecessor-version":[{"id":1309,"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=\/wp\/v2\/posts\/176\/revisions\/1309"}],"wp:attachment":[{"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=176"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=176"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.inkofpark.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=176"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}