| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
LETTERS TO EDITOR |
Bioinformatics Unit, Universität Hohenheim, Fruwirthstrasse 23, 70599 Stuttgart, Germany
1 Corresponding author: piepho{at}uni-hohenheim.de
Recently, Kratzer and Littell (2006) published a paper in Poultry Science that considered fitting of a nonlinear regression model. The purpose of the analysis was to compare dose reponses of 2 different sources of Met, DL-2-hydroxy-4-(methylthio) butanoic acid (HMTBA) and dry DL-Met (DLM). The fitted model was
 | ([1]) |
where BWG = BW gain; DOSEDLM and DOSEHMB = the doses of the 2 Met sources; and D and H = 2 selector variables, with D = 1 and H = 0 for DLM and D = 0 and H = 1 for HMTBA. The main focus of the paper was on parameters b2 and b4, the plateaus for increasing doses of DLM and HMTBA, respectively. Specifically, the authors were interested in possible differences among the plateaus. They fitted the model with NLMIXED of the SAS System (SAS Institute, 1999) and tested the null hypothesis H0: b2 = b4 using the ESTIMATE statement. This test was performed for data from 13 published studies. One example that is shown in the paper yielded a significant result [P = 0.0024, not P = 0.0014, as erroneously reported in the text of Kratzer and Littell (2006); also see their Figure 3] based on a t-test. As NLMIXED uses the maximum likelihood (ML) method, the P-value was computed assuming a t-distribution with df equal to the number of observations (n = 9). Similarly, the residual error variance was estimated by SS/n = 55.8 where SS = the residual sum of squares of the fitted model (Page 33 in Seber and Wild, 1989). The ML method is bound to yield excessive type I errors in small samples. This is also true for the confidence limits shown in Figure 5 of Kratzer and Littell (2006), which are, therefore, expected to have less than the nominal coverage probability.
The standard method for inference for nonlinear models is based on nonlinear least squares (Chapter 5 in Seber and Wild, 1989), as implemented in PROC NLIN of the SAS system (SAS Institute, 1999). This method uses n – p = 4 df for error (instead of 9 df) where p = 5 is the number of estimated parameters, and n = 9 is the sample size. Thus, whereas regression parameter estimates are identical to those obtained by the ML method, the P-value of the t-test is computed using a t-distribution with 4 df. Also, the error variance is estimated by SS/(n – p) = 125.5 (Page 21 in Seber and Wild, 1989). Using nonlinear least squares, 1 obtains a marginally nonsignificant result (P = 0.0504) for the test of H0: b2 = b4. The NLIN procedure has no ESTIMATE statement, so to obtain this test I reparameterized the model as
 | ([2]) |
where diff_b4_b2 = the difference in plateaus. The relevant SAS code is shown in Figure 1
.
|
nominal = 0.050 was NLMIXED = 0.172, which is rather liberal. According to common standards, this test would be regarded as invalid. By comparison, the test based on nonlinear least squares, as implemented in PROC NLIN, yielded an empirical type I error rate of NLIN = 0.035, which is slightly on the conservative side. This would commonly be regarded as a valid test, though some improvement may be possible using other methods, such as those based on likelihood profiles (Seber and Wild, 1989). Nonlinear least squares, as well as ML estimation of nonlinear models, provide only approximate inference. There are many potential problems with both methods in small samples. Details may be found in Seber and Wild (1989). In most cases, however, nonlinear least squares is preferable, mainly because it more appropriately deals with the error df, as is also the case in the present example.
The main conclusion in the paper by Kratzer and Littell (2006) was not based on t-tests for the 13 studies but rather on the sign of the difference in estimated plateau levels for the 2 Met sources. This type of inference is very useful, as it remains unaffected by unsatisfactory behavior of large-sample tests in small samples per study. The authors found that in 11 studies, the plateau was higher for HMTBA than for DLM. They reported that this proportion of 11/13 is significantly different from 0.5, according to a sign test at P < 0.01. This is most likely a computational error. A P-value P < 0.01 would be obtained only based on a 1-sided test, using an asymptotic procedure. Both would be problematic. The hypothesis to be tested poses a 2-sided problem, as there is no a priori reason to expect that departure from equality of plateaus is in any specific direction. Also, the asymptotic sign test, based on the normal approximation for binomial proportions, is not valid in small samples. Using PROC FREQ, I obtained the results given in Figure 2. Alternatively, PROC UNIVARIATE could have been used to perform the 2-sided sign test. The asymptotic 1-sided test yields P = 0.0063, whereas the exact 2-sided test has P = 0.0225. Although this is still significant at = 0.05, the P-value is rather larger than P = 0.01, so the evidence in favor of a real difference in plateaus is somewhat smaller than reported in the paper.
|
Received for publication April 19, 2006. Accepted for publication April 24, 2006.
 |
REFERENCES |
|---|
 TOP REFERENCES |
|---|
SAS Instit. 1999. SAS/SAT User’s Guide. Version 8. SAS Institute Inc., Cary, NC.
Seber, G. A. F., and C. J. Wild. 1989. Nonlinear Regression. John Wiley and Sons Inc., New York, NY.
van Houwelingen, H. C., L. R. Arends, and T. Stijnen. 2002. Advanced methods in meta-analysis: Multivariate approach and meta-regression. Stat. Med. 21:589–624.[ISI][Medline]
This article has been cited by other articles:
|
|
 |
|
  N. Sauer, K. Emrich, H.-P. Piepho, A. Lemme, M. S. Redshaw, and R. Mosenthin Meta-Analysis of the Relative Efficiency of Methionine-Hydroxy-Analogue-Free-Acid Compared with DL-Methionine in Broilers Using Nonlinear Mixed Models Poult. Sci., October 1, 2008; 87(10): 2023 - 2031. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. D. Rosen Nutritive Values of Methionine Sources Poult. Sci., February 1, 2007; 86(2): 209 - 210. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
