Too few samples and/or parameter difficult to predict and/or noise in the reference method are possible causes of large difference between SEC and SECV

From NIR Forum discussion Posted on Tuesday, February 08, 2011 - 4:42 am:    Dear all,  First, I want to say thank you in advance for every answer. I'm first time on this forum.  I have read a lot of things here and I  think that it is really useful. I have a question regarding the prediction of wood properties with NIR spectra.  I have a set of spectra from wood sample (calibration and test set) and I would like develop the best model for wood properties (eg wood density). However, I get higher error for cross validation (SECV) then for calibration set (SEC) and test set(SEP).Maybe I make a mistake in the application of cross validation. I use Unscrambler software. Can anyone tell me how to use the option of cross validation in Unscrambler software?  Thank you and best regards,  Nebojsa

Posted on Tuesday, February 08, 2011 - 5:44 am:    HI,  How large is the difference between SEC and SECV?  Before calibrating, you have to know the SEL (error of the reference method) and the SD of the calibration sample.  How many samples?  the gap between SEC and SECV is due to  - Too few samples and/or  - Parameter difficult to predict and/or  - Noise in the reference method.  SECV is always higher than SEC. My rule is to have SECV<=1.05*SEC with 2 groups of CV. Then I am pretty sure the model is robust.  Pierre

Posted on Tuesday, February 08, 2011 - 7:22 am:

NIR model should produce SEP lesser than 2.0SEL or 1.5SEL at least.

"There is also a statement of criteria that the NIR model should produce SEP lesser than 2.0SEL or 1.5SEL at least. If the SEP is larger then your model needs more calibration development and/or more samples."

(https://www.researchgate.net/post/Does_anybody_know_how_to_compare_NIR_error_and_lab_analysis_error1)



"Standard errors of performance (SEP) are frequently twice the magnitude of the standard error of the laboratory (SEL) in successful NIR calibrations. In spite of this the repeatability of NIR measurement is almost always better than the repeatability of the reference procedure."

(V. MÍKA, J. POZDÍŠEK, P. TILLMANN, P. NERUŠIL, K. BUCHGRABER4, L. GRUBER (2003) Development of NIR calibration valid for two different grass sample collections, Czech J. Anim. Sci., 48, 2003 (10): 419–424.)



"Westerhaus (1985, cited by Stimson, et al 1991) recommended that the SEP should be no greater than twice the SEL."

(G. McL. Dryden (2003) Near Infrared Reflectance Spectroscopy: Applications in Deer Nutrition. A report for the Rural Industries Research and Development Corporation, RIRDC Publication No W03/007 RIRDC Project No UQ-109A)