Citation: LI Zheng-Feng, XU Guang-Jin, WANG Jia-Jun, DU Guo-Rong, CAI Wen-Sheng, SHAO Xue-Guang. Outlier Detection for Multivariate Calibration in Near Infrared Spectroscopic Analysis by Model Diagnostics[J]. Chinese Journal of Analytical Chemistry, ;2016, 44(2): 305-309. doi: 10.11895/j.issn.0253-3820.150793 shu

Outlier Detection for Multivariate Calibration in Near Infrared Spectroscopic Analysis by Model Diagnostics

1.
R&D Center, China Tobacco Yunnan Industrial Co. Ltd., Kunming 650231, China;
2.
Research Center for Analytical Sciences, College of Chemistry, Nankai University, Tianjin 300071, China;
3.
College of Chemistry and Environmental Science, Kashgar University, Kashgar 844000, China

Corresponding author: SHAO Xue-Guang,
Received Date: 11 October 2015
Available Online: 28 October 2015
Fund Project: 本文系国家自然科学基金项目(No.21475068) (No.21475068)中国烟草总公司重大专项课题(No.Ts-03-20110020)资助 (No.Ts-03-20110020)

Outlier detection is an important task in multivariate calibration because the quality of a calibration model is determined by that of the calibration data. An outlier detection method is proposed for near infrared (NIR) spectral analysis. The method is based on the definition of outlier and the principle of partial least squares (PLS) regression, i.e., an outlier in a dataset behaves differently from the rest, and the prediction result of a PLS model is an accumulation of several independent latent variables. Therefore, the proposed method builds a PLS model with a calibration dataset, and then the contribution of each latent variable is investigated. Outliers can be detected by comparing these contributions. An NIR spectral dataset of orange juice samples is adopted for testing the method. Six outliers are detected in the calibration set. The root mean squared error of cross validation (RMSECV) becomes to 4.809 from 16.870 and the root mean squared error of prediction (RMSEP) becomes to 3.332 from 3.688 after the removal of the outliers. Compared with a robust regression method, the result of the proposed method seems more reasonable.
- Multivariate calibration,
- Outlier detection,
- Partial least squares,
- Near infrared spectroscopy,
- Quantitative analysis
References
1. [1]
  1 Wold S, Ruhe A, Wold H, Dunn W J. SIAM J. Sci. Stat. Comput., 1984, 5(3): 735-743
2. [2]
  2 LIANG Miao,CAI Jia-Yue, YANG Kai, SHU Ru-Xin, ZHAO Long-Lian, ZHANG Lu-Da, LI Jun-Hui. Chinese J. Anal. Chem., 2014, 42(11): 1687-1691 梁淼, 蔡嘉月, 杨凯, 束茹欣, 赵龙莲, 张录达, 李军会. 分析化学, 2014, 42(11): 1687-1691
3. [3]
  3 ZHANG Lu-Da, SU Shi-Guang, WANG Lai-Sheng, LI Jun-Hui, YANG Li-Ming. Spectroscopy and Spectral Analysis, 2005, 25(1): 33-35 张录达, 苏时光, 王来生, 李军会, 杨丽明. 光谱学与光谱分析, 2005, 25(1): 33-35
4. [4]
  4 Li Y K, Shao X G, Cai W S. Talanta, 2007, 72(1): 217-222
5. [5]
  5 LIN Hao, ZHAO Jie-Wen, CHEN Quan-Sheng, CAI Jian-Rong, ZHOU Ping. Spectroscopy and Spectral Analysis, 2010, 30(4): 929-932 林颢, 赵杰文, 陈全胜, 蔡健荣, 周平. 光谱学与光谱分析, 2010, 30(4): 929-932
6. [6]
  6 Shao X G, Bian X H, Liu J J, Zhang M, Cai W S. Anal. Methods, 2010, 2(11): 1662-1666
7. [7]
  7 Wold S, Antti H, Lindgren F, Ohman J. Chemom. Intell. Lab. Syst., 1998, 44(1-2): 175-185
8. [8]
  8 Shao X G, Leung A K M, Chau F T. Acc. Chem. Res., 2003, 36(4): 276-283
9. [9]
  9 Norgaard L, Saudland A, Wagner J, Wagner J, Nielsen J P, Munk L, Engelsen S B. Appl. Spectrosc., 2000, 54(3): 413-419
10. [10]
  10 Centner V, Massart D L, de Noord O E, de Jong S, Vandeginste M B, Sterna C. Anal. Chem., 1996, 68(21): 3851-3858
11. [11]
  11 Cai W S, Li Y K, Shao X G. Chemom. Intell. Lab. Syst., 2008, 90(2): 188-194
12. [12]
  12 Li H D, Liang Y Z, Xu Q S, Cao D S. Anal. Chim. Acta, 2009, 648(1): 77-84
13. [13]
  13 Araujo M C U, Saldanha T C B, Galvao R K H, Yoneyama T, Chame H C, Visani V. Chemom. Intell. Lab. Syst., 2001, 57(2): 65-73
14. [14]
  14 Xu H, Liu Z C, Cai W S, Shao X G. Chemom. Intell. Lab. Syst., 2009, 97(1): 189-193
15. [15]
  15 Liang Y Z, Kvalheim O M. Chemom. Intell. Lab. Syst., 1996, 32(1): 1-10
16. [16]
  16 Pierna J A F, Jin L, Daszykowski M, Wahl F, Massart D L. Chemom. Intell. Lab. Syst., 2003, 68(1-2): 17-28
17. [17]
  17 Bian X H, Cai W S, Shao X G, Chen D, Grant E R. Analyst, 2010, 135(11): 2841-2847
18. [18]
  18 Pierna J A F, Wahl F, de Noord O E, Massart D L. Chemom. Intell. Lab. Syst., 2002, 63(1): 27-39
19. [19]
  19 Walczak B, Massart D L. Chemom. Intell. Lab. Syst., 1998, 41(1): 1-15
20. [20]
  20 Hubert M, Vanden Branden K. J. Chemom., 2003, 17(10): 537-549
21. [21]
  21 Liu Z C, Cai W S, Shao X G. Sci. China Ser B-Chem., 2008, 51(8): 751-759
22. [22]
  22 Liu Z C, Ma X, Wen Y D, Wang Y, Cai W S, Shao X G. Sci. China Ser B-Chem., 2009, 52(7): 1021-1027
23. [23]
  23 Breunig M M, Kriegel H P, Ng R T, Sander J. Sigmod. Rec., 2000, 29(2): 93-104
24. [24]
  24 Li W, Goovaerts P, Meurens M. J. Arg. Food Chem., 1996, 44(8): 2252-2259
1. [1]
  1 Wold S, Ruhe A, Wold H, Dunn W J. SIAM J. Sci. Stat. Comput., 1984, 5(3): 735-743
2. [2]
  2 LIANG Miao,CAI Jia-Yue, YANG Kai, SHU Ru-Xin, ZHAO Long-Lian, ZHANG Lu-Da, LI Jun-Hui. Chinese J. Anal. Chem., 2014, 42(11): 1687-1691 梁淼, 蔡嘉月, 杨凯, 束茹欣, 赵龙莲, 张录达, 李军会. 分析化学, 2014, 42(11): 1687-1691
3. [3]
  3 ZHANG Lu-Da, SU Shi-Guang, WANG Lai-Sheng, LI Jun-Hui, YANG Li-Ming. Spectroscopy and Spectral Analysis, 2005, 25(1): 33-35 张录达, 苏时光, 王来生, 李军会, 杨丽明. 光谱学与光谱分析, 2005, 25(1): 33-35
4. [4]
  4 Li Y K, Shao X G, Cai W S. Talanta, 2007, 72(1): 217-222
5. [5]
  5 LIN Hao, ZHAO Jie-Wen, CHEN Quan-Sheng, CAI Jian-Rong, ZHOU Ping. Spectroscopy and Spectral Analysis, 2010, 30(4): 929-932 林颢, 赵杰文, 陈全胜, 蔡健荣, 周平. 光谱学与光谱分析, 2010, 30(4): 929-932
6. [6]
  6 Shao X G, Bian X H, Liu J J, Zhang M, Cai W S. Anal. Methods, 2010, 2(11): 1662-1666
7. [7]
  7 Wold S, Antti H, Lindgren F, Ohman J. Chemom. Intell. Lab. Syst., 1998, 44(1-2): 175-185
8. [8]
  8 Shao X G, Leung A K M, Chau F T. Acc. Chem. Res., 2003, 36(4): 276-283
9. [9]
  9 Norgaard L, Saudland A, Wagner J, Wagner J, Nielsen J P, Munk L, Engelsen S B. Appl. Spectrosc., 2000, 54(3): 413-419
10. [10]
  10 Centner V, Massart D L, de Noord O E, de Jong S, Vandeginste M B, Sterna C. Anal. Chem., 1996, 68(21): 3851-3858
11. [11]
  11 Cai W S, Li Y K, Shao X G. Chemom. Intell. Lab. Syst., 2008, 90(2): 188-194
12. [12]
  12 Li H D, Liang Y Z, Xu Q S, Cao D S. Anal. Chim. Acta, 2009, 648(1): 77-84
13. [13]
  13 Araujo M C U, Saldanha T C B, Galvao R K H, Yoneyama T, Chame H C, Visani V. Chemom. Intell. Lab. Syst., 2001, 57(2): 65-73
14. [14]
  14 Xu H, Liu Z C, Cai W S, Shao X G. Chemom. Intell. Lab. Syst., 2009, 97(1): 189-193
15. [15]
  15 Liang Y Z, Kvalheim O M. Chemom. Intell. Lab. Syst., 1996, 32(1): 1-10
16. [16]
  16 Pierna J A F, Jin L, Daszykowski M, Wahl F, Massart D L. Chemom. Intell. Lab. Syst., 2003, 68(1-2): 17-28
17. [17]
  17 Bian X H, Cai W S, Shao X G, Chen D, Grant E R. Analyst, 2010, 135(11): 2841-2847
18. [18]
  18 Pierna J A F, Wahl F, de Noord O E, Massart D L. Chemom. Intell. Lab. Syst., 2002, 63(1): 27-39
19. [19]
  19 Walczak B, Massart D L. Chemom. Intell. Lab. Syst., 1998, 41(1): 1-15
20. [20]
  20 Hubert M, Vanden Branden K. J. Chemom., 2003, 17(10): 537-549
21. [21]
  21 Liu Z C, Cai W S, Shao X G. Sci. China Ser B-Chem., 2008, 51(8): 751-759
22. [22]
  22 Liu Z C, Ma X, Wen Y D, Wang Y, Cai W S, Shao X G. Sci. China Ser B-Chem., 2009, 52(7): 1021-1027
23. [23]
  23 Breunig M M, Kriegel H P, Ng R T, Sander J. Sigmod. Rec., 2000, 29(2): 93-104
24. [24]
  24 Li W, Goovaerts P, Meurens M. J. Arg. Food Chem., 1996, 44(8): 2252-2259
1. [1]
  Wei Peng , Baoying Wen , Huamin Li , Yiru Wang , Jianfeng Li . Exploration and Practice on Raman Scattering Spectroscopy Experimental Teaching. University Chemistry, 2024, 39(8): 230-240. doi: 10.3866/PKU.DXHX202312062
2. [2]
  Ling Bai , Limin Lu , Xiaoqiang Wang , Dongping Wu , Yansha Gao . Exploration and Practice of Teaching Reforms in “Quantitative Analytical Chemistry” under the Perspective of New Agricultural Science. University Chemistry, 2024, 39(3): 158-166. doi: 10.3866/PKU.DXHX202308101
3. [3]
  Min Gu , Huiwen Xiong , Liling Liu , Jilie Kong , Xueen Fang . Rapid Quantitative Detection of Procalcitonin by Microfluidics: An Instrumental Analytical Chemistry Experiment. University Chemistry, 2024, 39(4): 87-93. doi: 10.3866/PKU.DXHX202310120
4. [4]
  Qi Wang , Yicong Gao , Feng Lu , Quli Fan . Preparation and Performance Characterization of the Second Near-Infrared Phototheranostic Probe: A New Design and Teaching Practice of Polymer Chemistry Comprehensive Experiment. University Chemistry, 2024, 39(11): 342-349. doi: 10.12461/PKU.DXHX202404141
5. [5]
  Jiahui CHEN , Tingting ZHENG , Xiuyun ZHANG , Wei LÜ . Research progress of near-infrared absorption inorganic nanomaterials in photothermal and photodynamic therapy of tumors. Chinese Journal of Inorganic Chemistry, 2024, 40(12): 2396-2414. doi: 10.11862/CJIC.20240106
6. [6]
  Han ZHANG , Jianfeng SUN , Jinsheng LIANG . Hydrothermal synthesis and luminescent properties of broadband near-infrared Na₃CrF₆ phosphor. Chinese Journal of Inorganic Chemistry, 2025, 41(2): 349-356. doi: 10.11862/CJIC.20240098
7. [7]
  Tianlong Zhang , Rongling Zhang , Hongsheng Tang , Yan Li , Hua Li . Online Monitoring and Mechanistic Analysis of 3,5-diamino-1,2,4-triazole (DAT) Synthesis via Raman Spectroscopy: A Recommendation for a Comprehensive Instrumental Analysis Experiment. University Chemistry, 2024, 39(6): 303-311. doi: 10.3866/PKU.DXHX202312006
8. [8]
  Zhuomin Zhang , Hanbing Huang , Liangqiu Lin , Jingsong Liu , Gongke Li . Course Construction of Instrumental Analysis Experiment: Surface-Enhanced Raman Spectroscopy for Rapid Detection of Edible Pigments. University Chemistry, 2024, 39(2): 133-139. doi: 10.3866/PKU.DXHX202308034
9. [9]
  Mengyao Shi , Kangle Su , Qingming Lu , Bin Zhang , Xiaowen Xu . Determination of Potassium Content in Tobacco Stem Ash by Flame Atomic Absorption Spectroscopy. University Chemistry, 2024, 39(10): 255-260. doi: 10.12461/PKU.DXHX202404105
10. [10]
  Jingyi Chen , Fu Liu , Tiejun Zhu , Kui Cheng . Practice of Integrating Ideological and Political Education into Raman Spectroscopy Analysis Experiment Course. University Chemistry, 2024, 39(2): 140-146. doi: 10.3866/PKU.DXHX202310111
11. [11]
  Tianlong Zhang , Jiajun Zhou , Hongsheng Tang , Xiaohui Ning , Yan Li , Hua Li . Virtual Simulation Experiment for Laser-Induced Breakdown Spectroscopy (LIBS) Analysis. University Chemistry, 2024, 39(6): 295-302. doi: 10.3866/PKU.DXHX202312049
12. [12]
  Xu Liu , Chengfang Liu , Jie Huang , Xiangchun Li , Wenyong Lai . Research on the Application of Diversified Teaching Models in the Teaching of Physical Chemistry. University Chemistry, 2024, 39(8): 112-118. doi: 10.3866/PKU.DXHX202402021
13. [13]
  Cheng Zheng , Shiying Zheng , Yanping Zhang , Shoutian Zheng , Qiaohua Wei . Synthesis, Copper Content Analysis, and Luminescent Performance Study of Binuclear Copper (I) Complexes with Isomeric Luminescence Shift: A Comprehensive Chemical Experiment Recommendation. University Chemistry, 2024, 39(7): 322-329. doi: 10.3866/PKU.DXHX202310131
14. [14]
  Kuaibing Wang , Honglin Zhang , Wenjie Lu , Weihua Zhang . Experimental Design and Practice for Recycling and Nickel Content Detection from Waste Nickel-Metal Hydride Batteries. University Chemistry, 2024, 39(11): 335-341. doi: 10.12461/PKU.DXHX202403084
15. [15]
  Yipeng Zhou , Chenxin Ran , Zhongbin Wu . Metacognitive Enhancement in Diversifying Ideological and Political Education within Graduate Course: A Case Study on “Solar Cell Performance Enhancement Technology”. University Chemistry, 2024, 39(6): 151-159. doi: 10.3866/PKU.DXHX202312096
16. [16]
  Liwei Wang , Guangran Ma , Li Wang , Fugang Xu . A Comprehensive Analytical Chemistry Experiment: Colorimetric Detection of Vitamin C Using Nanozyme and Smartphone. University Chemistry, 2024, 39(8): 255-262. doi: 10.3866/PKU.DXHX202312094
17. [17]
  Yongzhi LI , Han ZHANG , Gangding WANG , Yanwei SUI , Lei HOU , Yaoyu WANG . A two-dimensional metal-organic framework for the determination of nitrofurantoin and nitrofurazone in aqueous solution. Chinese Journal of Inorganic Chemistry, 2025, 41(2): 245-253. doi: 10.11862/CJIC.20240307
18. [18]
  Tingting Yu , Si Chen , Lianglong Sun , Tongtong Shi , Kai Sun , Xin Wang . Comprehensive Experimental Design for the Photochemical Synthesis, Analysis, and Characterization of Difluoropyrroles. University Chemistry, 2024, 39(11): 196-203. doi: 10.3866/PKU.DXHX202401022
19. [19]
  Yifang Xu , Jie Zheng , Liangbing Gan . The Conception and Practice of Cultivating Outstanding and Diverse Graduate Students in Basic Disciplines: A Case Study of Graduate Student Cultivation in Chemistry Program at Peking University. University Chemistry, 2024, 39(6): 1-6. doi: 10.3866/PKU.DXHX202404018
20. [20]
  Shengbiao Zheng , Liang Li , Nini Zhang , Ruimin Bao , Ruizhang Hu , Jing Tang . Metal-Organic Framework-Derived Materials Modified Electrode for Electrochemical Sensing of Tert-Butylhydroquinone: A Recommended Comprehensive Chemistry Experiment for Translating Research Results. University Chemistry, 2024, 39(7): 345-353. doi: 10.3866/PKU.DXHX202310096

Get Citation

PDF

XML

Metrics

PDF Downloads(0)
Abstract views(521)
HTML views(84)

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Outlier Detection for Multivariate Calibration in Near Infrared Spectroscopic Analysis by Model Diagnostics

References

Metrics

通讯作者: 陈斌, bchen63@163.com