基于机器视觉和非线性的芽形绿茶外形感官品质评价

Sensory quality evaluation for appearance of needle-shaped green tea based on computer vision and nonlinear tools 基于机器视觉和非线性的芽形绿茶外形感官品质评价 1 2 first-author 2 1 1 2 2 corresp ‡Corresponding Authors ‡Corresponding Authors †E-mail:8517809@qq.comq.s.chen@hotmail.com †E-mail:8517809@qq.comq.s.chen@hotmail.com 8517809@qq.com 1 corresp ‡Corresponding Authors ‡Corresponding Authors †E-mail:8517809@qq.comq.s.chen@hotmail.com †E-mail:8517809@qq.comq.s.chen@hotmail.com 8517809@qq.com School of Food and Biological Engineering, Jiangsu University, Zhenjiang 212013, China School of Food and Biological Engineering, Jiangsu University, Zhenjiang 212013, China Tea Research Institute, Chinese Academy of Agricultural Sciences, Hangzhou 310008, China Tea Research Institute, Chinese Academy of Agricultural Sciences, Hangzhou 310008, China Tea is one of the three greatest beverages in the world. In China, green tea has the largest consumption, and needle-shaped green tea, such as Maofeng tea and Sparrow Tongue tea, accounts for more than 40% of green tea (Zhu et al., 2017 2017 ). The appearance of green tea is one of the important indexes during the evaluation of green tea quality. Especially in market transactions, the price of tea is usually determined by its appearance (Zhou et al., 2012 2012 ). Human sensory evaluation is usually conducted by experts, and is also easily affected by various factors such as light, experience, psychological and visual factors. In the meantime, people may distinguish the slight differences between similar colors or textures, but the specific levels of the tea are hard to determine (Chen et al., 2008 2008 ). As human description of color and texture is qualitative, it is hard to evaluate the sensory quality accurately, in a standard manner, and objectively. Color is an important visual property of a computer image (Xie et al., 2014 2014 ; Khulal et al., 2016 2016 ); texture is a visual performance of image grayscale and color changing with spatial positions, which can be used to describe the roughness and directivity of the surface of an object (Sanaeifar et al., 2016 2016 ). There are already researchers who have used computer visual image technologies to identify the varieties, levels, and origins of tea (Chen et al., 2008 2008 ; Xie et al., 2014 2014 ; Zhu et al., 2017 2017 ). Most of their research targets are crush, tear, and curl (CTC) red (green) broken tea, curly green tea (Bilochun tea), and flat-typed green tea (West Lake Dragon-well green tea) as the information sources. However, the target of the above research is to establish a qualitative evaluation method on tea quality (Fu et al., 2013 2013 ). There is little literature on the sensory evaluation of the appearance quality of needle-shaped green tea, especially research on a quantitative evaluation model (Zhou et al., 2012 2012 ; Zhu et al., 2017 2017 ). 目的 2 针对传统人工感官评价缺陷，建立客观、量化、有效和无损的芽形绿茶外形品质表征方法。 创新点 2 采用图像特征（色泽和纹理）和AdaBoost改进的ELM（极限学习机）混合算法（Ada-ELM），明确了茶叶外形表象与人的感官感受间的非线性量化解析关系。 方法 2 基于机器视觉和图像处理技术，提取不同品质茶样的纹理和色泽等图像特征（表1），并与专家感官评分进行关联分析，筛选出10个极显著相关的特征变量（图1）。进而采用偏最小二乘法（PLS）和Ada-ELM，分别建立了针芽形绿茶外形感官品质的线性和非线性预测模型（表2）,并进行模型性能比较。 结论 2 非线性模型能更好地表征图像信息与感官评分间的关联，且AdaBoost集成算法能进一步提升ELM模型的预测精度和泛化性。综合而言，采用计算机图像特征量化评价芽形绿茶的外形品质是可行的，为拓展茶叶感官评审方法和规模化、自动化加工中品质的专家决策技术，提供了一种新的技术途径和思路。 Needle-shaped green tea Appearance quality Image feature Nonlinear tools Extreme learning machine (ELM) 芽形绿茶 外形品质 图像特征 非线性建模 极限学习机（ELM） Tea is one of the three greatest beverages in the world. In China, green tea has the largest consumption, and needle-shaped green tea, such as Maofeng tea and Sparrow Tongue tea, accounts for more than 40% of green tea (Zhu et al., 2017 2017 ). The appearance of green tea is one of the important indexes during the evaluation of green tea quality. Especially in market transactions, the price of tea is usually determined by its appearance (Zhou et al., 2012 2012 ). Human sensory evaluation is usually conducted by experts, and is also easily affected by various factors such as light, experience, psychological and visual factors. In the meantime, people may distinguish the slight differences between similar colors or textures, but the specific levels of the tea are hard to determine (Chen et al., 2008 2008 ). As human description of color and texture is qualitative, it is hard to evaluate the sensory quality accurately, in a standard manner, and objectively. Color is an important visual property of a computer image (Xie et al., 2014 2014 ; Khulal et al., 2016 2016 ); texture is a visual performance of image grayscale and color changing with spatial positions, which can be used to describe the roughness and directivity of the surface of an object (Sanaeifar et al., 2016 2016 ). There are already researchers who have used computer visual image technologies to identify the varieties, levels, and origins of tea (Chen et al., 2008 2008 ; Xie et al., 2014 2014 ; Zhu et al., 2017 2017 ). Most of their research targets are crush, tear, and curl (CTC) red (green) broken tea, curly green tea (Bilochun tea), and flat-typed green tea (West Lake Dragon-well green tea) as the information sources. However, the target of the above research is to establish a qualitative evaluation method on tea quality (Fu et al., 2013 2013 ). There is little literature on the sensory evaluation of the appearance quality of needle-shaped green tea, especially research on a quantitative evaluation model (Zhou et al., 2012 2012 ; Zhu et al., 2017 2017 ). Therefore, we applied a computer vision system to obtain the visible light image of tea’s appearance, extracted the color and texture characteristics, and associated the extracted characteristic variables with experts’ sensory scores. Then, with the combination of linear method and nonlinear tools, we established a sensory quality evaluation method for the appearance of needle-shaped green tea, and revealed the quantitative relation between image feature vectors and human senses. This study provided a theoretical reference and conducted exploration in the evaluation method and technical means on the appearance quality of tea. It has direct significance for the accurate evaluation of quality in tea transactions. In the study, in total 140 needle-shaped green tea samples were collected, which comprised 40 first-level, 77 second-level, and 23 third-level tea samples. The tea samples were evaluated by three sensory experts from the China Tea Science Society and Department of Tea Science of Zhejiang University (Hangzhou, China). The experts evaluated the appearance quality of the tea samples through the sensory evaluation method (GB/T 23776-2009) in the code review form, which includes color, cleanliness, and uniformity (AQSIQ and SAC, 2009 2009 ). The average score of their evaluation was taken as the final evaluation score. According to the appearance sensory scores divided by the Kennard-Stone method (Mir-Marqués et al., 2016 2016 ), 95 samples were used as the calibration set and the remaining 45 samples were used as the prediction set. First, a computer vision system was designed. The system consists of image sensor, sample cell, even light source, and a graphical user interface (GUI) software processing system, and realizes the image collection and data analysis as per the technical approach shown in Fig. 1 1 . The sensor uses the single lens reflex (SLR) camera (Canon EOS 60D 18MP, Japan); the GUI software processing system (software copyright No. 2013SR122183) uses MATLAB 2014b (The Mathworks, Natick, MA, USA) for compiling, which will automatically extract the image color and texture characteristics. SLR: single lens reflex; PLS: partial least squares; ELM: extreme learning machine; RGB: red-green-blue; HSV: blue-saturation-value; Lab: white-black, green-red, and blue-yellow Nine color indexes, including red channel mean ( R ), green channel mean ( G ), blue channel mean ( B ), hue mean ( H ), vision mean ( V ), saturation mean ( S ), lightness component mean ( L *), a component mean ( a *), and b component ( b *), were obtained by transforming the color models between red-green-blue (RGB), hue-saturation-value (HSV), and white-black, green-red, and blue-yellow (Lab). Based on the statistical attributes of a gray histogram, six texture characteristics as mean grey ( m ), standard deviation ( δ ), smoothness ( r ), third moment ( μ ), uniformity ( U ), and entropy ( e ) were calculated (Xie et al., 2014 2014 ; Sanaeifar et al., 2016 2016 ), and in total 15 image characteristic variables (color and texture) were obtained. Partial least squares (PLS) method and extreme learning machine (ELM) method (Tian and Mao, 2010 2010 ; Huang et al., 2012 2012 ) were used respectively to conduct linear and nonlinear quantitative modeling (Yu et al., 2016 2016 ). The performance parameters (PCs, number of principal components; R c , correlation coefficient of calibration; R p , correlation coefficient of prediction; RMSEC, root mean square error of calibration; RMSEP, root mean square error of prediction; Bias, bias ratio; SEP, standard error of prediction; CV, coefficient of variation; RPD, residual predictive deviation value of prediction) from the literature were used for reference for the evaluation indexes of the model performances (Huang et al., 2014 2014 ). Generally, the smaller the RMSEP, SEP, CV, and Bias, the higher the R c , R p , RPD, and the accuracy and generalization of the model will be (Chia et al., 2012 2012 ; Luo et al., 2014 2014 ). All the data were processed under MATLAB 2014b. Table 1 1 shows the results of image feature and sensory score for different levels of quality of needle-shaped green tea. The correlation analysis on the appearance evaluation and vision characteristic variables of the test samples showed that: except for the values of R , G , V , L *, and U , all the image characteristic parameters are significantly correlated with the appearance scores, and exhibited the highest correlation coefficient at b * value (0.740). The analysis indicated that the green tea has the highest appearance sensory score when the color is yellow-green or tender green rather than green, yellow, or dark. This is also in accordance with the sensory evaluation standard of green tea. First-level quality tea; Second-level quality tea; Third-level quality tea Parameter Number of samples Mean Minimum Maximum Standard deviation Variance R 23 a a 0.441 0.362 0.484 0.034 0.0011 76 b b 0.500 0.386 0.613 0.056 0.0031 40 c c 0.490 0.411 0.645 0.073 0.0053 G 23 a a 0.411 0.339 0.447 0.030 0.0009 76 b b 0.456 0.348 0.555 0.053 0.0028 40 c c 0.448 0.373 0.615 0.071 0.0051 B 23 a a 0.293 0.242 0.332 0.022 0.0005 76 b b 0.308 0.215 0.395 0.043 0.0018 40 c c 0.301 0.236 0.480 0.067 0.0045 H 23 a a 48.925 47.350 51.079 1.019 1.0391 76 b b 47.294 43.860 51.107 1.322 1.7470 40 c c 47.855 45.463 50.303 1.056 1.1149 S 23 a a 0.233 0.197 0.276 0.022 0.0005 76 b b 0.270 0.221 0.320 0.022 0.0005 40 c c 0.277 0.173 0.318 0.036 0.0013 V 23 a a 0.382 0.314 0.416 0.028 0.0008 76 b b 0.421 0.316 0.517 0.050 0.0025 40 c c 0.413 0.341 0.580 0.070 0.0049 a * 23 a a −2.202 −2.723 −1.730 0.303 0.0918 76 b b −2.147 −3.141 −1.255 0.389 0.1517 40 c c −2.340 −2.980 −1.693 0.323 0.1042 b * 23 a a 13.354 11.425 16.140 1.397 1.9510 76 b b 15.936 13.669 18.338 1.074 1.1545 40 c c 16.212 11.278 18.488 1.605 2.5753 L * 23 a a 70.045 64.715 72.526 2.142 4.5888 76 b b 73.065 65.473 79.323 3.448 11.8870 40 c c 72.426 67.388 82.466 4.496 20.2108 m 23 a a 0.380 0.313 0.414 0.028 0.0008 76 b b 0.420 0.315 0.516 0.050 0.0025 40 c c 0.412 0.340 0.579 0.070 0.0049 δ 23 a a 0.045 0.037 0.054 0.006 0.0000 76 b b 0.048 0.036 0.071 0.007 0.0001 40 c c 0.052 0.040 0.082 0.010 0.0001 r 23 a a 2.057 1.333 2.892 0.499 0.2486 76 b b 2.366 1.327 5.005 0.733 0.5372 40 c c 2.784 1.602 6.637 1.180 1.3915 μ 23 a a 9.295 4.600 12.008 2.232 4.9799 76 b b 6.847 3.306 10.297 1.478 2.1838 40 c c 6.361 2.738 8.809 1.256 1.5782 U 23 a a 6.558 6.066 7.288 0.320 0.1025 76 b b 6.122 5.267 8.464 0.562 0.3164 40 c c 6.811 5.060 21.905 3.390 11.4905 e 23 a a 7.536 7.387 7.608 0.052 0.0028 76 b b 7.638 7.506 7.743 0.044 0.0020 40 c c 7.658 7.336 7.803 0.086 0.0075 Appearance score 23 a a 84.304 79.500 85.500 1.724 2.9713 76 b b 87.967 86.000 89.500 1.008 1.0156 40 c c 90.675 90.000 92.500 0.730 0.5327 As a single-hidden layer feedforward neural network (SLFN) algorithm, the ELM has a better learning speed and generalization performance than classical quantitative analysis methods, such as PLS and back propagation artificial neuronal network (BP-ANN) (Tian and Mao, 2010 2010 ). AdaBoost is an integration machine learning algorithm, which is often used with a plurality of weak learning algorithms to enhance ultimate performance. This paper takes ELM as the weak predictor to form the AdaBoost strong predictor and the principal component as the input item of the strong predictor to establish the Ada-ELM hybrid modeling method. It optimizes the parameters based on the RMSEC values of the model, with PCs equal to 6, and parameter Φ (the prediction error threshold value) equal to 0.061, the RMSEC of the model reaches the minimum (0.547), and the R P , RMSEP, Bias, SEP, CV, and RPD of the prediction set are 0.892, 0.874, −0.148, 0.226, 0.018, and 2.014, respectively. The performances of the PLS linear model and the nonlinear model (ELM and Ada-ELM) are compared in Table 2 2 . As shown, the performance parameters of the nonlinear model prediction set are obviously better than those of the linear model. The Ada-ELM model has the best prediction performance. Small SEP and CV mean a small degree of sample deviation and discrete variation. In particular, the RPD value is greater than 2, which shows that the model has good prediction and can be used for quantitative analysis. PCs, used latent variables; R c , correlation coefficient of calibration; R p , correlation coefficient of prediction; RMSEC, root mean square error of calibration; RMSEP, root mean square error of prediction; Bias, bias ratio; SEP, standard error of prediction; CV, coefficient of variation; RPD, residual predictive deviation value of prediction 2 Method 2 PCs 3 Calibration sets 6 Prediction set R c RMSEC Bias R p RMSEP Bias SEP CV RPD  PLS 7 0.834 1.387 −0.003 0.777 1.215 −0.148 0.226 0.018 1.271  ELM 6 0.889 1.147 0.003 0.860 1.032 −0.326 0.246 0.019 1.625  Ada-ELM 6 0.973 0.576 −0.003 0.892 0.874 −0.148 0.226 0.018 2.014 As sensory evaluation is to use eyes to observe the color, evenness, strip thickness, purity, uniformity, and tenderness of the tea samples, integrate the vision information, and make a comprehensive evaluation through a complicated neural network system, the final sensory score has a certain nonlinear relationship with the color and the shape. The PLS method only treated the linear relationship between the variables and sensory score, which ignored the existence of the nonlinear relationship (Chen et al., 2008 2008 ; Fu et al., 2013 2013 ). However, ELM is a nonlinear artificial neural network modeling method, which has stronger adaptive and generalization ability (Tian and Mao, 2010 2010 ). Hence, it has better prediction accuracy than the PLS model. To make up for the shortage existing in the traditional sensory evaluation methods, and with the purpose of quantitatively and objectively evaluating the appearance quality of needle-shaped green tea, this paper collected the texture and color characteristics of different levels of tea samples, adopted a hybrid algorithm of AdaBoost algorithm and ELM neural network, and established a nonlinear Ada-ELM quantitative evaluation model. The results showed that the model can be used to evaluate the appearance quality of needle-shaped green tea, and the nonlinear modeling method can better represent the quantitative analytical relation between the image information and sensory scores. This study provided an effective technical approach and idea for the development of sensory evaluation methods of tea. It is possible that transferring veteran tea makers’ experience to a neural network to develop an expert decision support system or special purpose instrument, which would help achieve automatic control of needled-shaped green tea processing, and produce needled-shaped green tea with uniform quality and stable style. It is one of the most promising techniques for large-scale tea processing quality evaluation, which has become the main constraint in realizing automated and intelligent green tea processing technology. In addition, this has prospects of broad application in the tea trade as well as tea making and blending technology. References Q Chen , J Zhao , J Cai . Identification of tea varieties using computer vision . Trans ASABE , 2008 . 51 ( 2 ): 623 - 628 . DOI: 10.13031/2013.24363 http://doi.org/10.13031/2013.24363 . Identification of tea varieties using computer vision KS Chia , HA Rahim , RA Rahim . Neural network and principal component regression in non-destructive soluble solids content assessment: a comparison . J Zhejiang Univ-Sci B (Biomed & Biotechnol) , 2012 . 13 ( 2 ): 145 - 151 . DOI: 10.1631/jzus.B11c0150 http://doi.org/10.1631/jzus.B11c0150 . Neural network and principal component regression in non-destructive soluble solids content assessment: a comparison X Fu , L Xu , X Yu , 等 . Robust and automated internal quality grading of a Chinese green tea (Longjing) by near-infrared spectroscopy and chemometrics . Spectroscopy , 2013 . 2013 ( 1 ): 367 - 383 . DOI: 10.1155/2013/139347 http://doi.org/10.1155/2013/139347 . Robust and automated internal quality grading of a Chinese green tea (Longjing) by near-infrared spectroscopy and chemometrics AQSIQ (General Administration of Quality Supervision, Inspection and Quarantine), SAC (the Standardization Administration of China) Methodology of Sensory Evaluation of Tea, GB/T 23776-2009 , 2009 . China Standard Publishing House, Beijing , (in Chinese) . Methodology of Sensory Evaluation of Tea, GB/T 23776-2009 GB Huang , H Zhou , X Ding , 等 . Extreme learning machine for regression and multiclass classification . IEEE Trans Syst Man Cybern Part B (Cybern) , 2012 . 42 ( 2 ): 513 - 529 . DOI: 10.1109/TSMCB.2011.2168604 http://doi.org/10.1109/TSMCB.2011.2168604 . Extreme learning machine for regression and multiclass classification L Huang , J Zhao , Q Chen , 等 . Nondestructive measurement of total volatile basic nitrogen (TVB-N) in pork meat by integrating near infrared spectroscopy, computer vision and electronic nose techniques . Food Chem , 2014 . 145 ( 7 ): 228 - 236 . 24128472 DOI: 10.1016/j.foodchem.2013.06.073 http://doi.org/10.1016/j.foodchem.2013.06.073 . Nondestructive measurement of total volatile basic nitrogen (TVB-N) in pork meat by integrating near infrared spectroscopy, computer vision and electronic nose techniques U Khulal , J Zhao , W Hu , 等 . Nondestructive quantifying total volatile basic nitrogen (TVB-N) content in chicken using hyperspectral imaging (HSI) technique combined with different data dimension reduction algorithms . Food Chem , 2016 . 197 ( Pt B ): 1191 - 1199 . 26675857 DOI: 10.1016/j.foodchem.2015.11.084 http://doi.org/10.1016/j.foodchem.2015.11.084 . Nondestructive quantifying total volatile basic nitrogen (TVB-N) content in chicken using hyperspectral imaging (HSI) technique combined with different data dimension reduction algorithms Y Luo , WL Li , WH Huang , 等 . Rapid quantification of multi-components in alcohol precipitation liquid of Codonopsis Radix using near infrared spectroscopy (NIRS) . J Zhejiang Univ-Sci B (Biomed & Biotechnol) , 2014 . 18 ( 5 ): 383 - 392 . DOI: 10.1631/jzus.B1600141 http://doi.org/10.1631/jzus.B1600141 . Rapid quantification of multi-components in alcohol precipitation liquid of Codonopsis Radix using near infrared spectroscopy (NIRS) A Mir-Marqués , C Elvirasaez , ML Cervera , 等 . Authentication of protected designation of origin artichokes by spectroscopy methods . Food Control , 2016 . 59 ( 1-3 ): 74 - 81 . DOI: 10.1016/j.foodcont.2015.05.004 http://doi.org/10.1016/j.foodcont.2015.05.004 . Authentication of protected designation of origin artichokes by spectroscopy methods A Sanaeifar , A Bakhshipour , MD La Guardia . Prediction of banana quality indices from color features using support vector regression . Talanta , 2016 . 148 54 - 61 . 26653423 DOI: 10.1016/j.talanta.2015.10.073 http://doi.org/10.1016/j.talanta.2015.10.073 . Prediction of banana quality indices from color features using support vector regression H Tian , Z Mao . An ensemble ELM based on modified AdaBoost.RT algorithm for predicting the temperature of molten steel in ladle furnace . IEEE Trans Automat Sci Eng , 2010 . 7 ( 1 ): 73 - 80 . DOI: 10.1109/TASE.2008.2005640 http://doi.org/10.1109/TASE.2008.2005640 . An ensemble ELM based on modified AdaBoost.RT algorithm for predicting the temperature of molten steel in ladle furnace YL Yu , W Li , DR Sheng , 等 . A hybrid short-term load forecasting method based on improved ensemble empirical mode decomposition and back propagation neural network . J Zhejiang Univ-Sci A (Appl Phys & Eng) , 2016 . 17 ( 2 ): 101 - 114 . DOI: 10.1631/jzus.A1500156 http://doi.org/10.1631/jzus.A1500156 . A hybrid short-term load forecasting method based on improved ensemble empirical mode decomposition and back propagation neural network C Xie , X Li , Y Shao , 等 . Color measurement of tea leaves at different drying periods using hyperspectral imaging technique . PLoS ONE , 2014 . 9 ( 12 ): e113422 25546335 DOI: 10.1371/journal.pone.0113422 http://doi.org/10.1371/journal.pone.0113422 . Color measurement of tea leaves at different drying periods using hyperspectral imaging technique X Zhou , Y Ye , Z Zhou , 等 . Study on quality evaluation of Dafo Longjing tea based on near infrared spectroscopy . Spectrosc Spect Anal , 2012 . 32 ( 11 ): 2971 (in Chinese) DOI: 10.3964/j.issn.1000-0593(2012)11-2971-05 http://doi.org/10.3964/j.issn.1000-0593(2012)11-2971-05 . Study on quality evaluation of Dafo Longjing tea based on near infrared spectroscopy H Zhu , Y Ye , H He , 等 . Evaluation of green tea sensory quality via process characteristics and image information . Food Bioprod Proc , 2017 . 102 ( 03 ): 116 - 122 https://doiorg/101016/jfbp201612004 . . Evaluation of green tea sensory quality via process characteristics and image information Compliance with ethics guidelines Chun-wang DONG, Hong-kai ZHU, Jie-wen ZHAO, Yong-wen JIANG, Hai-bo YUAN, and Quan-sheng CHEN declare that they have no conflict of interest. This article does not contain any studies with human or animal subjects performed by any of the authors. 10.1631/jzus.B1600423 Project supported by the National Natural Science Foundation of China (No. 31271875), the Natural Science Foundation of Zhejiang Province (No. Y16C160009), and the Key Research Projects of Zhejiang (No. 2515C02001), China 2016-10-01 2017-05-10 2016-12-16 2022-01-05 Copyright © Zhejiang University and Springer-Verlag Berlin Heidelberg 2017 2017 Journal of Zhejiang University-SCIENCE B (Biomedicine and Biotechnology) 06 18