7.1 Introduction

The production of best featured quality seedlings is important if yields are to be improved and quality achieved. Plant seedling production is conducted in most developed countries that tackle by specialized companies for exclusive operation. In India, the plant seedling production methodology is moderately growing from susceptible field [1] to production in protected seedling trays or raised beds. In some intensive growing areas, advanced seedling manufacturing industries also take off. Recently, yields per unit area are normally at lowest point. Most of the seeds on the retail market are not certified and have inferiority low production quality. The prominent moment [2] of germination of seeds always necessitates extra care and attention. Automatic plant image recognition is the major promising approach for bridging the taxonomic gap in agriculture, which is attracting significant attention in both the farming and computer communities. Therefore, in this chapter we describe how to care for seedlings by classifying using deep learning plant technique that helps in growing up the healthy plants. Deep learning inclusively is referred as one of the machine learning comprehensive techniques which usually involves the hierarchical features of learning. In several forms of computer vision activities, these techniques have been appearing to be effective, including image classification and their detection [3]. Learning is an agile area of research in field of agriculture and its practices to science of plant even now in early stages. Previous research has essentially shown the benefit of profound learning for the same task in complex images plant tasks over conventional engineering under computer vision activities. These activities involve counting of leaves, estimating the age of plants, mutant classification, identification of plant diseases [4], fruit and other seedling classification [5]. The small part of the existing research on the deep learning in plant image classification [6] explores assurance for future work in this area. The weed classification model is then checked for images of seedling and evaluation done using precision, selectivity and loss. The findings of the study examined would be a magnificent offering to the ongoing development growth of crop germinating management care and to providing the newest technique for promoting and encouraging agricultural practices. Convolutional Neural Networks are nowadays considered narrowly the dominant method just for object classification. The Deep Convolutional Neural Network (CNN) is trained by tuning the parameters of the network to enhance linking during the training process and performs conscientiously in plant seedling classification [7]. In computer vision concept, CNNs were known to the strong visual models which produce feature hierarchies that allow for accurate and remarkable segmentation and also execute predictions comparatively faster than other more algorithms, and simultaneously retaining competitive performance.

7.1.1 Architecture of CNN

In bygone models of image classification, there is a use of actual raw pixels to classify or identify the images. Images can be categorized by color histogram and edge detection. These approaches have been exclusively successful but before more complex variants are encountered. That is where the classical image recognition keeps failing, because some other features are not taken into consideration by the model and thus the consideration of convolutional neural network (ConvNet). CNN is a type of model of the neural network that helps to extract feature or higher image representations [8]. CNN takes the raw pixel data from the image, trains the model, and then automatically extracts features [9] such as margins, highlighted patterns from the image for better classification. CNN began in 2012 with AlexNet, and has grown exponentially [10, 11]. Researchers have advanced from AlexNet with 8 layers to ResNet 152 layers [12]. CNN is blowing competition in concept of accuracy. Recommending system programs, natural language processing and much more are also successfully implemented. CNN’s one biggest benefit compared with its predecessors is that it identifies the essential features automatically without any human oversight interaction.

7.1.1.1 Principles of ConvNet

Convolution: On the image, a convolution is applied then calculates its input data and the pixel values of the filter dot product using a convolution filter to generate the appropriate feature chart. The convolved features will also adjust in order to minimize loss of prediction that depends onto the filter values impacted by some gradient descent.

Max Pooling: ConvNet requires max pooling for substitute output to max analysis, reducing data size as well as time for processing. This approach is useful in defining or removing highly impacting features which limits the chance of overfitting. This layers learn each feature map independently and reducing the height and width without depth intact. Activation Function: Apply activation function after every convolutional and max pooling operation such rectified linear unit (ReLU), sigmoid and softmax. This approach is effective in solving the diminishing gradients.

Figure 7.1 shows the cumulative CNN architecture comprises of two primary parts: Extractor feature functionality and Classifier. The layer throughout the network expects to receive certain output from the immediate prior feature map layer as in its input throughout the feature extraction stages, and transfers its output as its input towards the subsequent layer. The architecture of the CNN [13] comes in the form of a mixture of three-layered types: convolution, max pooling, and identification. Throughout the middle and lower stages of the network there have been two categories of layers: deep convolutional layers and max-pooling layers. There is an even equally spaced layers are designed for convolutions, as well as the odd marked layers are there for max-pooling. The convolution network output and also the max-pooling layers are grouped together into the 2D plane designated feature mapping. Typically, each plane across layers is generated from the composition of one or many prior layer planes. Plane nodes are directly linked to a slight region of every one of the preceding layers integrated planes. That node throughout the convolution layer extracting features mostly on input layer from its input images through convolution operations.

7.1.2 Residual Network (ResNet)

ResNet is a strong backbone model which is very widely used in multiple computer vision tasks and developed by research team from Microsoft. Model design is focused on the principle of residual blocks that enable use of shortcut connections. Throughout the network architecture, these are basically connections where all the input is kept as it is (not weighted) and conveyed to a deeper layer. It implemented revolutionary to prove a new approach for deep neural networks to a huge problem. By using Identification shortcut link or skipping connections that miss one or several layers, ResNet addresses the problem of gradient vanishing. ResNets have varying sizes depends on how broad any of the model layers are and how many layers it has, such that ResNet-18, ResNet-34, ResNet-50, ResNet-101, ResNet-110, ResNet-152, ResNet-164, ResNet-1202. Deep residual networks (ResNets) consist of several stacked “Residual Units”.

In a simplified form, each unit can be represented by Equations (7.1) and (7.2):

(7.1)

(7.2)

where x_i and x_{i + 1} are input and output of the ith unit, and F is a residual function with h(x_i) = x_i is an identity mapping [14] and f is a ReLU function.

Figure 7.2 shows the mapping between regular and residual block. The ideal mapping of regular block that obtain by learning is: f(x) and Ideal mapping of residual block is:

7.2 Related Work

The key premises of the automated system for classifying plants are shown in Figure 7.3. The plant images will originally be acquired through real camera, scanner, as well as some other equipment. Then, those images have been pre-processed to complete remove of noise and enhance efficiency. Noise appears as image pixels value during image processing and does not reflect the actual intensities of an image.

Image enhancement is indeed a practice often used highlight each image’s feature information [14]. Deleting the image noises is indeed a required step to emphasize or highlight the important image features. Accordingly, the region of interest (ROI) was segregated mainly from images, assisted by feature extraction. Finally, the characteristics extracted are transferred to a method of classification or identification.

Very recently Jiang et al. [13] reviewed CNN-based processing plant phenotyping approaches that have been extensively examined to have advantages and drawbacks of just using them with different particular plant phenotyping activities. Effective learning is yet another attempt to reduce the risk of labeling results. Associated with normative data augmentation, learning environment seeks to identify and mark samples which really optimize the output of a model. So, most samples should not be validated to just save time as well as labor costs. Data collection is indeed a productive way to annotate data which needs lesser labor productivity expenditure. Some experiments have shown how quantitative research can be used to rapidly mark massive image repositories for automated learning algorithms. There are several commercial data annotation applications, in particular, including as amazon robotic Turk as well as Crowd flower. Through all those services, data annotation can indeed be ensured with a satisfactory standard and throughput. CNN-based approaches stated their tremendous potential through all these studies to solve the most difficult problems faced in different plants phenotyping application scenarios. Other forms of end-to-end CNN structure in general and especially have greatly simplified the mechanism of order to extract phenotypic traits through images. This would allow data processing when being improved, and eventually plant phenotyping techniques.

Smith et al. [15] concentrated on annotation period, as time criteria for annotation instead of the amount of images readily accessible to become more important to the needs of their majority bulk of plant data analysis groups wanting to use profound learning towards image analysis. The results, especially for appropriate remedial training, affirm first hypothesis besides demonstrating that any deep learning prototype could be trained with high accuracy with less of around two hours for annotation also for three corresponding datasets of contrasting target items, context and image reliability and illustrate the feasibility through training a specific model utilizing short-term annotations questioning arguments that scores to around thousands of data image or significant marking data required for using CNNs. In reality, authors are often anticipating longer cycles of annotation to offer more change. The R2 to ward corrective development had a strong association with length of annotation suggesting whether it would help to enhance efficiency by investing more time annotating. There was the tendency to accept recognize increasing percentage of perceived images without any further annotation probably later in mostly in the preventive training suggesting less of that images needed corrections because as performance throughout the model started improving. This aligns again with decrease to growth rate in the total quantity of corrections showing continued progress throughout the accuracy within the model across time during their corrective training. Although preventive annotation appeared to produce better accurate than dense models, their lack of such a statistically substantial difference prohibits a more concrete conclusion about its benefits for corrective against dense annotation.

Tan et al. [16] stated that automated fully plant species recognition system may aid speedy species of plant identification by botanists and laymen. Advanced learning is comprehensive in extracting features, since it becomes superior in producing deeper image knowledge. A novel CNN-based approach called D-Leaf has been introduced in this study. The leaves images became pre-trained as well as the features have been extracted employing three separate CNN concepts, namely pre-processing AlexNet, AlexNet as well as D-Leaf, which were finely tuned. The researchers concluded that CNN is best than conventional morphological approaches towards the extracting features of plant species. In comparison with the CNN, more quite pre-processing work needs to be accomplished when utilizing conventional strategies. In addition, in this research, CNN would be found be a preferable method of extraction of features, instead of a method of classification. Compared to various classifiers, the ANN classifier along with its CNN feature extractor recorded a very optimal result. In fact, best result of this research would be to achieve better precision testing since using a D-Leaf besides extraction of the feature as well as the ANN as a classifier. In addition, D-Leaf achievement is validated using CV methods, and also some MalayaKew, Flavia and Swedish datasets. The success of the validation showed that the D-Leaf system could be used for automatic classification of plant species.

Alimboyong et al. [17] concluded a new approach for early growth process categorization of plant species was researched using advantage of deep learning neural convolutional networks. This methodology includes a CNN during training and do augmentation of data to classify 12 species of plant using a range of image transformations: equalization of the dimensions, rotation, shift, levelling, resizing and histogram and became a relevant contribution fairly towards ongoing development suggested in the field of environmental research as well as to universal objective of increase in global agricultural output yield. The classification model of plant is then checked for seedling different images and evaluated successfully using precision, sensitivity and specificity and the findings of this research has an outstanding supplement to the ongoing growth of crop management and to offering a newest technique for promoting and encouraging farming different practices.

Rashad et al. [9] introduced a plant classification strategy that is focused on characterizing the properties of the textures. Author used the combined classifiers learning vector quantization, and randomly took out 30 blocks of every texture as a training sample and then some 30 blocks as a test sample. The study identified that integrated classification method offers high performance which is preferable to another test method. The experimental findings demonstrated the applicability of the algorithm and its average correct recognition rate was good. The proposed system has an advantage in its capabilities to recognize and classify the crop from the small proportion of every leaf without depending on whether the texture of a leaf and its color characteristics, although the system essentially depends mostly on textural characteristics. The system is therefore useful to botany researchers when they want to recognize a damaged plant, as this can only be done depending on a small part of the rotted and damaged plant.

Nkemelu et al. [18] addressed an efficient deep learning model for seedlings classification that helped farmers for optimize crop yields and significantly reduce losses. In this paper, author proposed a deep, convolutional neural network approach for classification of plant seedlings. A dataset has been used which contains images of about 960 unique plants belonging to 12 species at several growth stages. In the wild the model detected and distinguished a weed from other plants. A baseline version of the proposed system obtained a precision. The system proposed could be expanded to operate with robotic arms to conduct actual weeding operations in large farmlands. The findings have shown that, when used in farming automation, CNN-driven seedling classification applications have the potential to optimize crop yield and improve productivity and efficiency when appropriately designed.

7.3 Proposed Work

The aim of the weed cultivation activity is to provide the best crop opportunity to develop and grow vigorously up to maturity time. The key goals of weed control are to improve the soil conditions by reducing soil surface convection. It encourages the growth of the desirable plant species and prevents the establishment of weeds in a cultivated crop, pasture, or farmland. Encouraged by this presumption, weed cultivation applications are seeking to better classify input data into the underlying seedling group. Plant Seedling Dataset has been used from an evolutionary experimental viewpoint and includes the following twelve types of plant crop groups for plant classification: BlackGrass, Charlock, Cleavers, Common Chickweed, Common wheat, Fat Hen, Loose silky bent, Maize, Scentless Mayweed, Shepherd Purse, Small flowered Cranesbill and Sugar beet category. The objective is to train an Advance CNN model (Resnet50) efficient of classifying the plant crop into these twelve categories and this makes it a challenge in respect of deep learning classification.

7.3.2 Data Pre-Processing

As per ResNet’s need for transfer learning implementation, images are pre-processed as performed in the architecture of the origin. Additionally, their processing system required input in (B, G, R) order while by standard python prefers (R, G, B), so the images have to be transformed from BGR to RGB. Therefore, in data pre-processing, we implemented color space transformation and image enhancement to unique images of plants pertinence with 12 species of plants in separate seedling stages development and conversion of color space to evaluate chromaticity and luminosity layers in order to improve visual analysis. If images vary then resize them from 54 × 54 to as much as M × N image sizes. The RGB images of that same crops are modified into the L × a × b color space by empirically adjusting the existing color channel for creating a new color channel appropriate for classification within almost every species of plants.

(7.3)

(7.4)

By comparison Equations (7.3) and (7.4) represent each pixel’s nonconformity.

R and G and B denoted the pixel represented values for red and green and blue channels, while e is used to prevent zero divisions. To allow the very same image dimensions for training and validation using the machine learning model, and the image sizes should be of M × M size, in which M = 256. All images of whole datasets are resized to a color space of 256 × 256 RGB. Following resize of image presents a newest different image with the set of rows and also columns defined by a 1D array of two elements with the representation of [256, 256].

7.3.3 Data Annotation and Augmentation

Image annotation, the process in which images are categorized according to features extracted. This approach functions as an individual class through semantic concept and applies each concept as a single classifier. For image retrieval networks this method is often used to classify and accurately locate specific images from a database automatically. However, in this proposed methodology we manually annotate the locations of each image which contain the leaves with the help of bounding box. Depending on its clearness status certain leaves can look identical. Figure 7.6 shows the annotated image.

Annotation procedure can be capable of labeling the class and position in the image of the leaf areas. The results of the this stage are the dimensions including its bounding boxes of varying sizes with their respective crop class, which will then be assessed as the intersection over union (IoU) with the expected results during the testing.

7.3.4 Training and Fine-Tuning

Architecture to random weights is initialized and trained for a number of epochs. The model recognizes attributes from data with each epoch. The training was using the stochastic momentum gradient descent (SGDM) optimizer, with an initial learning rate of 0.001, a mini batch size of 8 and 65 epochs overall. Then complete full connected layers of ResNet-50 are accurately replaced with dense layer which has new segments such as batch normalization and dropout to learn various features that specific to the seedling of plant. For each batch, batch normalization normalizes previous layer activations as it helps train networks faster and also normalizes each layer’s inputs to address the internal covariate, change problem. Dropout is a strategy used during the training to prevent a model from overfitting by dropping the parts of hidden units into one layer. The performance of evaluated model is assessing using accuracy as shown in Equation (7.5).

(7.5)

7.4 Result and Evaluation

In ANACONDA 3.0 the method was put into practice. A standard PC fitted with 2.30 GHz Intel(R) Core(TM) i5-3567 CPU, 64-bit operating system, 8 GB RAM and 2 GB AMD Radeon R5 M330 graphics engine run on Windows 10 operating system has been configured throughout the entire training and testing phase of the model mentioned significantly in this chapter. First, the description is given for the dataset used in this analysis and our dataset was accurately divided into 85% training set and 15% validation set to conduct the experiments. Evaluation is carried out on the Validation set after the testing is completed on the training set.

7.4.1 Metrics

Logarithmic loss of multiple classes also widely recognized as the categorical cross entropy is used as a metric for this work and calculated using Equation (7.6). A perfect classifier gets the log loss of 0.

(7.6)

7.4.2 Result Analysis

7.4.2.1 Experiment I

The first experiment on the plant seedling dataset performed with batch normalization. The training accuracy was (95%) with logarithm loss (0.0634) and the validation accuracy was (85%) with logarithm loss (0.567). Figures 7.7 and 7.8 show the accuracy and loss curve for the experiment 1.

7.4.2.2 Experiment II

The second experiment performed without batch normalization. The training accuracy was (93%) with logarithm loss (0.2578) and the validation accuracy was (83%) with logarithm loss (0.8382). Figures 7.9 and 7.10 show the accuracy and loss curve for the experiment 2.

7.5 Conclusion

Protection of crops is not really a trivial matter in organic farming. It depends on a detailed knowledge of that same harvested crops and their probable weeds. In our system we developed comprehensive deep learning models for the classification of plant species by means of leaf images of healthy plants, predicated on advanced convolutional neural networks framework. A dataset has been used that includes images of around 2,400 distinct plants actually belonging to twelve species at several significant growth stages. Our experimental findings showed how our deep-learning model is capable of successfully categorizing various types of plants. We hope that our proposed program will make a significant contribution to the research in agriculture.

References

1. Kołodziejek, J., Effect of seed position and soil nutrients on seed mass, germination and seedling growth in Peucedanum oreoselinum (Apiaceae). Sci. Rep., 7, 1, 1–11, 2017.

2. Yuan, X. and Wen, B., Seed germination response to high temperature and water stress in three invasive Asteraceae weeds from Xishuangbanna, SW China. PLoS One, 13, 1, e0191710, 2018.

3. He, K., Zhang, X., Ren, S., Sun, J., Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016.

4. Ubbens, J., Cieslak, M., Prusinkiewicz, P., Stavness, I., The use of plant models in deep learning: An application to leaf counting in rosette plants. Plant Methods, 14, 1, 6, 2018.

5. Akila, M. and Deepan, P., Detection and classification of Plant Leaf diseases by using Deep Learning Algorithm. Int. J. Eng. Res. Technol., 6, 2–7, 2018.

6. Mohanty, S.P., Hughes, D.P., Salathé, M., Using deep learning for image-based plant disease detection. Front. Plant Sci., 7, 1419, 2016.

7. Malinao, R.M.L. and Hernandez, A.A., Artocarpus Trees Classification using Convolutional Neural Network, in: 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM), IEEE, pp. 1–6, 2018.

8. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15, 1, 1929–1958, 2014.

9. Rashad, M.Z., El-Desouky, B.S., Khawasik, M.S., Plants images classification based on textural features using combined classifier. Int. J. Comput. Sci. Inf. Technol., 3, 4, 93–100, 2011.

10. Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Chen, T., Recent advances in convolutional neural networks. Pattern Recognit., 77, 354–377, 2018.

11. O’Shea, K. and Nash, R., An introduction to convolutional neural networks, arXiv preprint arXiv:1511.08458, 2015. https://white.stanford.edu/teach/index.php/An_Introduction_to_Convolutional_Neural_Networks

12. Khan, R.U., Zhang, X., Kumar, R., Aboagye, E.O., Evaluating the performance of ResNet model based on image recognition, in: Proceedings of the 2018 International Conference on Computing and Artificial Intelligence, 2018, March, pp. 86–90.

13. Jiang, Y. and Li, C., Convolutional Neural Networks for Image-Based High-Throughput Plant Phenotyping: A Review. Plant Phenomics, 4152816, 1–22, 2020.

14. Labach, A., Salehinejad, H., Valaee, S., Survey of dropout methods for deep neural networks, arXiv preprint arXiv:1904.13310, 2019. https://arxiv.org/abs/1904.13310

15. Smith, A.G., Han, E., Petersen, J., Olsen, N.A.F., Giese, C., Athmann, M., Dresbøll, D.B., Thorup-Kristensen, K., Rootpainter: Deep learning segmentation of biological images with corrective annotation. BioRxiv, 2020. https://www.biorxiv.org/content/10.1101/2020.04.16.044461v2

16. Tan, J.W., Chang, S.W., Kareem, S.B.A., Yap, H.J., Yong, K.T., Deep learning for plant species classification using leaf vein morphometric. IEEE/ACM Trans. Comput. Biol. Bioinf., 17, 1, 82–90, 2018.

17. Alimboyong, C.R., Hernandez, A.A., Medina, R.P., Classification of plant seedling images using deep learning, in: TENCON 2018-2018 IEEE Region 10 Conference, 2018, October, IEEE, pp. 1839–1844.

18. Nkemelu, D.K., Omeiza, D., Lubalo, N., Deep convolutional neural network for plant seedlings classification, arXiv preprint arXiv:1811.08404, 2018. http://arxiv.org/abs/1811.08404