84 lines
3.2 KiB
Markdown
84 lines
3.2 KiB
Markdown
# Flower State Classification for Watering System
|
|
|
|
This project aims to develop a machine learning model to detect plants
|
|
and classify them as water-stressed or not.
|
|
|
|
## Overview of the Approach
|
|
|
|
The machine learning pipeline is divided into two stages: detection
|
|
and classification. Images are first passed to the object detection
|
|
model which draws bounding boxes where it thinks plants are. These
|
|
bounding boxes (image coordinates) allow us to get a cutout of the
|
|
image and pass said cutout to the classifier.
|
|
|
|
The classifier takes the cutout and predicts one of two classes:
|
|
*healthy* or *stressed*. The confidence (probability) of the
|
|
classifier for either class is seen as a proxy for the degree of
|
|
stress. A plant which has been classified as 70% stressed, for
|
|
example, will get a rating of 7/10 regarding its stress level, where
|
|
10 is the most severe stress. Conversely, a 60% healthy plant is put
|
|
into the 4/10 stress category.
|
|
|
|
### Plant Detection
|
|
|
|
To detect plants, we chose the popular single stage detector
|
|
YOLO. There are multiple versions available, with the most recent one
|
|
YOLOv8 available since 2023. At the start of the project YOLOv8 did
|
|
not exist yet, which is why the pipeline is built on YOLOv7.
|
|
|
|
### Classification
|
|
|
|
The classifier is a ResNet50 model which has been chosen again due to
|
|
its popularity and proven track record. It has been trained on a
|
|
dataset of only 452 healthy and 452 stressed plants, but works
|
|
surprisingly well nevertheless.
|
|
|
|
## Installing the Dependencies
|
|
|
|
In order to run the model(s), a couple of dependencies have to be
|
|
installed. These are listed in the `classification/requirements.txt`
|
|
file. The code is tested under python versions 3.6 and 3.7 and might
|
|
not work for other versions.
|
|
|
|
```bash
|
|
python3.7 -m pip install -r classification/requirements.txt
|
|
```
|
|
|
|
Afterwards, it is necessary to either add the `classification`
|
|
directory to the `PYTHON_PATH` environment variable or to install the
|
|
package locally.
|
|
|
|
```bash
|
|
# From the classification folder
|
|
python3.7 -m pip install -e .
|
|
```
|
|
|
|
## Running the Model(s)
|
|
|
|
The models can be run by calling the function `detect()` in
|
|
`evaluation/detect.py`. This will return a pandas dataframe with the
|
|
coordinates of the bounding boxes and their confidence values. This
|
|
dataframe together with the original image can be passed to the
|
|
`draw_boxes()` function in `utils/manipulations.py`. A new image with
|
|
the bounding boxes drawn on the original image can then be either
|
|
shown to the user via OpenCV or saved to disk.
|
|
|
|
## Model Evaluation
|
|
|
|
The `evaluation` directory contains multiple Jupyter notebooks which
|
|
evaluate the models separately as well as together. These notebooks
|
|
contain explanations of the code and what the results are in quasi
|
|
[literate
|
|
programming](https://en.wikipedia.org/wiki/Literate_programming)
|
|
style.
|
|
|
|
## Deploying Model to Jetson Nano
|
|
|
|
The folder `jetson-deployment` contains a script to periodically run
|
|
detections and expose the results over a simple API. The python
|
|
bindings for OpenCV are somehow very difficult to install, primarily
|
|
because NVidia does not provide regular updates to the Python and
|
|
Ubuntu distributions. Therefore, Image capture is not implemented in
|
|
Python but in C++. The binary has to be compiled with the provided
|
|
`Makefile`.
|