diff --git a/README.md b/README.md new file mode 100644 index 0000000..ebe7a89 --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +# Flower State Classification for Watering System + +This project aims to develop a machine learning model to detect plants +and classify them as water-stressed or not. + +## Overview of the Approach + +The machine learning pipeline is divided into two stages: detection +and classification. Images are first passed to the object detection +model which draws bounding boxes where it thinks plants are. These +bounding boxes (image coordinates) allow us to get a cutout of the +image and pass said cutout to the classifier. + +The classifier takes the cutout and predicts one of two classes: +*healthy* or *stressed*. The confidence (probability) of the +classifier for either class is seen as a proxy for the degree of +stress. A plant which has been classified as 70% stressed, for +example, will get a rating of 7/10 regarding its stress level, where +10 is the most severe stress. Conversely, a 60% healthy plant is put +into the 4/10 stress category. + +### Plant Detection + +To detect plants, we chose the popular single stage detector +YOLO. There are multiple versions available, with the most recent one +YOLOv8 available since 2023. At the start of the project YOLOv8 did +not exist yet, which is why the pipeline is built on YOLOv7. + +### Classification + +The classifier is a ResNet50 model which has been chosen again due to +its popularity and proven track record. It has been trained on a +dataset of only 452 healthy and 452 stressed plants, but works +surprisingly well nevertheless. + +## Installing the Dependencies + +In order to run the model(s), a couple of dependencies have to be +installed. These are listed in the `classification/requirements.txt` +file. The code is tested under python versions 3.6 and 3.7 and might +not work for other versions. + +```bash +python3.7 -m pip install -r classification/requirements.txt +``` + +Afterwards, it is necessary to either add the `classification` +directory to the `PYTHON_PATH` environment variable or to install the +package locally. + +```bash +# From the classification folder +python3.7 -m pip install -e . +``` + +## Running the Model(s) + +The models can be run by calling the function `detect()` in +`evaluation/detect.py`. This will return a pandas dataframe with the +coordinates of the bounding boxes and their confidence values. This +dataframe together with the original image can be passed to the +`draw_boxes()` function in `utils/manipulations.py`. A new image with +the bounding boxes drawn on the original image can then be either +shown to the user via OpenCV or saved to disk. + +## Model Evaluation + +The `evaluation` directory contains multiple Jupyter notebooks which +evaluate the models separately as well as together. These notebooks +contain explanations of the code and what the results are in quasi +[literate +programming](https://en.wikipedia.org/wiki/Literate_programming) +style. + +## Deploying Model to Jetson Nano + +The folder `jetson-deployment` contains a script to periodically run +detections and expose the results over a simple API. The python +bindings for OpenCV are somehow very difficult to install, primarily +because NVidia does not provide regular updates to the Python and +Ubuntu distributions. Therefore, Image capture is not implemented in +Python but in C++. The binary has to be compiled with the provided +`Makefile`.