Finding Flowers Fast: Bee Foraging Maps Update

For the last year I have been working on automatically creating maps of foraging resources for honeybees. The goal of this project is to create a system that quickly and easily maps a large area and identifies the kind and quantity of resources available to bees. By automatically detecting available resources this system will help other researchers reduce the amount of field work and time required to get an accurate overview of the environment the bees are foraging in. Last summer I wrote about my progress in automatically identifying flower species. Since then there have been a few more updates. First, we have two new members of the bee lab working on this project! I’m excited to work with Annalise and Alasdair this semester. 

In the last few weeks I have worked on combining the two kinds of training data we have. The training data for this project is a set of images and a list of the species that appear in each image. This data is used as an input for the classification algorithm to teach it what each species looks like. The first kind of data is transect data, which is made up of images and data taken along 50 meter transects in the field station. This data is relatively easy to collect because taking data along a single transect yields 50 data points that can be turned into training data with minimal effort. Transects are thus a great way to get a lot of data quickly. However, the drawback is that the training images are not targeted on one specific plant and may contain other plants, paths, or other features. This means that the algorithm may have a harder time distinguishing between nearby features because the area around a plant is included in the training images.

A typical transect training image, containing a plant and the surrounding area.

A typical research area training image, containing only the flowering plant of interest.

The second type of training data is “research area data.” This is data that was collected during the pollinator visitation experiment, which looked at pollinator visitation to several different flowering plants in the Spring. It is composed of training images containing only the flowering plant of interest and corresponding species identifications. These images have to be manually found and cropped from a larger map, so creating this data set is much more time intensive. Because of this there are far fewer data points of this kind available. By combining these two data sets I hope to achieve a balance of high quality research area data and high quantity transect area data so that each machine learning algorithm can successfully be trained.

After creating the combined training set I tested six machine learning algorithms: perceptron, SVM, SGD, KNN, random forest, and decision tree. Of these, the random forest, decision tree, and KNN algorithms showed considerably better performance than the others. Performance was quantified through three different metrics.

1.     The overall cross-validation score for the algorithm, based on a 10-fold cross validation

2.     The precision of the algorithm for the two species with the most training data available and “non-flower” classes.

3.     The recall of the algorithm for the two species with the most training data available and “non-flower” classes.

In this context, precision is defined as the number of true positives divided by the total number of positives (both true and false). Thus precision is a measure of how many of the classifications for a given species were actually that species. Recall is defined as the number of true positives over the number of true positives plus the number of false negatives (i.e. the number of classifications for a species over the number of images that actually contained that species). Recall thus measures the proportion of a given species that an algorithm can detect. Cross-validation is an overall, rather than species specific, measure, and is simply the number of correct classifications over the total number of classifications. This score is computed 10 times using different splits between training and testing data to ensure that overtraining on a particular data set is not an issue. In the ideal case of a perfect classification algorithm, cross-validation, precision, and recall would all be 1. Below is a table of the results achieved with each of these algorithms. 

From these results we can see where the algorithm is currently doing well and where it needs improvement. All three classification algorithms have relatively high precision and recall scores for the non-flower class. This indicates that they do a fairly good job of distinguishing between flowers and non-flowers. However, the scores for precision and recall within each flower species are still much lower than we would like. This indicates that the main focus of improvement should be classifying between species of flower.

In addition to the numerical results, I obtained graphical results that visualize some of this information.

The original image cropped from a larger map of the BFS

Classification using the KNN algorithm, which highlights several flowering plants, but does not correctly classify most of the species. Each color in the map represents a different species, with the darkest blue representing non-flowers.

Classification with the decision tree algorithm

Classification with a random forest algorithm

In each of these maps the majority of the image is classified as non-flower. It seems to be clearly and consistently distinguishing shrubs and vegetation from the background and classifying these as some kind of flower. This is exactly the result we want! However, the species actually present in the map are consistently misclassified. Based on these results I will be focusing on improving the classification algorithm between species as I continue my research. There are a few areas that we are currently looking to improve. First, we are researching new features that could be useful in distinguishing between species. Second, we are looking into using multiple algorithms so that each can be used only in areas where it works particularly well. For example, one algorithm could be used to classify flower vs. non-flower while another could classify each species within the flower group. Utilizing more tailored algorithms may be more successful than attempting to find a single, general algorithm that can perform well on everything. Finally, we are considering reversing the order of analysis and stitching so that a map is analyzed and classified based on the original, unstitched photos, and then these results are stitched together to create a large map. This reversal would help to eliminate the issue of artefacts in the images caused by the stitching process that may be affecting the classification results.

Training image with examples of artefacts that caused odd breaks and blurring in an image of flowers outlined in red. These artefacts may be reducing the accuracy of the classification algorithm by blurring the distinction between flowers and their surroundings.

All in all a lot of progress has been made towards automatically classifying species. We now have an algorithm that performs fairly well in distinguishing flowers from non-flowers, and several ideas on how to further improve that algorithm. I’m excited to see how the results change as I update the algorithm, and hopefully we will see some improved precision and recall scores soon!