PARTNERING WITH MICROSOFT AI FOR EARTH TEAM

The Microsoft AI for Earth team was a key enabler and influencer for project’s success, through timely access for technical support and resolution of AI platform queries.

Technology Used to Resolve the Issue

• For a reliable cloud solution with machine learning capabilities, Microsoft Azure Data Science Virtual Machine (DSVM) was chosen.
• For the purpose of extracting frames from the videos and tag them, Microsoft Visual Object Tagging Tool (VOTT) came in handy.
• The final object detection algorithm chosen was the YOLO V3 Video detection algorithm.

Our Contribution to Augment the Process

• The first challenge was to process the videos and tag the fish. The heavy manual work involved in this was automated by leveraging the MS VOTT tool.
• The tagged frames were then used to train a model using CNTK and Faster RCNN. This model was then tested against more frames extracted from the videos. While this solution was good, it lacked speed and real-time video detection capabilities.
• As an enhancement to the solution, we moved to video object detection using YOLO V3, which provides a faster solution with real-time capabilities.

QUANTIFIABLE RESULTS FROM THE INTERVENTION

This web-based AI solution would save the client, valuable hours of expert biologist time and infrastructure costs spent in manually reviewing the videos. As part of a planned upgrade, an enhanced version of the solution has been provided to the customer, which predicts to deliver substantial cost savings to the tune of 80%.

OUR WORK MENTIONED IN SCIENCE JOURNALS

• Elsevier: MSR-YOLO: Method to Enhance Fish Detection and Tracking in Fish Farms
• Georgia State University Scholarworks: Technology Enabled Social Responsibility Projects and an Empirical Test of CSR’s Impact on Firm Performance