Every player in the survey industry has been affected by the pandemic in unique ways. What each had in common though, was an almost complete halt of face-to-face interaction with competitors and collaborators alike. Seafloor Systems, based in Northern California, took this opportunity to focus on its product development and advance the autonomous capabilities of its existing remote survey platforms.
Automation for Hydrographers
Seafloor Systems has been designing and building Uncrewed Surface Vessels (USVs) for over a decade. From drawing board to field testing, the team of twenty-five employees pumps out remote platforms for a variety of clients and disciplines within the land and hydrographic survey sectors. Its star products, robot boats, make it possible to send pro-grade instrumentation and sensors into areas that a crewed vehicle would find challenging (streams, dredge ponds, shorelines, etc).
The employees at Seafloor Systems are no strangers to pioneering technologies for remote, hydrographic survey applications. When Seafloor launched its AutoNav control system for USVs in 2015, users were given the ability to pre-program remote vessels to follow track lines in the water and carry out hands-free missions. The AutoNav, which is still a staple product, reduces overlap in sonar beam swathes and ensures clean, consistent data collection. Even so, the AutoNav has limitations when obstacles stray into the pre-planned route without warning.
However, the biggest obstacle in the way of recent efforts to address the AutoNav’s limitations was the onslaught of the worldwide pandemic. Fractures in the global supply chain and mandatory stay-at-home orders made it difficult for businesses to operate, let alone invest in new projects. Despite this setback, the nimble Seafloor crew set a goal to pioneer a package that could be integrated into USVs to autonomously navigate around obstacles during survey missions on water. In other words, they wanted their robot boats to see, identify, and react to surface threats in real time.
A Vision for Future Uncrewed Vessels
It is no easy task building a computer vision navigation system for uncrewed vessels. Not only would it have to identify obstacles in the path of the survey mission, but also react to them in real time and adjust the steering propellers to avoid collisions. Luckily, there is a company out there that helps clients do just that. AlwaysAi, a leading provider of flexible ways to build and deploy computer vision applications to a wide variety of devices, offered up its platform for the Seafloor team to build from.
With a strong artificial intelligence partner secured, Jenna Opsahl and Marcos Barrera, Seafloor’s Computer Vision and Robotics specialists, began the daunting task of formulating a computer vision model specific to maritime environments. (There are several land-based models out there, but limited options for the niche maritime survey industry). Opsahl collected and sifted through video clips frame by frame that contained objects commonly found in bodies of water. Boats, buoys, ducks, plants — anything that was above the surface that might pose an obstacle for unmanned vessels — were selected with bounding boxes and labelled so the artificial intelligence model could learn and adapt.
Barrera was tasked with developing a perception and control infrastructure that would work in tandem with the computer vision model. For this, he utilized Amazon Web Services (AWS) IoT Greengrass, which enables programmers to remotely manage and operate software on devices in the field. The obstacle avoidance package for USVs began to take shape.
Testing the Waters – CV Success
Fast forward a few months and hundreds of hours of annotating through video footage, and the team was on the cusp of testing a prototype. The computer vision model had seen leaps in object recognition abilities, and COVID restrictions were simultaneously being lifted in California. The Seafloor crew met in San Diego (the first time in person in almost a year) to test the obstacle recognition and avoidance system on an EchoBoat-160 USV.
Initial testing garnered promising feedback. The computer vision model, now integrated into the EchoBoat-160’s AutoNav, picked up on several objects in the water and indicated its confidence (in a percentage) that it was, in fact, that object. For example, when the vessel stumbled upon an inflatable inner tube, the model determined it was 94.01% certain that it was a “raft”.
Now, it was time to see if the vessel could identify objects along a plotted survey route and react accordingly. Seafloor’s Ocean Engineer, JT Meyers, volunteered to cruise out on his paddle board to play the role of an unsuspecting bystander. A navigation route was programmed into the vessel’s AutoNav from the shore, and Meyers positioned himself between two of the wayfinding points to create an obstacle. To the team’s surprise, the computer vision system picked up the paddleboarder with no issue, and quickly altered the vessel’s path to avoid a collision. Computer vision success.
Machine Learning for the Future
With a successful obstacle avoidance test under their belts, the Research and Development Team at Seafloor could now take a deep breath. Months of tinkering with machine learning, and limited face-to-face collaboration stemming from the global pandemic were turned into the triumph they were all hoping for. This technology is slated to be introduced into various Seafloor remote survey platforms of the future to advance the autonomous capabilities of unmanned vessels. For now, the team says their work is not done. Further training of computer vision models will take place to account for different environments, objects, and scenarios. Innovation can arise in some of the most turbulent times, and obstacle avoidance is just one of the advancements driving the hydrographic survey industry forward.
