The Road to Autonomy: Unmanned Subsea Asset Inspection

In 2019, Swire Seabed successfully used a novel AUV/USV solution to complete a commercial pipeline inspection campaign, surveying 175km of pipe. This involved the use of a HUGIN AUV paired with the SEA-KIT Maxlimer USV. The 2019 survey campaign formed the final part of a developmental project involving Swire Seabed, EIVA, Kongsberg and SEA-KIT in which inspection work scopes were completed for Equinor. In this, emphasis was placed on the automation of survey processes, allowing remote command and control of operations. This work typifies the direction in which the industry is heading, with many survey companies now seeking to reduce requirements for manual intervention.

We are currently witnessing a rapid shift in the field of subsea survey and inspection, towards remote operations completed by unmanned vehicles. Automation is at the heart of this shift, and to reduce reliance on manual intervention and control, increasing levels of automation are required. Between 2018 and 2020, a commercial developmental project was undertaken by Swire Seabed, with Equinor as a client. The project aimed to implement automated processes in remote survey operations, to enable unmanned subsea pipeline inspections to be completed. The project successfully demonstrated how inspection campaigns can be undertaken with no personnel at sea, using an Autonomous Underwater Vehicle (AUV) operated from an Unmanned Surface Vehicle (USV), with missions controlled from a remote operations centre on land. Survey software running onboard the AUV facilitated in-mission data processing and the generation of vital alarms and quality control (QC) messages. These were relayed to shore, allowing mission alterations and retasking decisions to be made. The project resulted in the successful completion of the first commercial combined AUV/USV pipeline inspection campaign with over-the-horizon operations, and with pipeline extents surveyed up to 100km from shore. This unmanned system was effectively used to create a standard set of pipeline inspection deliverables, as defined by Equinor.

Survey Workflow

In 2018, Swire Seabed (who have now ceased operation), formed an agreement with Equinor to undertake a series of pipeline inspection work scopes using unmanned survey methods. The project required the combination and integration of existing technologies and methods in a novel manner. This also required new processes and procedures to be formulated, in addition to the automation of some of the standard, routine tasks usually completed as part of a survey workflow. Therefore, to meet obligations to Equinor, operational capacity had to be developed to allow unmanned subsea pipeline inspections to be completed using an AUV, launched and recovered from a USV. The Kongsberg Maritime (KM) HUGIN AUV was selected for the project and two additional onboard computers, supplied by EIVA, were installed on the HUGIN, one of which ran automated survey workflows (via Workflow Manager (WFM) software), and the other Deep Learning (DL) processes. A simple set of objectives was linked to the software implemented on these boards, including the implementation of functionality for real-time pipeline tracking and eventing, the creation of self-actuating data processing workflows, active QC, alarm generation, the automation of data analytics, and navigation aiding. Together, these functions form some of the fundamental actions required within the data ETL (Extract, Transform, Load) pipeline for autonomous subsea inspection operations. Furthermore, if successfully implemented onboard unmanned maritime systems (UMS), these processes could pave the way to removing existing client requirements for the use of large survey vessels, manned by extensive crews, for the completion of subsea pipeline inspection tasks.

The main activities completed during the project comprised of ongoing R&D works undertaken by development teams at Swire Seabed and EIVA, sea trials completed at KM’s base in Horton using the HUGIN, and two pilot projects in which inspection campaigns were completed for Equinor. These pilot projects, which ran in 2018 and 2019, involved the inspection of seven of Equinor’s pipelines located close to Bergen, Norway, with a combined length of over 350km. The project was drawn to a premature close early in 2020 due to the unexpected closure of Swire Seabed. Prior to this, however, the next phase of the project had commenced. This involved the design and build of a large USV, with facilities for launch, recovery and charging of an AUV, capacity for data download, and an onboard data management centre. Construction and subsequent use of such a craft was recognized as necessary to allow completion of extended missions, involving multiple AUV dives. The concept of UMS subsea pipeline inspection was effectively proven during the pilot projects; however, for this to be realized as a realistic commercial alternative to existing survey solutions, the operational limitations, which imposed constraints on the number of AUV dives per USV voyage, would need to be removed.

A flow diagram of the typical processes to be completed in a combined AUV/USV inspection campaign is detailed in Figure 1. During this project, it was not possible to implement all the processes listed in this diagram as only a single AUV dive was completed within each mission offshore. The operational capacity implemented during the pilot projects did allow processes one through to five to be completed. It was envisaged that utilization of the larger, planned USV would facilitate completion of the remaining processes.

Pilot Projects

The first pilot project was completed for Equinor in 2018. This tested the operational procedures that Swire had developed for running a full survey campaign remotely from a shore-based control centre. Overall, Pilot 1 was successful and the pipeline assets linking Equinor’s Troll field to their Kollsnes processing plant on the island of Sotra, close to Bergen, were inspected. The communication infrastructure at the heart of these remote operations was, and is, the enabling factor allowing such unmanned processes to occur. Figure 2 provides a rough indication of the communication lines utilized in the project, with satellite communications and Maritime Broadband Radio (MBR) forming the main communication links.

The SEA-KIT Maxlimer USV was planned for the project in Pilot 1, but due to a clash in timings it was not available. As a result, the project resorted to using a manned vessel (the Simrad Echo), which simulated the role of a USV. Operations were still completed in the same manner, with command and control situated onshore, but manual AUV launch and recovery was required. This was not the case in Pilot 2, where it was possible to pair the HUGIN with the SEA-KIT USV (Figure 3). During Pilot 2 missions, successful unmanned AUV launch and recovery was completed offshore. Data acquisition was controlled and monitored from the shore-based control centre, and four pipelines with a combined length of 175km were surveyed. Operations undertaken as part of Pilot 2 included a world-first for an over-the-horizon commercial pipeline inspection using UMS. No external assistance was provided to the operation of the USV and AUV; however, due to regulatory requirements in place in 2019, a guard vessel remained in the vicinity of the USV during missions.

Software Automation

EIVA developed software functionality during the project, which was installed on the two boards integrated into the HUGIN AUV. In pilot project operations, EIVA’s DL board was used primarily for pipe tracking using image data, whilst their WFM board enabled data processing to begin onboard the AUV, for alarms to be generated and transmitted to shore, and for data QC and analysis to be undertaken. Figure 4 shows a post-mission visualization generated, illustrating the Top of Pipe (TOP) position created in real time, using functionality installed on the DL board; image data corresponding to the on-screen tracking point is shown in the left of the figure. Additional pipe tracking methods based on Multibeam Echosounder (MBES) and laser data were developed during the project and tested in simulator dives following Pilot 2; however, it was not possible to implement these methods on the HUGIN prior to the project’s cessation.

The DL pipeline eventing methods developed by EIVA were implemented offline during the course of the project and tested using data obtained from both pilot projects. However, the longer-term goal was to migrate these processes to the DL board onboard the HUGIN, so that eventing could be carried out in near real time. The DL eventing methods that were developed did prove effective in identifying field joints and anodes (Figure 5), and in addition to providing event listings, functionality was developed to use detected features for navigation aiding. Post Pilot 2, EIVA refined a navigation aiding method based on anode identification, which involved comparison of the known and observed positions of detected anodes. The long-term aim of this, and other similar navigation aiding methods under development, is to generate outputs that can feed back into the HUGIN’s control system to form dynamic navigation aiding inputs.

It was possible to implement a range of software functions during the pilot projects, most of which involved processes developed for the WFM board. Self-actuating data processing workflows were tested during Pilot 2, and these allowed data processing, cleaning and QC tasks to commence onboard the AUV and then to continue onshore post-mission. In addition to this, the WFM board also ran processes that triggered alarms when specific conditions were present, such as poor visibility, inadequate data coverage, missing sensor inputs, and the pipeline not being detected. It was possible to transmit these alarms to the USV via a low-bandwidth acoustic link. The alarms were then transmitted back to shore, alerting operators of critical conditions so that a manual decision could be taken for the in-mission retasking of the AUV. During the pilot project, alarms were only used to inform operators; however, the longer-term plan is for the outputs of these alarms and QC processes to form inputs to automated AUV retasking routines.

A range of pipeline inspection-specific analytical processes were also developed during the project and run offline. However, EIVA’s aim is to slowly migrate these processes to the onboard computer running WFM. Despite these processes not being tested operationally onboard an AUV, simulator tests revealed that significant time and efficiency gains were possible by running the automated analytical processes developed, such as pipeline digitization, tagging, data classification and QC. For example, functionality was developed for the automated cleaning of MBES data points based on pipe diameter, once TOP was determined from pipe tracker data. The automation of data processing, cleaning, QC and analysis was recognized as essential to furthering the levels of automation possible in UMS inspection missions. Many of these tasks would normally be completed manually offline, yet they form vital inputs to AUV retasking decision-making processes. Automation of such functions, and the ability for these to be completed onboard an AUV prior to its recovery, could radically reduce the overall duration of inspection campaigns, especially if it were possible for the outputs of these tasks to form inputs to automated retasking routines.

The End of the Beginning

Despite this project having ended, in the words of a great man: “Now this is not the end, it is not even the beginning of the end, but it is, perhaps, the end of the beginning” (Winston Churchill, 1942). Work in this field is progressing rapidly, and the benefits of further progressing unmanned pipeline inspection solutions are obvious: efficiency gains, reduction in costs (vessel time and man hours), and minimization of environmental impacts. However, a significant distance is yet to be covered along this road to automation. While we are now able to complete unmanned operations with humans in the loop as remote decision makers, in the near future we aim for the automatic in-mission retasking of UMS, allowing larger work scopes to be completed. The ultimate goal is for the role of humans in this loop to diminish to a level where we are only required to oversee operations. Lessons learnt during this project have highlighted the importance of focusing on a number of key areas to achieve this goal, including hardware and software development, optimization of maritime communications, automation of data processing and analysis, navigation aiding, and automated AUV retasking.