Hydrographic data science

We are seeing the introduction and application of automated technologies everywhere that supplant the common and accepted ways of doing jobs, writes Paul Cooper. Indeed, the pages of Hydro International are often dominated by examples of such applications, and most of us can think of concrete situations or comparisons between ‘then’ and ‘now’. Modernization and new thinking are dramatically changing the way that hydrography is collected, processed and made available to users. As the focus is shifting from data collection to data analysis, hydrographic data scientists are becoming increasingly important.

When I reported on board at the Naval Oceanographic Office in 1971, we had just taken delivery of two amazing new hydrographic surveying ships: USNS Harkness and USNS Chauvenet. They were 390ft long, 54ft beam and 31ft draft. Onboard, they carried about 200 personnel, including the ship’s merchant crew, US Navy, NAVOCEANO Surveyors and Cartographers, helicopter and crew, Navigational Aids Support Unit, and a US Marine Corps Topographic Platoon. The ship maintained and operated four 36ft Hydrographic Survey Launches, two Landing Craft, one helicopter, vehicles, and manned three shore-based radio positioning sites providing electronic positioning for the ships. They carried the latest survey equipment suites including echo sounders, gravity meters, magnetometers, and among the first survey automation computers. There was a large format camera and a colour printing press to compile, print and distribute charts to the US Navy fleet while at sea. We had wet and dry labs for processing oceanography observations and cores taken on board. They had more hydrographic capabilities afloat than most coastal states in their home offices.

These ships were the sea surveyor’s equivalent of Noah’s Ark in their manning and equipping. They operated for about 20 years as the US Navy’s main source of hydrographic survey data. In that period using the best technology available (pre MBES) both ships provided relatively small amounts, less than 1 million soundings, to our military mapping agency for charting purposes.

Fast forward to 2015 and in a demonstration survey at an IHO conference in the Caribbean using a two metre long radio controlled boat outfitted with multibeam echosounder, GPS, and one hydrographer driving the boat from shore collected and automatically processed about 200,000 soundings in several hours which were then decimated, and transmitted to a cartographer updating the relevant chart and returned the updated chart to the authority the next day.

The rush towards innovative technologies is on

Modernization and new thinking are being applied to dramatically change the way hydrography is collected, processed and served out to users. With new techniques and equipment in constant development, it is possible to see hydrographers who are office bound and do not go to sea at all. Here are three examples that are currently in use that point toward a significantly reduced hydrographic corps at sea.

Unmanned technology which has been proven with remotely operated vehicles (ROV) and Autonomous Underwater Vehicles (AUV) to collect seafloor and oceanographic information has been proven. Uncrewed surface platforms (USPs) have now been proven for hydrography. On-board batch data processing of multibeam data has been proven to provide near real-time hydrographic data reduced for sound velocity corrections, tide, draft, and other motion. One only needs to see the many articles in Hydro International dealing with unmanned technologies. Examples in transportation, medicine, fast food show robots and innovation will erase recognizable professions as we know them.

Satellite-derived bathymetry (SDB) has become increasingly useful and is now recognized as a good reconnaissance tool for nearshore bathymetry. It is can be utilized in areas where surveying assets have not been deployed for a long period or in areas where they will likely never be deployed. The refinement of SDB processing by applying a bathymetry algorithm to Landsat 7 and 8 as well as Worldview 2 imagery is strengthening the case for SDB and eventually leading to use on charts.

Slightly more controversial is crowdsourced bathymetry (CSB). CSB is data collection by volunteers on ships and boats of opportunity. CSB is gaining recognition and popularity as a source of data that strongly supplements the work of traditional hydrography. One major benefit of CSB is the data are gathered by vessels that are already transiting in the area of interest. The IHO has established a Working Group to develop a CSB policy and guidelines which will allow Hydrographic Offices to utilize CSB as they see fit.

We must suspend old thinking, prepare for and embrace new while maintaining quality services and products that we have provided over the years. The time for smart charts developed from data-centric environments, and user-designed navigation tools that are updated from the cloud in real-time is coming – with and without the traditional hydrographer and cartographer.

Open data and big data

More and more data is becoming available at all levels, from new sources, and all disciplines. Making data available and useful to all starts with the principles described by the G8 Open Data Charter of 2013 and the US Executive Order which say that open, machine-readable data are the new default. The United States Open Data Policy requires US government agencies to collect or create information that supports downstream information processing and dissemination activities. In the Age of Exploration raw materials were the drivers of advancements in the world. Now the Age of Data data is what will drive success by making more available from all commercial and government sources.

Rising alongside open data and big data is the increasing trend to recognize data science as a way to exploit data. The data scientist will exploit data through the access, integration, predictive analytics, and assimilation of multi-discipline datasets. Challenges are dealing with data volume, data variety, pace of data collection, data variability, data fidelity, data complexity and understanding data value and pricing to negotiate data purchases needed for adding value to products, maintaining records and cooperating with other institution’s management activities required to maintain data long-term for archiving - in other words data curation.

Data science

Data science is the extraction of knowledge from large volumes of data that are structured or unstructured, known as knowledge discovery and data mining. Every aspect of our lives, from life-saving disease treatments, to national security, safety of navigation, to economic stability and even the convenience of selecting a restaurant, can be improved by creating better data analytics through data science.

The hydrographic data scientist

A data scientist with a solid foundation, typically in computer science and applications, modelling, statistics, analytics and math represents an evolution from the strict data analyst role. A data scientist is somebody who is curious, who can concentrate on data and identify trends, an individual who really wants to bring change to an organization. With the advances in sea surveying technologies such as autonomous surface and subsurface vessels, satellite-derived bathymetry and crowdsourcing, the days of at sea hydrographers are going to be limited more and more.

Rather than deployment onboard, our professionals will be office-bound, staring at data and putting new uses and applications into the hands of users and decision makers. Hydrographic data scientists (HDS) will use their domain experience, together with computer science, and mathematics/statistics education to create data. While important, there will be less emphasis on traditional hydrographic data collection skills and experience. Traditionally collected data will often be a small segment of the dataset. Whereas a traditional data analyst may look only at data from a single source - an HO dataset, for example – a HDS will most likely explore and examine data from multiple disparate sources. The HDS will sift through incoming data from current survey collection operations, on several different levels depending on the immediate mission. His role on the immediate requirement will be discovering previously hidden trends and provide insight, which in turn can provide a solution to a technical problem, provide a competitive advantage or address a pressing business problem. Access to old data and relevant but not directly related data such as Marine Information Overlays will inform his work. An HDS does not simply collect and report on data, but also looks at it from many angles, determines what it means, then recommends ways to apply the data. A HDS explores, asking questions, doing ‘what if’ analysis, questioning existing assumptions and processes and will then communicate informed conclusions and recommendations across an organization’s leadership structure.

Summary

The predictable growth and integration of new methods and thinking is at once exciting and cause for concern in our community. In many cases, one can see there is no longer a need for a complete crew of hydrographers at sea. With the challenge of reduced resources, the likelihood of non-traditional methods for collecting data will grow to support the ever-increasing needs and uses for hydrographic/bathymetric data for purposes other than safety of navigation. The development of the hydrographic data scientist will meet that challenge, but who is going to meet the challenges for education, training, implementation, standardization and creation of official products. That will fall squarely on the shoulders of academia, industry, government and international organizations. That quartet must work together, be interoperable, be inter communicative and be interchanging.