Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The AusSeabed Data Hub is being developed as a multi-component suite of services that are underpinned by a federated data storage model, with each connected data store being referred to as a “contributing hub”. At the time of writing, there are two contributing hubs under development, one at GA and one at CSIRO. Outlined below are the incoming data policies for the GA contributing hub.

...

  1. Data must be owned by GA (GA is the lead organisation of the survey) or GA must be a partner/collaborator in its acquisition, OR

  2. If GA does not held hold an interest in the acquisition or preparation of the data, the GA contributing hub will accept the data if it is not readily available through another publicly accessible channel. 

  3. Data can only be submitted by an agreed contributing data partner. The basis for becoming a contributing data partner can be found in the becoming a contributing partner document

  4. Data must be provided in the prescribed formats outlined in appendix 1

  5. The contributing organisation profile must be submitted at least 1month prior to submitting the data to allow for internal approval processes to be completed.

  6. Data can not be submitted without a contributing organisation profile, as this will provide GA with the right to store, distribute and process the data for its own outputs.  It is important to note:

    1. Authority to publish does not negate the need for GA to provide appropriate citation to the data.

    2. The contributing organisation is aware that the visualisation of the data may not be identical to what is supplied from them (e.g. the publishing pipeline standardizes visualisation output of any survey data, ensuring that hillshades and standard colour ramps are applied to the data).

  7. Data will only be prioritised for publication if it is supplied with appropriate metadata and conforms to the standard template that will be supplied to the contributing organisation.

  8. In order for data to be processed to a AusSeabed consistent output, data should be provided in one of the identified incoming formats outlined in the AusSeabed Guidelines ver 2.0 or should be easily converted to align with these formats. Specifically:

    1. All incoming data should include any auxiliary data required to meet the AusSeabed Guidelines. 

    2. All incoming data should include all relevant metadata outlined in the guidelines.

...

Standard definitions of data levels are necessary to inform how data has been processed and to ensure consistently and limit ambiguity when discussing, delivering or describing data. The following definitions are modelled on those defined by NASA for Earth Observations data products.

Include Page
GA Contributing Hub Data Submission RequirementsGA Contributing Hub Data Submission RequirementsAIDPP:Required Incoming Data Filetypes - By Data Processing Level
AIDPP:Required Incoming Data Filetypes - By Data Processing Level

...

Data Versions

Versioning of bathymetry data is classified as V0 and V1 and directly refers to the vertical datum that the data is published as, as well as the state of the data.

...

V0 (Version 0) bathymetry data is published at geodetic vertical datums (e.g. Mean Sea Level (MSL)) and is mainly refers to data that has been manually cleaned, and distributed for release.

Rough Notes

At the time of writing there is a considerable “backlog” of data that has been cleaned for delivery in response to a client request but has either:

  1. Been distributed through GA’s public facing tools as a bespoke deliverable (either for an individual survey, or as part of a compilation), or

  2. Has never been distributed at all. 

In both of these instances, the data represents a product of value, albeit limited in that the processing that has been undertaken to date has not been explicitly captured. It is with this data in mind that the GA Contributing Hub is defining a “version 0” for this backlog data. 

  • Important to note – “version 0” is not relevant to any new, incoming data, or data that has never been cleaned or is only partially cleaned previously. Version 0 represents a “quick win” and will be replaced at a later date. 

Version 1

V1 (Version 1) is classified as bathymetry data that may have been manually cleaned, surface cleaned, and exported to two vertical datums, Ellipsoid and MSL. This version will be used for all new surveys.

Rough Notes

Version 1 represents the first, truly consistent processing outcome for the GA Contributing Hub. Any data within the existing backlog that has not been produced as a V0 will be produced as a V1, via a prioritised publication schedule that takes into account both backlog and new incoming data. 

Version 1 will be:

  1. Created using an automated, reproducible processing pipeline