Definition
Completeness relates to the degree the data collected is as intended and complies with minimum requirements as per ASB standards. The Completeness Assessment is a cursory examination of data acquired or submitted to detect any obvious data gaps. This applies to the presence of required attributes, metadata, and acquisition reports detailing survey scope and any information relevant to the collection and supply of the dataset. This check consists of two elements: Commission (excess or duplication) and omission (missing files/data).
Checks
Missing files, as apart from the subsequent Logical Consistency, are noted on a cursory check of supplied data (verify files are present).
Date of data reception is recorded.
Is the data published and available elsewhere?
Survey priority in the context of the Publication Schedule has been assigned.
Metadata including additional ancillary data is supplied as per outline of data submission. Storage on the Clearing House has been allocated.
Pass Requirements
External/internal dataset review. Data is identified as from a ‘Registered Contributing Partner’ or otherwise noted. Data submission retains a logical consistency to expectations (files appear consistent with initial data submission documents):
L0 data – have initial QAX QA/QC checks been completed?
...
L2 data – initial QAX report included with data?
...
a first-order measure of data integrity assessed upon receipt. Once received, ‘Data Packets’ are inspected and their contents examined to substantiate content (reasonable file sizes) and verify file formats. Expected file formats for either L0, L2 or L3 products should be consistent with Multibeam Guidelines (section 2.3.1.2 Table 5). This data is saved and archived as a directory index of data received.
Checks
The directory structure and contents of data packets are scanned, tallied and listed upon reception to provide first-order verification and identify anomalies.
Listed files are checked against expected/recommended Multibeam Guidelines (section 2.3.1.2 Table 5) for their respective process level (either L0, L2 or L3). File sizes are assessed by the user for legitimacy (e.g. > 0 kb)
Listed files are archived as a text file with the date of data reception recorded.
Pass Requirements
Files are of the appropriate format (e.g. HDCS compatible extensions) and are sufficiently substantial (e.g. > 0 kb).
Examples
| Expand | ||
|---|---|---|
| ||
Description Deakin University post-processed dataset. Handover dataset package includes three band GeoTIFF (or QAX QA/QC pass/fail check as per ASB standards). Also includes necessary metadata documentation for eCat record lineage and publication to the AusSeabed Portal. Test results What elements were there or not:
Pass Yes Explanation Passes are recorded in the ASB Survey checklist. Exceptions can be made for L3 datasets in the case where missing data is non-critical to publication (to be deliberated by the team and contributing hub). These are recorded in the checklist procedureThe ‘Falkor’ dataset is received and stored within the Clearing House by AusSeabed. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb). Test results Summary of directory scan: Count Name Total KB ----- ---- -------- 1 .asc 16326 2591 .all 331416250 3787 .gsf 374252094 Pass Yes Explanation Falkor L2 data contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected formats and tallies. |
| Expand | ||
|---|---|---|
| ||
Description Australian Antarctic datasets are newly implemented for processing/publication. RSV Nuyina maiden voyage raw dataset was processed and the procedure documented. Test results What elements were there or not:
Pass Under Revision Explanation Passes are recorded in the ASB Survey checklist. L0 raw datasets must meet minimum metadata/file requirements as per ASB standardsThe ‘Kerguelen Plateau’ dataset is received and stored by AusSeabed. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb). Test results Summary of directory scan: Count Name Total KB ----- ---- -------- 5 .txt 7 1 .asc 563684 60 .xml 35 4 .csar 2168 219 .prj 0 529 .tif 3093997 2 .pdf 2531 5 .tiff 2137785 218 .dbf 16812 218 .shp 170527 218 .shx 6254 Pass Yes Explanation Kerguelen Plateau (20220004C) L2 processed data packet contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected tallies and formats. Directory contents largely resemble typical file formats and tallies. |
| Expand | ||
|---|---|---|
| ||
Description Australian Hydrographic Office HIPP datasets are a special case where specific datums/metadata lineage requirements are used. The ‘Austrea 01’ dataset is at L2 stage. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb). Test results What elements were there or not:
Pass Under Revision Explanation Passes are recorded in the ASB Survey checklist. L0 raw datasets must meet minimum metadata/file requirements as per ASB standards. Summary of directory scan: Count Name Total KB ----- ---- -------- 1 .jpg 866 2 .hips 8292 10 .txt 164 1 .old 5275 1 .hips_rxl 1 1 .hpf 0 1 .tif 5284 33 .csar 9048 40 .xml 1146 26 .hndf 2152 27 .lsf 348 2 .zdf 16 58 .tid 148828 2 .csv 1 23 .bak 5208 26 .log 3406 1 .gsf 18606 1 .hvf 5 Pass Yes Explanation Austrea 01 (20220004C) L2 processed data packet contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected tallies and formats. |