Definition
Completeness relates to the supply of the dataset.
Is it the full dataset, are there missing elements or files
Core requirements…
...
xxx
...
xxx
...
xxx
...
is a first-order measure of data integrity assessed upon receipt. Once received, ‘Data Packets’ are inspected and their contents examined to substantiate content (reasonable file sizes) and verify file formats. Expected file formats for either L0, L2 or L3 products should be consistent with Multibeam Guidelines (section 2.3.1.2 Table 5). This data is saved and archived as a directory index of data received.
Checks
The directory structure and contents of data packets are scanned, tallied and listed upon reception to provide first-order verification and identify anomalies.
Listed files are checked against expected/recommended Multibeam Guidelines (section 2.3.1.2 Table 5) for their respective process level (either L0, L2 or L3). File sizes are assessed by the user for legitimacy (e.g. > 0 kb)
Listed files are archived as a text file with the date of data reception recorded.
Pass Requirements
Files are of the appropriate format (e.g. HDCS compatible extensions) and are sufficiently substantial (e.g. > 0 kb).
Examples
| Expand | ||
|---|---|---|
| ||
Description The ‘Falkor’ dataset is received and stored within the Clearing House by AusSeabed. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb). Test results Summary of directory scan: Count Name Total KB ----- ---- -------- 1 .asc 16326 2591 .all 331416250 3787 .gsf 374252094 Pass Yes Explanation Falkor L2 data contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected formats and tallies. |
| Expand | ||
|---|---|---|
| ||
Description The ‘Kerguelen Plateau’ dataset is received and stored by AusSeabed. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb). Test results Summary of directory scan: Count Name Total KB ----- ---- -------- 5 .txt 7 1 .asc 563684 60 .xml 35 4 .csar 2168 219 .prj 0 529 .tif 3093997 2 .pdf 2531 5 .tiff 2137785 218 .dbf 16812 218 .shp 170527 218 .shx 6254 Pass Yes Explanation Kerguelen Plateau (20220004C) L2 processed data packet contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected tallies and formats. Directory contents largely resemble typical file formats and tallies. |
| Expand | ||
|---|---|---|
| ||
Description The ‘Austrea 01’ dataset is at L2 stage. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb). Test results Summary of directory scan: Count Name Total KB ----- ---- -------- 1 .jpg 866 2 .hips 8292 10 .txt 164 1 .old 5275 1 .hips_rxl 1 1 .hpf 0 1 .tif 5284 33 .csar 9048 40 .xml 1146 26 .hndf 2152 27 .lsf 348 2 .zdf 16 58 .tid 148828 2 .csv 1 23 .bak 5208 26 .log 3406 1 .gsf 18606 1 .hvf 5 Pass Yes Explanation Austrea 01 (20220004C) L2 processed data packet contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected tallies and formats. |