Completeness

Definition

Completeness is a first-order measure of data integrity assessed upon receipt. Once received, ‘Data Packets’ are inspected and their contents examined to substantiate content (reasonable file sizes) and verify file formats. Expected file formats for either L0, L2 or L3 products should be consistent with Multibeam Guidelines (section 2.3.1.2 Table 5). This data is saved and archived as a directory index of data received.

Checks

The directory structure and contents of data packets are scanned, tallied and listed upon reception to provide first-order verification and identify anomalies.
Listed files are checked against expected/recommended Multibeam Guidelines (section 2.3.1.2 Table 5) for their respective process level (either L0, L2 or L3). File sizes are assessed by the user for legitimacy (e.g. > 0 kb)
Listed files are archived as a text file with the date of data reception recorded.

Pass Requirements

Files are of the appropriate format (e.g. HDCS compatible extensions) and are sufficiently substantial (e.g. > 0 kb).

Examples

1. Pass example

Description

The ‘Falkor’ dataset is received and stored within the Clearing House by AusSeabed. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb).

Test results

Summary of directory scan:

Count Name Total KB

----- ---- --------

1 .asc 16326

2591 .all 331416250

3787 .gsf 374252094

Pass

Yes

Explanation

Falkor L2 data contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected formats and tallies.

2. Example 2

Description

The ‘Kerguelen Plateau’ dataset is received and stored by AusSeabed. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb).

Test results Summary of directory scan:

Count Name Total KB

----- ---- --------

5 .txt 7

1 .asc 563684

60 .xml 35

4 .csar 2168

219 .prj 0

529 .tif 3093997

2 .pdf 2531

5 .tiff 2137785

218 .dbf 16812

218 .shp 170527

218 .shx 6254

Pass

Yes

Explanation

Kerguelen Plateau (20220004C) L2 processed data packet contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected tallies and formats. Directory contents largely resemble typical file formats and tallies.

3. Example 3

Description

The ‘Austrea 01’ dataset is at L2 stage. A script is applied to the ‘Data Packet’ which generates a summary of the directory contents. The operator examines the summary as required to verify that data is present in expected formats and is substantial (> 0 kb).

Test results

Summary of directory scan:

Count Name Total KB

----- ---- --------

1 .jpg 866

2 .hips 8292

10 .txt 164

1 .old 5275

1 .hips_rxl 1

1 .hpf 0

1 .tif 5284

33 .csar 9048

40 .xml 1146

26 .hndf 2152

27 .lsf 348

2 .zdf 16

58 .tid 148828

2 .csv 1

23 .bak 5208

26 .log 3406

1 .gsf 18606

1 .hvf 5

Pass

Yes

Explanation

Austrea 01 (20220004C) L2 processed data packet contents are consistent with Multibeam Guidelines Data Formats (section 2.3.1.2) and expected tallies and formats.