Nil Values Data Quality

SWE Common Data Model OGC 08-094r1

6.4.3 Full lineage and traceability

Full lineage and traceability is not in the scope of this specification. It is fully addressed by the OGC ® Sensor Model Language Standard, which allows robust definition of measurement chains, with detailed information about the processing that takes place at each stage of the chain. This means that complex lineage guarantying full traceability can be recorded in a SensorML process chain, separately from the data itself. Datasets can be associated to lineage information described using the Sensor Model Language by using a metadata wrapper such as the “Observation” object defined in the OGC ® Observations and Measurements Standard OM. In this standard, the “procedure” property of the “Observation” class allows attaching detailed information about the measurement procedure, that is to say a description of how the data was obtained i.e. lineage, to the data itself.

6.5 Data Structure

Data structure defines how individual pieces of data are grouped, ordered, repeated and interleaved to form a complete data stream. The SWE Common models are based on data structures commonly accepted in computer science and formalized in ISO 11404. Classical aggregate datatypes are defined below: - Record: consists of a list of fields, each of them being keyed by a field identifier and defining its own type that can be any scalar or aggregate structure. - Array: consists of many elements of the same type, usually indexed by an integer. The element type can be any data structure including scalars and aggregates. The array size constitutes the upper bound of the index. - Choice: consists of a list of alternatives, each of them being keyed by a tag value and having its own type. Only values for one alternative at a time are actually present in the data stream described by such a structure. Requirement http:www.opengis.netspecSWE2.0reqcoreaggregates-model-valid Req 12. Aggregate data structures shall be implemented in a way that is consistent with definitions of ISO 11404. This standard also defines the concept of “data component” as any part of the structure of a dataset, aggregate or not. It is thus the superset of all the aggregate structures described above and of all scalar elements implementing the representations described in clause 6.2. Copyright © 2011 Open Geospatial Consortium 17 OGC 08-094r1 SWE Common Data Model Examples A dataset representing a time series of observations acquired by a mobile sensor can be encoded with various methods depending on the requirements: - XML encoding can be used when data needs to be easily styled to other markup formats such as HTML or when precise error localization in the case of an error in the stream is needed. - ASCII encoding can be used to achieve a good compromise between readability and size efficiency. - Binary encoding can be used eventually with embedded compression when pure performance i.e. size but also reading and writing throughput is the main concern. A data component can be both a data descriptor and a data container: - A data component used as a data descriptor defines the structure, representation, semantics, quality, and other metadata of a data set but does not include the actual data values. - A data component used as a data container equally defines the dataset but also includes the actual property values.

6.6 Data Encoding

A key concept of the SWE Common Data Model is the ability to separate data values themselves from the description of the data structure, semantics and representation. This allows verbose metadata to be used in order to robustly define the content and meaning of a dataset while still being able to package the data values in very efficient manners. Data encoding methods define how the data is packed as blocks that can efficiently be transferred or stored using various protocols and formats. Different methods allow encoding the data as XML, text CSV like, binary and even compressed or encrypted formats in a way that is agnostic to a particular structure. This allows any of the encodings methods to be selected and used based on a particular requirement, such as performance, re-use of tools, alignment with existing standards and so on. Requirement http:www.opengis.netspecSWE2.0reqcoreencoding-method-valid Req 13. All encoding methods shall be applicable to any arbitrarily complex data structures as long as they are made of the data components described in clause 6.5. 18 Copyright © 2011 Open Geospatial Consortium