OWS-8 Information Model for Moving Target Indicators and Moving Object Bookmarks (Engineering Report)

Open Geospatial Consortium

Date:2011-11-23

Reference number of this document: OGC 11-113r1

Category: Engineering Report

Editor: Ingo Simonis

OWS-8 Information Model for Moving Target Indicators and

Moving Object Bookmarks (Engineering Report)

Copyright © 2011 Open Geospatial Consortium.

To obtain additional rights of use, visit http://www.opengeospatial.org/legal/.

Warning

This document is not an OGC Standard. This document is an OGC Public

Engineering Report created as a deliverable in an OGC Interoperability Initiative and is not an official position of the OGC membership. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an OGC Standard. Further, any OGC Engineering Report should not be referenced as required or mandatory technology in procurements.

Document type: OpenGIS® Engineering Report Document subtype: NA

Document stage: Approved for public release Document language: English

ii Copyright © 2011 Open Geospatial Consortium

Preface

This engineering report describes the results of the VMTI/GMTI in OGC concept of operations study as performed as part of OWS 8 and the EC co-funded research project Emergency Support System - ESS” (contract number 217951).

This document is a deliverable for the OGC Web Services 8 (OWS-8) testbed activity. OWS testbeds are part of OGC's Interoperability Program, a global, hands-on and collaborative prototyping program designed to rapidly develop, test and deliver proven candidate standards or revisions to existing standards into OGC's Standards Program, where they are formalized for public release. In OGC's Interoperability Initiatives,

international teams of technology providers work together to solve specific geoprocessing interoperability problems posed by the Initiative's sponsoring organizations. OGC

Interoperability Initiatives include test beds, pilot projects, interoperability experiments and interoperability support services - all designed to encourage rapid development, testing, validation and adoption of OGC standards.

The OWS-8 sponsors are organizations seeking open standards for their interoperability requirements. After analyzing their requirements, the OGC Interoperability Team

recommend to the sponsors that the content of the OWS-8 initiative be organized around the following threads:

* Observation Fusion

* Geosynchronization (Gsync)

* Cross-Community Interoperability (CCI) * Aviation

More information about the OWS-8 testbed can be found at:

http://www.opengeospatial.org/standards/requests/74

OGC Document [11-139] “OWS-8 Summary Report” provides a summary of the OWS-8 testbed and is available for download:

Copyright © 2011 Open Geospatial Consortium iii License Agreement

Permission is hereby granted by the Open Geospatial Consortium, Inc. ("Licensor"), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.

If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR. THIS LICENSE IS A COPYRIGHT LICENSE ONLY, AND DOES NOT CONVEY ANY RIGHTS UNDER ANY PATENTS THAT MAY BE IN FORCE ANYWHERE IN THE WORLD.

THE INTELLECTUAL PROPERTY IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE DO NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE INTELLECTUAL PROPERTY WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION OF THE INTELLECTUAL PROPERTY WILL BE

UNINTERRUPTED OR ERROR FREE. ANY USE OF THE INTELLECTUAL PROPERTY SHALL BE MADE ENTIRELY AT THE USER’S OWN RISK. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ANY CONTRIBUTOR OF

INTELLECTUAL PROPERTY RIGHTS TO THE INTELLECTUAL PROPERTY BE LIABLE FOR ANY CLAIM, OR ANY DIRECT, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM ANY ALLEGED INFRINGEMENT OR ANY LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR UNDER ANY OTHER LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH THE IMPLEMENTATION, USE, COMMERCIALIZATION OR PERFORMANCE OF THIS INTELLECTUAL PROPERTY. This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party. Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications.

This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.

None of the Intellectual Property or underlying information or technology may be downloaded or otherwise exported or reexported in violation of U.S. export laws and regulations. In addition, you are responsible for complying with any local laws in your jurisdiction which may impact your right to import, export or use the Intellectual Property, and you represent that you have complied with any regulations or registration procedures required by applicable law to make this license enforceable.

iv Copyright © 2011 Open Geospatial Consortium

Contents

1  _{INTRODUCTION  1} 

1.1  _SCOPE  1 

1.2  _DOCUMENT CONTRIBUTOR CONTACT POINTS  1 

1.3  _{REVISION HISTORY  1} 

1.4  _FUTURE WORK  1 

1.5  _FORWARD  3 

2  _{TERMS AND DEFINITIONS  3} 

2.1  _MOTION IMAGERY  3 

3  _{CONVENTIONS  4} 

3.1  _{ABBREVIATED TERMS  4} 

3.2  _UML NOTATION  5 

4  _{VMTI, GMTI AND TRACKING IN THE CONTEXT OF OGC  5} 

4.1  _{MANAGEMENT SUMMARY  5} 

4.2  _INTRODUCTION  5 

5  _{USED STANDARDS  6} 

5.1  _{MISB & NATO  6} 

5.2  _OBSERVATION AND MEASUREMENT  6 

6  _{KLV TO XML  8} 

6.1  _KLV AND XML ENCODINGS  8 

6.2  _KLV TO XML CONVERSION  9 

6.2.1  CONVERSION GUIDELINES  9 

6.2.2  CONVERSION FRAMEWORK  10 

6.2.3  CONVERSION TABLE  10 

7  _{VMTI IN THE OGC CONCEPT OF OPERATIONS  10} 

7.1  _VMTI STANDARDS  11 

7.1.1  MISB STD 0601.4  12 

7.1.2  MISB RP 0903.2  13 

7.1.3  STANAG 4609V3  15 

7.1.4  OTHER RELEVANT STANDARDS  15 

7.2  _{VMTI TO UML/XML MAPPINGS  16} 

7.3  _{OGC VMTI INFORMATION MODEL  17} 

7.3.1  FEATURE OF INTEREST  17 

7.3.2  OBSERVATION SPECIALIZATIONS  18 

7.3.3  VMTI COMMON ELEMENTS  23 

8  _{OVERVIEW GMTI  30} 

Copyright © 2011 Open Geospatial Consortium v

8.2  _{GMTI TO UML/XML MAPPINGS  32} 

8.3  _OGC GMTI INFORMATION MODEL  33 

8.3.1  GMTI SEGMENTS  33 

8.3.2  GMTI EXPLOITATION CLASSES  34 

8.4  _GMTI OBSERVATION MODEL  35 

8.4.1  CORE DATA  36 

8.4.2  MISSION  38 

8.4.3  INHERITED PROPERTIES  39 

8.4.4  FEATURE OR INTEREST  39 

8.4.5  PROCEDURE  40 

8.5  _SITUATION AWARENESS OBSERVATION  41 

8.6  _TARGETING AND TRACKING OBSERVATION  42 

8.7  _{TARGET AND TRACKING HRR OBSERVATION  45} 

9  _{TRACKING MODEL  49} 

9.1  _STANAG 4676 STANDARD  49 

9.2  _{GENERAL MAPPING CONSIDERATIONS  49} 

9.3  _STANAG UML TO OGC UML MAPPING  51 

9.4  _INFORMATION MODEL  51 

9.4.1  FEATURE OF INTEREST  51 

9.4.2  OBSERVATION SPECIALIZATIONS  52 

9.4.3  TRACK ITEM  54 

9.4.4  TRACKINFORMATION  55 

9.4.5  COMMON  56 

9.4.6  ENUMERATIONS  57 

10  _{BOOKMARKING MODEL  58} 

 

Figures

Page

Figure 1: Observation as defined by O&M ... 7 

Figure 2: STANAG and MISB Overview ... 12 

Figure 3: TargetResult element if used without MISB 0601 LDS ... 14 

Figure 4: VMTI Features of Interest ... 18 

Figure 5: Mapping between KLV and XML elements (VMTI) ... 18 

Figure 6: Mapping between KLV and XML elements (VMTI) ... 19 

Figure 7: UML model of the VMTIObservation ... 20 

Figure 8: UML model of TargetObservation ... 22 

Figure 9: UML model of UASProcess ... 23 

Figure 10: UML model of VMTI ... 24 

Figure 11: UML model of VTargetPack ... 25 

vi Copyright © 2011 Open Geospatial Consortium

Figure 13: UML model of PixelNumber ... 28 

Figure 14: UML model of VFeature ... 28 

Figure 15: UML model of VMask and BitMask ... 29 

Figure 16: GMTI data flow (Nato2008) ... 31 

Figure 17: GMTI Transmissions (Nato2008) ... 32 

Figure 18: GMTI segments overview ... 33 

Figure 19: UML model of GMTIObservation ... 35 

Figure 20: UML model of GMTI ... 36 

Figure 21: UML model of Security ... 37 

Figure 22: UML models of Security fields ... 37 

Figure 23: UML model of ExerciseIndicator ... 38 

Figure 24: UML model of Mission ... 38 

Figure 25: UML model of PlatformType ... 39 

Figure 26: UML model of DwellArea ... 40 

Figure 27: UML model of GMTIProcess ... 41 

Figure 28: UML model of SituationAwarenessObservation ... 42 

Figure 29: UML model of SituationAwarenessTargetReport ... 42 

Figure 30: UML model of TargetingAndTrackingObservation ... 43 

Figure 31: UML model of TargetingAndTrackingReport ... 44 

Figure 32: UML model of TargetClassification ... 45 

Figure 33: UML model of TargetingAndTrackingHRRObservation ... 46 

Figure 34: UML model of TargetingAndTrackingHRRReport ... 47 

Figure 35: UML model of ProcessingTechniqueTypes ... 48 

Figure 36: UML model of HRRRDMType ... 48 

Figure 37: UML model of ScattererRecord ... 48 

Figure 38. NATO planned tracking integration architecture (source unknown) ... 49 

Figure 39: STANAG 4676 track model ... 50 

Figure 40: UML model of the feature of interest Track _{together with associated feature} types ... 52 

Figure 41: UML of alternative feature of interest SurveilanceArea _{... 52} 

Figure 42: UML model of AreaObservation ... 53 

Figure 43: UML model of TrackingObservation ... 54 

Figure 44: UML model of TrackItem ... 55 

Figure 45: UML model of TrackInformation ... 56 

Copyright © 2011 Open Geospatial Consortium vii

Figure 47: UML model of STANAG 4676 enumerations ... 58 

Figure 48: UML model of Bookmark ... 59 

Tables

Page Table 1: Overview of relevant standards ... 6 

Table 2: KLV to UML type mappings ... 16 

Table 3: VTargetPack KLV mappings ... 25 

Table 4: VTracker KLV to UML mapping ... 27 

Table 5: VChip KLV to UML mappings ... 29 

Table 6: VTargetPack LDS to UML Mappings ... 29 

Table 7: KLV to UML type mappings ... 32 

Table 8: GMTI to UML mappings ... 36 

Table 9: Mission to UML mappings ... 38 

Table 10: Dwell Area STANAG to UML mapping ... 40 

OWS-8 Information Model for Moving Target Indicators

and Moving Object Bookmarks

1 Introduction

1.1 Scope

This report aims at providing an information model for the usage of video moving target indicator data (VMTI), ground moving target indicator (GMTI) and tracking information (STANAG 4676) in the context of standardized spatial data

infrastructures compliant to OGC and ISO standards. If possible, precedence was given on using the OGC Sensor Web Enablement suite of standards, as this suite provides a homogeneous suite of standards to express sensor and sensor observation data in the context of OGC. This means that all encodings are based on Observation and Measurements version 2 (O&M) and implemented as an application schema according to the rules of Geography Markup Language version 3.2 (GML). An information model – so called ‘bookmark’ – to conserve the trace from a moving object back to the original base data is discussed briefly.

1.2 Document contributor contact points

All questions regarding this document should be directed to the editor or the contributors:

Name Organization

Ingo Simonis iGSI GmbH

1.3 Revision history

Date Release Editor Primary

clauses modified

Description

1.0 01/09/2011 Ingo Simonis all Initial version

1.0.1 28/09/2011 Ingo Simonis Document numbers of referenced material added. Spell checked.

Current 6/10/2011 Carl Reed Various Prepare for publication

1.4 Future work

The implementation of the MISB and NATO specifications using OGC models and guidelines works well. Further discussion is desirable to the following aspects:

Trace to evidence: The bookmark model is just a first high-level realization of a potential trace mechanism. Basically, STANAG 4676 already provides an option to link any track item to other data by providing a simple URI pointer to an additional resource. Further discussion is necessary with tracking experts to identify the possible options for optimizing this approach. Currently, the URI points to an arbitrary resource, which might be a radar data file sitting on a FTP server, a textual description on a Webserver, a motion imagery frame in a Web Accessible Folder, or any other type that can be referenced by a URI pointer.

Harmonization of approaches: The goal of this report was to analyze and demonstrate a potential mapping of VMTI, GMTI, and tracking data into the OGC concept of operations. This was done using different OGC base models and implementation practices. In future, the optimal approach shall be

identified and the respective standards modeled accordingly. This mainly requires intensive discussions with data producers and data users, as all tested approaches have been proven to be reliant and effective options.

Synchronization aspects between Local Data Set and Motion Imagery data: The current model requires intensive testing in terms of alignment of local data sets and motion imagery data. VMTI does not require providing both types of data in a synchronized way. This might lead to severe consequences in terms of bookmarking and tracing. The various levels of abstraction that may

Value range definitions, which are defined in the MISB specifications, have been modeled as UML constraints. In general, XML serialization using XML Schema as the only expression mechanism supports only rudimentary value range definitions. It needs further investigations to analyze to which extent other technologies such as e.g. Schematron or RelaxNG are necessary to ensure interoperability between communication partners by strictly defining the allowed content of the data exchanged.

Sensor description using OGC sensor description languages: Both, VMTI and GMTI specifications include information about the sensor and the

platform the sensor is mounted on. Both take the potential mobility of the platform into account. Due to the narrow timeframe of OWS8, all sensor and platform data was modeled using direct mapping to OGC types but ignored a possible representation using SensorML or SFL. A first brief analysis that is not documented in this report shows that the mobility aspect requires further investigations if it should be modeled as part of the sensor description rather than as observations. In general, the simpler and more efficient SFL provide reasonable options to express both GMTI and VMTI sensor description data. Intensive testing: The models described in this report have not been tested in real implementations. Though representing the results of intensive discussions between VMTI, GMTI, Tracking Model and OGC experts, an implementation may identify additional requirements that need to get incorporated by the models presented in this report.

Security:The transfer of the 4676 model to the OGC concept of operations requires a re-organization of the messages into complex feature-oriented data structures. This reorganization maps message-oriented data packages into end-user products such as either tracks or track segments. Eventually, a consumer can request all tracks for a given area of interest and receives nicely packed track point and track segment information that does not require any further alignment in order to be used. The re-organization from messages to processed features requires some agreement. For example, the “‘TrackMessage’ element contains a 'Track’ and each track contains one or more ‘Track Segments’, each of these elements having its own security classification.” If a number of track points, each with its individual security setting, get aggregated and form a track segment, the definition of the track segment security (and further on of the track itself) needs to be based on the aggregated security settings. The rule set for security aggregation requires further discussion with the STANAG 4676 community. At this stage, we make do with highlighting those aspects. 1.5 Forward

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium Inc. shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

2 Terms and definitions

All terms and definitions are used according to corresponding NATO and MISB standards. For a list of standards, please consult the bibliography at the end of this document.

2.1 Motion Imagery

(Definition from [MISB2010]) Motion Imagery is defined as imagery [a likeness or representation of any natural or man- made feature or related object or activity] utilizing sequential or continuous streams of images that enable observation of the dynamic, (temporal), behavior of objects within the scene. Motion Imagery temporal rates—nominally expressed in frames per second—must be sufficient to characterize the desired dynamic phenomena. Motion Imagery is defined as including metadata and nominally beginning at frame rates of 1 Hz (1 frame per second) or higher within a common field of regard. Full Motion Video (FMV) falls within the context of these standards. Within motion imagery, the following domains are currently specified:

Electro Optical (including video and television) Infrared (including low-light television)

LVSD – Large Volume Streaming Data Multispectral (MSI) / Hyperspectral (HSI)

3 Conventions

3.1 Abbreviated terms

ConOps Concept of Operations FoI Feature of Interest

FOV Field of View

GMTI Ground Moving Target Indicator GMTIF GMTI Format

HFOV Horizontal Field of View

HIS Hyperspectral

HRR High Range Resolution

ISR Intelligence, Surveillance, and Reconnaissance

KLV Key-Length-Value

LDS Local Data Set

MI Motion Imagery

MISB Motion Imagery Standards Board MPEG Moving Picture Experts Group

MSI Multispectral

MTI Moving Target Indicator

NATO North Atlantic Treaty Organization NTIF National Imagery Transmission Format

NSIF NATO Secondary Imagery Format (STANAG 4545) RRCA Radar Referenced Coverage Area

SA Situation Awareness

SMPTE Society of Motion Picture Television Engineers

UAS Unmanned Air System

UAV Unmanned Aerial Vehicle UML Unified Modeling Language VFOV Vertical Field of View

VMTI Video Moving Target Indicator

VSS VMTI Source Sensor

WA Wide Area

WAPS Wide Area Persistent Surveillance 3.2 UML notation

Most diagrams that appear in this standard are presented using the Unified Modeling Language (UML) static structure diagram, as described in Subclause 5.2 of [OGC 06-121r3].

4 VMTI, GMTI and Tracking in the Context of OGC

4.1 Management Summary

This report provides an information model for the usage of video moving target indicator data (VMTI), ground moving target indicator (GMTI) and tracking information (STANAG 4676) in the context of standardized spatial data infrastructures compliant to OGC and ISO standards.

It has been proven that all three specifications can get mapped to the OGC concept of operations using elements from the Geography Markup Language (GML) and

SweCommon, the data model and encoding defined by the Sensor Web Enablement suite of standards.

One issue is the definition of value ranges, which cannot be properly defined by XML Schema. XML Schema allows the definition of syntax patterns and enumeration values, but does not support the definition of e.g. the dwell area range half extent to be withinthe interval 0 to 255.9928km. There are additional technologies providing the required functionality available, e.g. Schematron.

The definition of bookmarks is partly covered by STANAG 4676, but requires further research. Currently, a single pointer with corresponding description field is all that is available. The referenced resource remains undefined otherwise. The developed model in OWS8 serves as a first high-level approach but requires further

investigation. For additional further work items, please consult section 1.4.

All information models documented in this report have been serialized using XML Schema. The schemas are introduced in and bundled to OGC 11-108.

4.2 Introduction

This report provides an information model for the usage of video moving target indicator data (VMTI), ground moving target indicator (GMTI) and tracking information (STANAG 4676) in the context of standardized spatial data

infrastructures compliant to OGC and ISO standards. If possible, precedence shall be given on using the OGC Sensor Web Enablement suite of standards, as this suite provides a homogeneous suite of standards to express sensor and sensor observation data in the context of OGC. This means that all encodings shall be developed based on Observation and Measurements version 2 (O&M) and implemented as an

application schema according to the rules of Geography Markup Language version 3.2 (GML).

The mapping of GMTI, VMTI, and Track Model can be done following different OGC traditions by implementing it as a GML application schema using either base GML types, using SweCommon as well as base GML types, or using SweCommon wherever possible. All three approaches shall be executed to test their applicability. All target indicator and tracking data is provided in the form of Key-Length-Value encodings. KLV (Key-Length-Value) is a byte-level data-encoding standard used for binary data byte-packing and metadata embedding into video feeds. Data is encoded into Key-Length-Value triplets, where Key identifies the data, Length specifies the data's length, and Value is the data itself. It is defined in SMPTE 336M-2007 (Data Encoding Protocol Using Key-Length Value), approved by the Society of Motion Picture and Television Engineers. KLV encoding protocol defines a data structure, which is independent of the application or transportation method used. In contrast, OGC preferred encodings make use of the extensible markup language XML. The goal of the work presented here was to develop a consistent conceptual mapping between incoming KLV data and XML encoded data provided at OGC service endpoints. The implementation of the data conversion as well as synchronization aspects between video or radar data with corresponding metadata was not subject of this work. A potential service portfolio to handle VMTI, GMTI and Tracking data is documented in OGC Engineering Report 11-134.

5 Used Standards

5.1 MISB & NATO

The following standards, engineering guidelines, and recommended practices form the base for this analysis:

Table 1: Overview of relevant standards

VMTI GMTI Tracking

MISB STD 0601 NATO 4607 v3 NATO 4676

MISB RP 0903 NATO 4607 AEDP-8 Edition

1 Implementation Guide Annex B to MISB RP

0903.2 NATO 4609

5.2 Observation and Measurement

The Observation & Measurement (O&M) specification defines a conceptual schema encoding for observations, and for features involved in sampling when making observations. As a central part of the OGC SWE suite of standards, it is used by all service encodings to express observation data. O&M is complemented by an XML implementation that uses an automated framework to convert the strictly profiled

UML models to XML Schema, i.e. derives a physical schema from the conceptual model. The XML model is compliant to ISO 193136 (Geography Markup Language). O&M defines an observation as “…an act associated with a discrete time instant or period through which a number, term or other symbol is assigned to a phenomenon. It involves application of a specified procedure, such as a sensor, instrument, algorithm or process chain. The procedure may be applied in-situ, remotely, or ex-situ with respect to the sampling location. The result of an observation is an estimate of the value of a property of some feature.” The following UML diagram illustrates this concept.

Figure 1: Observation as defined by O&M

An observation O&M emphasizes the relationship between the observed property, the feature of interest (FoI) and the procedure that estimates/assigns the value of that feature’s property. In situations where the observation either does not obtain values for the whole of a domain feature or the observation procedure obtains values for properties that are not characteristic of the type of the ultimate feature (e.g. beam response times as a proxy for targets), O&M uses the concept of the ‘sampling feature’. O&M defines the concept of sampling features as a “feature, such as a station, transect, section or specimen, which is involved in making observations concerning a domain feature.”

This concept can be applied to the VMTI, GMTI, and Tracking data. The feature of interest can be an individual target (moving object), an area under surveillance, a transect etc. The decision on the feature of interest may has effects on the observation specialization, as discussed in sections further below.

«FeatureType»

OM_Observ ation + parameter: NamedValue [0..*] + phenomenonTime: TM_Object + resultTime: TM_Instant + validTime: TM_Period [0..1] + resultQuality: DQ_Element [0..*]

constraints

{observedProperty shall be a phenomenon associated with the type of the feature of interest} {procedure shall be suitable for observedProperty} {result type shall be suitable for observedProperty} {a parameter.name shall not be used more than once} «FeatureType» OM_Process «Type» GFI_PropertyType «FeatureTyp... GFI_Feature MD_Metadata «type» Any {root} «metaclass» GF_FeatureType {root} «metaclass» GF_PropertyType {root} «DataType» NamedValue + name: GenericName + value: Any

Observ ationContext + role: GenericName

Phenomenon +observedProperty 1 +propertyValueProvider 0..* Domain +featureOfInterest 1 0..* +relatedObservation 0..* +generatedObservation 0..* ProcessUsed +procedure 1 Metadata +metadata 0..1 «instanceOf» «instanceOf» +result Range +carrierOfCharacteristics 0..* +theGF_FeatureType 1

The OM_Observation defines a number of properties with strict semantics and provides an option for further extensions by providing an parameter:NamedValue

placeholder: “If present, the attributes parameter:NamedValue shall describe an arbitrary event-specific parameter. This might be an environmental parameter, an instrument setting or input, or an event-specific sampling parameter that is not tightly bound to either the feature-of-interest (6.2.2.7) or to the observation procedure (6.2.2.10). To avoid ambiguity, there shall be no more than one parameter with the same name.”[ISO2010]

The other properties are defined as follows[ISO2010]:

phenomenonTime: The attribute phenomenonTime:TM_Object shall describe

the time that the result (6.2.2.9) applies to the property of the feature-of-interest (6.2.2.7). This is often the time of interaction by a sampling procedure (8.1.3) or observation procedure (6.2.2.10) with a real-world feature.

resultTime: The attribute resultTime:TM_Instant shall describe the time when the result became available, typically when the procedure (6.2.2.10) associated with the observation was completed

valideTime: If present, the attribute validTime:TM_Period shall describe the time period during which the result is intended to be used.

resultQuality:If present, the attributes resultQuality:DQ_Element shall describe the quality of the result (6.2.2.9). This instance-specific description complements the description of the observation procedure (6.2.2.10), which provides information concerning the quality of all observations using this procedure. Quality of a result may be assessed following the procedures in ISO19114:2003. Multiple measures may be provided (ISO/TS 19138:2006).

6 KLV to XML

6.1 KLV and XML Encodings

All discussed MISBmetadata-sets rely on SMPTE 336M Key-Length-Value

constructs in order to provide transport encodings optimized for minimum bandwidth consumption. Early datasets in the VMTI processing chain from on-board VMTI processors may contain only minimum VMTI data in order to minimize the overhead on streams with high sampling rates such as 60 frames per second during transport over the air. Once more powerful transports become available or feature-richness compensates for derivation from real-time processing, more metadata elements will be added to the VMTI stream. In order to be even more bandwidth efficient, VMTI differentiates three different types of data: “dynamic data, periodic data, and static data. Dynamic data changes continuously and is only valid at a specific instant in time. Periodic data changes periodically and is valid for a period of time. Static data rarely, if ever, changes within a single mission.” [MISB0903.2].

GMTIF uses a somewhat similar approach. The GMTI format “is structured as a set of Message Segments, with each Message Segment designed to carry specific types of information. STANAG 4607 transmission is accomplished by means of packets, where each packet consists of a Packet Header and a number of Message Segments

size limit of a GMTIF packet or the constraints of the transmission media, the format allows a portion of the data to be sent in one GMTIF packet and the remainder of the data to be sent in subsequent GMTIF packets. A Segment Header, which defines the type of message and the length (in Bytes) of the following segment, precedes each Message Segment.” [Nato2008].

OGC ConOps make use of XML encodings to exchange data between communication partners, which, even though the XML data may get efficiently compressed for

transport, usually do not allow real time processing of data, but are optimized for expressing complex hierarchical data structures. Referencing mechanisms can get applied for static or periodic data to minimize the size of exchanged data to some extent, but in general the OGC ConOps give expressiveness precedence over compactness. Having this in mind, any XML encoding of VMTI/GMTI data shall focus on the usability of exchanged data, which includes a well-balanced trade-off between in-line encodings and data elements provided by reference.

It has to be acknowledged that OGC Web services usually do not support communication patterns that conserve state. Thus referencing and caching

mechanisms require client-side implementation support. For this reason, in-protocol size and caching optimization have not been addressed in this report.

In summary, the OGC XML implementation shall focus on optimized data structures rather than transport efficiency. It can be assumed that provision of static data has more weight than message size minimization to support state-less communication with arbitrary clients.

6.2 KLV to XML Conversion

6.2.1 Conversion Guidelines

The conversion of KLV-encoded VMTI/GMTI data to OGC XML models has been performed along the guidelines and rules that can be derived from the OGC feature model and the Observation and Measurement information model:

Avoid multiple inheritance, as experiences have shown that usually it can be avoided and may cause trouble during serialization and later usage

Put role-names on association ends in the form of meaningful nouns, not verbs such as ‘uses’ or ‘has’, as this would lead in a number of associations labeled equally

Use navigable association ends if possible to clarify the readout direction Use lexical forms of nouns for all terms and labels; don’t use codes, but short singular nouns or camel case compound words if necessary

Use constraints to further clarify your model; simple text or constraint languages such as OCL may be used

Don’t confuse diagrams and models: Use multiple diagrams to model the various aspects of the application model and avoid putting everything into a single diagram

Use package diagrams to model ownership, dependencies, and maintenance arrangements between your models or between your model and the ISO base models and use package stereotypes to allow for automated model

o _{Stereotype ‘Application Schema’ for the independent packages in their}

own namespace

o Stereotype ‘Leaf’ for convenience packages to be serialized in separate XSD documents but in ‘Application Schema’ namespace and with no further sub-packages

Use class diagrams to develop your model and follow strict class stereotyping rules to allow for automated model serialization:

o _{Stereotype ‘FeatureType’ to express domain classes with instances}

having identity

o _{Stereotype ‘Type’ for those aspects that have identity but are not a}

feature

o _{Stereotype ‘DataType’ for data structures with no external identity} o Stereotype ‘Union’ for ad-hoc choices

o Stereotype ‘Enumeration’ for closed enumerations

o _{Stereotype ‘Codelist’ for extensible enumerations}

Use object diagrams to instantiate your model, as those diagrams are often easier to read for end users than the more abstract class models

6.2.2 Conversion Framework

The conversion process of UML based models to XML Schema was supported by a framework called “Fullmoon”, developed by CSIRO Australia. The framework processes and transforms serialized UML models to XML Schema according to the conversion rules defined in ISO 19136. The rules are maintained as XQuery scripts external to the application and can be modified as necessary.

In order to apply automated conversion using Fullmoon, the UML models shall follow a strict profile (patterns, stereotypes, tagged values). This profile is provided by another framework called HollowWorld, which again is developed by CSIRO Australia. It implements ISO 19136 stereotypes, ISO 19136 conformant tagged values, and all schemas from the ISO 191xx suite of standards.

Both frameworks have been used to convert the UML models defined in this report to XML schema. The necessary additional class maps have been defined.

6.2.3 Conversion Table

Different format mappings have been explored to map between VMTI respectively GMTI and UML/XML data types. For VMTI, types and patterns from the geography markup language (GML) have been used, whereas SweCommon data types have been used to map between GMTI and UML/XML data except for time and textual

(alphanumeric) fields.

In summary, it could be stated that both mappings work fine. As SweCommon does not provide all necessary base types (e.g. spatial types), SweCommon has been

amended with those types where necessary. Further research is necessary to decide for one or the other approach.

7 VMTI in the OGC Concept of Operations

This chapter describes the UML models developed based on MISB documents 0903.2 and 0601.4. Though general knowledge of those standards is expected, the following

aspects shall be highlighted, as they influenced a number of OGC information model design decisions:

0903.2 data usually is embedded in 0601 data streams. If provided “naked”, a few information items from 0601 stream elements need to get conserved in order to allow proper interpretation of 0903.2 elements.

There is no 1:1 mapping between VMTI data and MI frames. “VMTI data does not have to be delivered at the frame rate of the motion imagery. Data that rarely changes should only be delivered often enough to assure that it is included in any clip extracted from the motion imagery stream.

[MISB0903.2]”.

Association between VMTI and MI cannot always be assumed. For example in cases of wide area persistent surveillance, it may not be possible to

download MI for all objects/areas. One ofthe two following solutions is usually applied: (1) Usage ofspotlights to carry MI data for dedicated regions, or (2) Usage of cues; cues require self-contained VMTI data with no

dependencies on other data streams, as associated MI streams cannot be assumed.

In addition to the discussion of KLV vs. XML data encodings (see clause 6), it has to be noticed that VMTI data in its simplest form may only include target identifiers and centroid pixel numbers, but can get enriched to hold multiple features and descriptive elements about each target. The XML implementation shall cater for all possible content in a VMTI data stream. It shall allow

simplistic data sets as well as full-featured target representations. In contrast to the VMTI data stream, where “it is undesirable to populate elements just because tags exist to support the data [MISB0903.2]”, XML representations shall provide as much information as possible, and may provide a variety of offerings adapted to specific clients’ requirements.

7.1 VMTI Standards

The information models developed in this report are mainly based on the MISB standard 0601.4, MISB recommended practice 0903.2, and NATO standard 4609.3. Those specifications reference or make use of a set of other standards, recommended practices, and engineering guidelines. The most important ones are outlined in the figure below to allow for a quick relationship assessment.

Figure 2: STANAG and MISB Overview

7.1.1 MISB STD 0601.4

MISB standard 0601.4 details the Unmanned Air System (UAS) Datalink Local Data Set (LDS) for UAS platforms. The UAS Datalink LDS is an extensible SMPTE (Society of Motion Picture Television Engineers) Key-Length-Value (KLV) Local Metadata Set designed for transmission through a wireless communications link (Datalink).

This Standard provides direction and requirements for the creation of a SMPTE 336m-2007 (data encoding protocol using key-length-value) compliant Local Data Set for a reliable, bandwidth-efficient exchange of metadata among digital motion

imagery systems. The LDS enhances sensor-captured imagery with relevant metadata. It also provides a mapping between UAS Datalink Local Data Set items, Exploitation Support Data (EDS) items, and Universal Data Set (UDS) items defined in the

SMPTE KLV dictionary (RP-210) as well as in the MISB-managed Department of Defense (DoD) keyspace. The latest revision of the specification (0601.4, released 4 March 2010) defines some 80 keys and their mapping to the corresponding UDS items.

The standard does not define how LDS metadata and MI data shall be synchronized, but puts this into the responsibility of the system designer.

MISB 0601.4 allows embedding STANAG 4609 (VMTI) metadata. 7.1.2 MISB RP 0903.2

According to [MISB0903.2], MISB Standard 0601 UAS Datalink Local Data Set (current version 0601.4 of March 2010) “has become the accepted standard Local Data Set within Defense agencies for the transmission of metadata elements within motion imagery streams. 0601 includes numerous individual elements and a few local data subsets […]. The purpose of [0903.2] is to define an LDS for VMTI that

complements 0601 metadata. Tag 74 from 0601 has been assigned for the VMTI LDS.”

Though MISB 0903.2 complements MISB 0601.4, it has been assigned a full 16-byte key to the VMTI LDS and can be transferred without any video stream and/or

metadata such as MISB 0601.4. This allows detectors and trackers to communicate VMTI data that only exists in pixel space. It would be valid to encode pure MISB 0903.2 VMTI data according to OGC models and encodings. Although to be useful, the detector shall fill-in real world coordinates in all target results. Figure 3illustrates this requirement. The red classes (optional by default) become mandatory if VMTI LDS is used “naked” (in order to define the position of the target by real world coordinates). All grey elements become irrelevant, as they rely upon the frame center coordinates that appear in the 0601 LDS (or other data in 0601 LDS that allow the calculation of the frame center coordinates). The yellow elements define elements that can exist independently of the definition of real world or pixel space coordinates.

Figure 3: TargetResult element if used without MISB 0601 LDS

If the initial detector does not provide the geo-location in addition to pixel

coordinates, the embedding of MISB 0903 data in MISB 0601 LDS is essential for the pixel space to geo-location mapping.Thus the OGC ConOps-based implementation of VMTI largely depends on incoming data streams. Any incoming ‘naked’ VMTI data gets mapped directly to the corresponding XML elements. Embedded VMTI data will be analyzed for content. Potentially required data to allow deriving absolute values out of values provided relative to some point of reference needs to be extracted from the embedding protocol and provided as part of the XML representation. Motion imagery material can be provided either inline or by reference depending on availability and storage capacities. It shall be noted that there is currently no specialized motion imagery data access service interface provided by OGC. The general rule set in MISB 0903.2, “a VMTI LDS is embedded within a 0601 LDS”, can be released for transactional OGC ConOps if the coordinates are provided as part of the target results. Nevertheless, following the principle that as much as possible incoming data shall be preserved during the mapping of VMTI data streams to OGC ConOps, we focused on a model that supports the entire TargetResult spectrum of possible elements. To be useful, the full application schema needs to provide mapping information coming from the embedding 0601 stream.

7.1.3 STANAG 4609v3

The aim of this STANAG is to promote interoperability for the exchange of digital Motion Imagery among NATO C3I Systems. The current version is edition 3,

released October 13, 2009. The standard played a subordinated role in this analysis, as STANAG 4609v3 contains the older MISB engineering guideline 0903.0. MISB RP 0903.2 will be part of the next version of STANAG 4609, to be offered up to NATO in the Fall 2011.

7.1.4 Other Relevant Standards

Other important standards, recommended practice, engineering guidelines and concepts that are not covered in this engineering report include

MISB STANDARD 0102.9 - Security Metadata Universal and Local Sets for Digital Motion Imagery “describes the use of security metadata in MPEG-2 digital motion imagery applications”. MISB 0102.9 is referenced by MISB 0903.2 and – at least an earlier version of MISB 0102 – in MISB 0902.1. MISB STANDARD 0604.2 - Time Stamping and Transport of Compressed Motion Imagery and Metadata: “This Standard defines methods to time stamp compressed motion imagery streams and to transport compressed motion imagery and metadata in an MPEG-2 Transport Stream protocol.” This standard is not further elaborated, as synchronization between LDS items and motion imagery data is out of scope for this analysis. It shall be assumed that OGC services providing VMTI or GMTI

MISB STANDARD 0902.1 – Motion Imagery Sensor Minimum Metadata Set “consists of metadata elements taken from MISB Standard 0601 to enable the minimum functionality required for both Discovery & Retrieval of source imagery and the Situational Awareness Product for ISR mission accomplishment.” It is not further elaborated here, as the root standard MISB 0604.1 is addressed.

MISB RP 0603 – Common Time Reference for Digital Motion Imagery Using Coordinated Universal Time (UTC) “defines setting and using common UTC time reference for digital motion imagery”. This deterministic common time reference allows for correlation of motion imagery frames and metadata. UTC time is used throughout all models.

MISB EG 0104.5 – Predator UAV Basic Universal Metadata Set: “This EG provides direction on the creation of a standard metadata set for reliable exchange of Predator closed caption (CC) data among digital motion imagery systems. The scope of this EG was originally intended for metadata that originated as closed caption metadata in analog video from the Predator UAV. After several years the use of this standard has grown beyond its original purpose; some systems are using this standard to directly create universal sets. To address this issue the MISB is working on a future standard, which will encompass a large list of possible metadata, provide more flexibility and aid in better motion imagery analysis. Analog video and closed caption metadata are legacy systems that may continue to be used during the transition to all-digital sensors and information infrastructures. This EG facilitates that transition only and is not intended to constitute an approved end-system implementation.” MISB EG 0104.5 is referenced by 0903.2. It is not further elaborated here, as the important aspects have merged or re-defined in MISB 0903.2.

MISB EG 0801.2 - Engineering Guideline: Profile 1: Photogrammetry Metadata Set for Digital Motion Imagery (from December 2009, currently under review): This Engineering Guideline presents the KLV metadata and metadata structures necessary for the dissemination of data required for the photogrammetric exploitation of motion imagery. The ability to geo-locate points on an image with known confidence is a necessary pre-requisite to targeting. This Standard therefore is intended as a necessary step along the way to meeting the legal and operational requirements to allow targeting from motion imagery. The metadata structures of this Standard are designed to allow for flexible, bit-efficient packaging of the necessary data. This document concerns itself solely with the metadata and metadata structures specific to photogrammetry; metadata necessary for the primary exploitation of the motion imagery (including such elements as mission number, sensor type, platform type, etc.) and security metadata are not addressed in this Engineering Guideline. This Engineering Guideline is designed to be used in conjunction with MISB EG 0601.1: UAV Datalink Local Data Set, MISB RP 0603: Common Time Reference for Digital Motion Imagery Using Coordinated Universal Time, and MISB EG 0102.5: Security Metadata Universal and Local Data Sets to cover security, timing, and primary exploitation metadata.

SMPTE RP210.10-2007, SMPTE Metadata Dictionary Contents; defines the Universal Data Set items

7.2 VMTI to UML/XML Mappings

The following format mappings have been applied to map base KLV types to UML types.

Table 2: KLV to UML type mappings

Data Type KLV UML

Unsigned Integer Uint8 Integer

Uint16 Integer

Uint24 Integer

Uint48 Integer

Uint64 Integer

Float Int24 Real

Uint16 Real

Uint24 Real

Structure <Name> <Name>

All structures have been mapped to either UML Datatypes or FeatureTypes. Those mappings are elaborated in full detail in the following subclauses.

7.3 OGC VMTI Information Model

The objective of mapping VMTI data to the OGC concept of operations was seamless integration of moving object information in spatial data infrastructure based on OGC information models, encodings, interfaces and protocols. The SWE suite of standards had been identified as the ideal collection of standards, because it provides a

harmonized set of service interfaces, information models and corresponding encodings. SWE utilizes the O&M standard to encode observation data, whereas sensor specific data gets represented using SensorML. The OGC information model for VMTI data therefore requires a VMTI specific profile of O&M, which manifests in the definition of derived specializations of the base observation types defined by O&M. Sensor and platform specific data might get encoded as SensorML and it used in the observation model as part of the process description.

7.3.1 Feature of Interest

For the OGC VMTI information model, we follow the classic modeling approach and define the feature of interest first.Specialized O&M observations will then bind result values to properties of that feature of interest. For VMTI data, two features of interest types can be differentiated:

1. An individual frame from the video stream

2. The area under surveillance, which is observed using video sensors

No precedence can be given any precedence for one option over the other, as both realizations need to be considered in real world applications. For most applications, it can be assumed that the user of VMTI data is interested in detected moving objects for a given real world area rather than individual frames from a video stream. In this situation, the frames are just sampling features with the goal to provide data for the ultimate feature of interest ‘surveillance area’.

Nevertheless, we demonstrate both options, knowing that in other applications, video frames canbe more appropriate ultimate features of interest.Though not further elaborated in this report, both the individual Frame and the

SurveillanceAreacanoptionally be associated, as each surveillance area might be covered by any number of frames. Even additional associations are possible but require further discussions.

Eventually, it should be acknowledged that there are many more potential types of feature of interest. For example, a video stream could be considered as a feature of interest, or even an individual moving object. The two types described in this report have been identified as most relevant after much discussion with VMTI experts and developers.

Figure 4: VMTI Features of Interest

The SurveillanceArea plays the role of the ultimate feature of interest for the Frame

class, though – as described above – the Frame class can act as the ultimate feature of interest as well.We considered developing a number of additional associations, but dropped the idea to keep the model as clear as possible.

As the decision on the feature of interest has effects on the observation type, we differentiate two perspectives in the following subclauses:

1. The VMTI-oriented perspective with a single frame as feature of interest. 2. The target-oriented perspective with the surveillance area as feature of interest Though the different perspectives result in different observation specializations, most of the elements will be identical.

7.3.2 Observation Specializations

The VMTI-oriented perspective is implemented in the form of a VMTIObservation, a specialization of the OM_ComplexObservation defined by O&M. In the highest granularity, VMTI KLV elements provide data for a single frame that is therefore modeled as the feature of interest in the VMTIObservation. The KLV element contains further an element called VTargetSeries that acts as a container for

VTargetPacks. VTargetPacks include the information for a single target each. The mapping between KLV VMTI and OGC VMTI is illustrated below.

All VTargetPacks as well as the VMTI dataset are transferred into the result container of the observation, whereas the VTargetSeries container becomes superfluous in the hierarchical UML/XML model.

Figure 5: Mapping between KLV and XML elements (VMTI)

The target-oriented perspective focuses on delivering observation data for a given area. Consequently, the area under surveillance is modeled as the feature of interest.

Because data for a given area may result from any number of sensors, each observation is modeled to bind the data of a single target to a single process and references a single VMTI instance. The KLV to UML mapping is illustrated below.

Figure 6: Mapping between KLV and XML elements (VMTI)

In contrast to the VMTI-oriented perspective, each target described in a VTargetPack results in a single observation instance.

7.3.2.1 VMTI-Oriented Perspective

VMTI can be either transported embedded in 0601 data streams or “naked”, i.e. without embedding 0601 KLV data. In the first case, the 0601 elements contain essential frame center information necessary to interpret relative data provided in the

VTargetPack.This data is provided using the ‘parameter pattern’ of O&M and described using constraints (see Figure 7).

Further 0601 information may end up in the procedure element of type UASProcess. The detailed definition of this element is subject to further research.

The feature of interest Frame element may contain either a reference to or the motion imagery frame itself. If no motion imagery frames are available for a given VMTI data set, then the feature of interest remains an empty element, i.e. does neither provide the imagery inline nor a reference to it.

Figure 7: UML model of the VMTIObservation

The VMTIObservation itself is derived from OM_ComplexObservation(grey element in Figure 7) and therefore provides a number of attributes that shall be further defined here:

phenomenonTime: this mandatory parameter shall be used to encode the timestamp from the VMTI KLV element. This is according to O&M: “the attribute phenomenonTime:TM_Objectshall describe the time that the result applies to the property of the feature-of-interest” (O&M).

resultTime: this mandatory parameter shall provide the same time as the

phenomenonTime. If the observation data is the result of further VMTI data processing, the time when the final result was assigned could be provided alternatively.

validTime: this optional parameter shall according to O&M to “describe the time period during which the result is intended to be used” (O&M). It does not have any equivalent in KLV VMTI.

resultQuality: this optional parameter shall only be used if quality information in addition to the quality information provided in elements of the result set can be provided. It does not have any equivalent in KLV VMTI.

7.3.2.2 Target-Oriented Perspective

The specialized observation model for the target-oriented perspective looks very similar to the VMTI-oriented model described above. The properties

phenomenonTime, resultTime, validTime and resultQuality shall be used as defined above. Illustrated below, the target-oriented model differentiates in three properties:

1. The feature of interest is SurveillanceArea instead of Frame

2. The cardinality of the associated VTargetPack in the result element is 1 instead of 0..*

3. The result element optionally associates the Frame class to provide a link to the evidence for the target information

Figure 8: UML model of TargetObservation

The SurveillanceArea element does not have any equivalent in KLV VMTI. It is characterized by a name and a spatial property to define the area geographically. It may require further concretization. It has been introduced to support the

geographically oriented usage, which is often applied to data served at OGC service interfaces. It allows querying tracking data based on the spatial extent of the target area.

7.3.3 VMTI Common Elements

Most elements of the VMTIObservation and the TargetObservation are used

identically. Those elements are further defined in the following sections. The list of common elements include:

UASProcess VMTI VTargetPack VTracker 7.3.3.1 UASProcess

The UASProcess is derived from the abstract OM_Process defined by O&M. It provides a container for all sensor or platform data that needs to get conserved. Its content is not further defined in this report, as no specific requirements have been identified for the integration of VMTI data in the OGC concept of operations.

UASProcessis compliant with the O&M model, which foresees for all sensor, process, or platform specific data that further describes the generation of the result values for the properties of the feature of interest a container element.

Figure 9: UML model of UASProcess

7.3.3.2 VMTI

The VMTI maps all data from the VMTI element defined in table 2 of MISB 0903.2 with the only exception of VTargetSeries. VTargetSeries is a container that becomes superfluous in the hierarchical UML/XML models. The content of that container is provided as part of the result element of an observation.

Figure 10: UML model of VMTI

VMTI makes exclusively use of the standard mappings defined in Table 2. 7.3.3.3 VTargetPack

The VTargetPack delivers metadata for individual targets. Each VTargetPack

supports one target. The VTargetPack has been modeled to map all elements defined in table 3 of MISB0903.2. Individual fields have been aggregated according to the Video Moving Target Class Model described in Annex B to MISB 0903.2.

Figure 11: UML model of VTargetPack

All properties that do not use any of the standard mappings defined in Table 2are identified in the following table and will be further described in the following subsections if the UML data type contains non-trivial mappings.

Table 3: VTargetPack KLV mappings

Key Name Data Type KLV Format UML Rational

Target Location Structure Location Location Complex structure

mapping Target

Boundary

Structure Boundary Boundary Complex structure

VMask LDS Structure N/A VMask Complex structure mapping

VObject Structure N/A VObject Complex structure

mapping

VFeature LDS Structure N/A VFeature Complex structure

mapping

VTracker Structure N/A VTracker Complex structure

mapping

VChip Structure N/A VChip Complex structure

mapping Target Location

Latitude Offset

Float Int24 LocationOffset Aggregation of related

properties Target Location

Longitude Offset

Float Int24

Bounding Box Top Left Pixel Number

Unsigned Integer

Uint48 BoundingBox

Pixel

Aggregation of related properties Bounding Box Bottom Right Pixel Number Unsigned Integer Uint48 Bounding Box Top Left Latitude Offset

Float Int24 BoundingBox

GeoOffset

Aggregation of related properties

Bounding Box Top Left Longitude Offset

Float Int24

Bounding Box Bottom Right Latitude Offset

Float Int24

Bounding Box Bottom Right Longitude Offset

Float Int24

It shall be mentioned that MISB RP0903.2 has a redundancy in “Table 3: VTarget Pack”, page 41, in conjunction with “Table 9: Location Truncation Pack” on page 48. Both tables list an element labeled TargetHeight with the same description. Though it is in general possible that the target height can get transferred in different KLV structures, it seems that ID12 in table 3 has been introduced by mistake, as it is

provided in direct vicinity of the target location longitude and latitude offset values, which describe the offset for target from frame center (0601) and not as absolute positions.

Independently of the potential redundancy issue, the target height can be either transferred together with target location offset data, which is relative to the frame center location, or together with further location data as absolute position data. The UML features both options. The targetLocation element of data type Location has two options to encode the target height. It either allows encoding a three-dimensional point structure to include the height, or provides a dedicated property height to provide target height information in case the height is provided in absolute values but the target location longitude and latitude are provided relative to some frame center positions.

7.3.3.4 VTracker

The VTargetPack element VTracker is further associated as follows:

Figure 12: UML model of VTracker

The VTracker uses a number of non-trivial mappings as defined in the table below.

Table 4: VTracker KLV to UML mapping

VTracker Start Time Stamp

Unsigned Long

Uint64 TM_Instant

End Time Stamp

Unsigned Long

Uint64 TM_Instant

Locus Structure Series Location

Velocity Structure Velocity Velocity

Acceleration Structure Acceleration Acceleration

7.3.3.5 PixelNumber

The PixelNumber element as illustrated below specifies the position of a pixel within a frame. An array of pixels can be used to define a polygonal structure.

Figure 13: UML model of PixelNumber

The calculation of the pixel number uses the equation: X + ((Y-1) x Frame Width)). The top left pixel of the frame equates to (X,Y) = (1,1) and a pixel number of 1. 7.3.3.6 VFeature

This class contains data that describes the properties or features of a target.

VFeature is currently modeled at a very abstract level. As illustrated in the figure below, VFeature serves as a container element for additional features (feature:

GFI_Feature) modeled according to a named schema (schema: CharacterString).

Figure 14: UML model of VFeature

A potential extension would define a specialized observation as GFI_Feature. That observation would bind the motion imagery frame or any subsection thereof with the edge detection algorithm and with the resulting edges that led to the identification of that target. This extension would further define the evidence trace from the initial video material to the detected targets.

7.3.3.7 VMask

This class defines a bounding box around the target consisting of two geo coordinates, one for the upper left and one for the lower right corner.

Figure 15: UML model of VMask and BitMask

VMask may associate a BitMask, which captures a run-length encoding of a bit mask describing the pixels that subtend a target within the video frame.

7.3.3.8 VChip

This class specifies a “chip” of the image frame, capturing the target image.

Table 5: VChip KLV to UML mappings

Key Name Data Type

KLV Format

UML Rational

Image Type String ISO-7 CharacterString Direct mapping

Image URI String ISO-7 URI Direct mapping

Embedded Image

- Wrapper base64Binary Direct mapping to GML standard

image encoding

7.3.3.9 Location

This class provides geo coordinates for a point on or near the surface of the Earth as well as standard deviation and correlation coefficients. Though the latter two make use of trivial mappings, the latitude, longitude and height elements are aggregated in a

GM_Point.

Table 6: VTargetPack LDS to UML Mappings

VMTI LDS Key Name Data Type KLV Format UML

Location Truncation Pack

Latitude Float Uint32 GM_Point

Longitude Float Uint32

8 Overview GMTI 8.1 GMTI Standards

“The GMTIF is the standard for formatting and exchanging ground moving target indicator information and related products between NATO nations. The GMTIF standard is part of a family of standards that are assembled under the NATO Joint Capability Group on Intelligence, Surveillance and Reconnaissance (JCG-ISR, formerly Air Group IV for ISR), to ensure the exchange of multi-national intelligence and reconnaissance information. [...]

The NATO GMTIF standard (STANAG 4607) defines the data format for ground moving target indicator radar data, regardless of the level of sophistication of the radar system. Conformance with the NATO GMTIF does not in itself provide

complete interoperability, since it defines only the presentation layer protocol (Layer 6) of the International Standards Organization - Open Systems Interconnection model (ISO/IEC 7498-1), and other layers must be defined by additional specification. However, STANAG 4607 does provide data that can be interpreted by any compliant ground system. The format is scalable to all levels of capability. Small-scale systems can use only those elements of the format required to transmit their data, while more robust systems can use more aspects of the format to encode all of the information. To accomplish this scalability, the format uses two technical approaches. First, the format is divided into segments, with no predefined order or sequence other than the requirement to preface data segments with appropriate header segments, as defined in the standard. Each system using the standard is free to select the particular segments it requires for the data produced. Secondly, the data fields within the segments are identified as either Mandatory, Conditional, or Optional. Mandatory Fields are essential to the format and must always be sent. Conditional Fields are dependent on the presence or absence or the value of certain other fields and are sent only if they meet established conditions. Optional fields are not required but may be transmitted if they are available and if they provide added value or utility and are not constrained by communications or operational considerations. With these

approaches, each segment can be tailored to the data format requirements of the particular system.

In addition to its use as a stand-alone format, the GMTI data can also be formatted in accordance with this standard and then encapsulated in either of the NATO image formats (the NATO Secondary or Primary Imagery Formats, STANAGs 4545 or 7023, respectively). This feature allows additional data, not included in this format, to be transmitted in conjunction with the GMTI data.”[NATO2008].

In this analysis, we focused on the 4607 core data and did not take into account any potential embedding option. This means that the link between the GMTI to raw radar data or radar image data link gets broken.After initial processing, radar data is usually stored in a frame-oriented format, such as the NATO Secondary Imagery Format

(NSIF, STANAG 4545) or the National Imagery Transmission Format (NITF, MIL-STD-2500) for the dissemination of secondary imagery, or in a message- oriented format such as the NATO Primary Imagery Format (STANAG 7023) for the dissemination of primary imagery.Those formats can embed (usually by reference) GMTI data, but not vice versa.

The following diagram illustrates the typical data flow from sensor systems to exploitation systems and shows where GMTIF is usually used. The sensor system provides “typically a digitized video signal, such as I and Q data, and is not suitable for transmission using the GMTI format” (Nato2008). The IQ data contain (pre-spectral) in-phase (I) and quadrature (Q) radar return samples, which are both required for Doppler spectral calculation.

From on the detection system onwards, GMTIF is used. Initial detection systems can transfer the data optionally to additional processing systems or directly to exploitation systems.Exploitation Systems include: “Trackers, situational awareness displays, evidence accumulators, automatic target recognition (ATR), fusion or correlation with other sensor data, and other systems that exploit GMTI data. Note that the GMTI Format is intended to provide the detections and the supporting information needed by those exploitation systems; it is not intended to provide a format for exploitation products.” (Nato2008)

Figure 16: GMTI data flow (Nato2008)

In general, GMTI data is used at various direct links between systems. Data links for (over the air) transmissions may be used. The following diagram illustrates the usage of GMTIF within different systems and platforms.

Figure 17: GMTI Transmissions (Nato2008)

8.2 GMTI to UML/XML Mappings

The following format mappings have been applied to map base GMTI types to UML types.

Table 7: KLV to UML type mappings

Data Type GMTI UML

Integer I8, E8, S8, I16,

S16, I32, S32

Count

Float B16, B32, H32,

SA32, BS32

Quantity

Structure <Name> <Name>

Character Alphanumeric CharacterString

All structures have been mapped to either UML Datatypes or FeatureTypes. Those mappings are elaborated in full detail in the following subclauses.

8.3 OGC GMTI Information Model

GMTIF uses a packet oriented encoding model. A packet header is sent at the beginning of each packet to identify “the format version of the data contained in the packet, the size of the packet, and information pertaining to the platform, security, and the mission.”

8.3.1 GMTI Segments

STANAG 4607 v3 defines ten message segments to encode GMTI data (all used in this report), two message segments for interactive communication (ignored in this report, as interactive communication was not addressed), and a number of

placeholders, which are not further defined yet. Figure 18 provides an overview of all segments. Green segments have been used in this report, red ones ignored, and grey ones represent placeholders in the current specification.

Figure 18: GMTI segments overview

According to STANAG 4607v3, the individual segments are defined as follows: The Mission Segment provides information concerning the mission plan, the flight plan, the platform type and configuration, and the reference time for the mission.

The Dwell Segment is sent for each dwell of the radar beam. It provides information related to dwells and revisits, the sensor location, the coverage area, the time of the dwell, sensor orientation, and sensor parameters. It

includes Target Reports for any GMTI detections observed within that dwell and shall be sent even if no targets are detected.

The High Range Resolution (HRR) Segment provides data on HRR and Range-Doppler measurements, which may be performed in conjunction with MTI detections. It includes HRR Scatterer Data pertaining to the HRR measurements.

The Job Definition Segment provides a definition of the radar job performed by the sensor, including information pertaining to the geolocation model used in the sensor measurement. The Free Text Segment provides a means of sending alphanumeric text messages. The Test and Status Segment provides a means of exchanging health and status information of the platform systems. The Processing History Segment provides a means of annotating the radar data to show its history as it is processed through various systems during

transmission.

The Platform Location Segment provides the means for the platform to transmit its location during periods when it is not collecting data.

The Job Request and Job Acknowledge Segments are recommendations only and are not required for this format. The Job Request Segment provides a recommended format for requesting service from the sensor platform. The Job Acknowledge Segment provides a recommended format for acknowledging a sensor service request by a sensor platform, defining the job to be performed by the sensor, and notifying the requesting operator whether the task can be accomplished or not during the mission.

The Group, Attached, LRI, and System-Specific Segments are undefined at this time and left for future definition.

8.3.2 GMTI Exploitation Classes

As there is no “predefined order or sequence other than the requirement to preface data segments with appropriate header segments”, raw GMTIF can hardly get

directly converted into a hierarchical UML/XML model compliant to the OGC feature model approach. Instead, the GMTI data needs to get structured and organized to prepare it for further efficient processing. In this regard, we follow the

recommendations provided in the STANAG 4607 implementation guide, which defines three exploitation classes:

Situation Awareness (SA): The minimum data required for Moving Target Indicator (MTI) target display.

Targeting and Tracking (TT): The minimum data required for targeting and tracking of MTI targets using current or advanced automatic tracking

algorithms or precision location systems such as the Global Positioning System (GPS).

Targeting and Tracking with High Range Resolution (HT): The minimum data required for targeting and tracking of MTI targets using current or

advanced automatic tracking algorithms or precision location systems and High Range Resolution/Range-Doppler (HRR/R-D) analysis of associated MTI targets

8.4 GMTI Observation Model

Each exploitation class is modeled as a specialization of OM_Observation. The following figure illustrates this mapping, which allows using GMTI data inline with the recommended practice of the STANAG 4607 implementation guide. The

approach documented here makes use of different observation types. Alternatively, we could have modeled different result types, which are bound to a single GMTI observation type. The first approach was given precedence here, as it has the

advantage that the mappings become more transparent and observation parsers can be better optimized for observation interpretation.

The GMTIObservation contains two properties in addition to those inherited from

OM_ComplexObservation: gmti and mission.

In the GMTIObservation, we explored the usability of data types defined in

SweCommon to evaluate the applicability of this standard in the context of moving target imagery data. Though there is an ongoing discussion if SweCommon types shall be used in situation where a suitable and potentially simpler GML type or pattern is available – e.g. gml:MeasureType, gml:TimePeriod, gml:TimeInstant, xlink, we used this opportunity and used SweCommon throughout the entire model. Further discussion is necessary to discuss the future final solution.

8.4.1 Core Data

The gmti property contains all information provided in a GMTI header except the packet size information, which becomes superfluous in the UML model and further derived XML Schema implementations.

Figure 20: UML model of GMTI

The mappings between 4607v3 and UML are defined in the following table. The mappings to the complex elements Security and ExerciseIndicator are discussed further below.

Table 8: GMTI to UML mappings

4607 Field Name Form UML Type

Version ID Alphanumeric CharacterString

Packet Size I32 ---

Nationality Alphanumeric CharacterString

Packet Security – Classification E8 Security Packet Security – Class. System Alphanumeric

Packet Security – Code FL

Exercise Indicator E8 ExerciseIndicator

Platform ID Alphanumeric CharacterString

Job ID I32 Integer

8.4.1.1 GMTI Security Fields

The packet security fields have been mapped to the complex element Security as a natural aggregation of connected elements. The element is illustrated below.

Figure 21: UML model of Security

The security model aggregates two enumerations and a code list. Enumerations of type Enumeration are used to define closed enumerations, whereas code lists (type

CodeList) define extensible enumerations. The different types have been used here to illustrate this concept. It requires further discussion if enumerations of type

Enumeration or CodeList shall be used in future. An argument for type Enumeration

is the fact that we may have exhaustive lists: The classification and the classification system are defined as a list of expressions that shall be developed and published by each nation. Each nation is responsible for their own packet security handling codes. Thus, to any given point in time, we have a fixed list of codes. Any extension to this list would result in a new schema.

Figure 22: UML models of Security fields

In summary, we recommend to use enumerations of type Enumeration to make sure only those codes are used that are defined in (potentially distributed but in total exhaustive) lists. The CodeList has been used here to illustrate the issue. 8.4.1.2 GMTI ExerciseIndicator

The ExerciseIndicator indicates“whether the data contained in this packet is from a real-world military operation or from an exercise, and whether the data is real (originates from live-fly or other non-simulated operational sources), simulated (originates from target simulator sources), or synthesized (a mix of real and simulated data)”.

Figure 23: UML model of ExerciseIndicator

The ExerciseIndicator field is mapped to an Enumeration, because all allowed values are defined by 4607v3.

8.4.2 Mission

The Mission element maps information stored in the GMTI Mission Segment. This segment “includes information on the mission and flight plans, the type and

configuration of the platform, and the reference time”.

Figure 24: UML model of Mission

The UML model provides a direct mapping of all alphanumeric fields to

CharacterString. Only the reference time and the platform type have been mapped to the UML elements TM_Object and PlattformType to aggregate connected fields.

Table 9: Mission to UML mappings

4607 Field Name Form UML Type

Mission Plan Alphanumeric CharacterString

Flight Plan Alphanumeric CharacterString

Platform Type complex PlattformType

Platform Configuration Alphanumeric CharacterString

Reference Time - Year I16 TM_Object

Reference Time - Month I8 Reference Time - Day I8

OGC 11-113r1

Figure 44: UML model of TrackItem

9.4.4 TrackInformation

The TrackInformation class is an abstract super class to a number of specializations as illustrated below. Each specialization is a direct mapping of the STANAG 4676 class.

OGC 11-113r1

Figure 45: UML model of TrackInformation

Although there is the option to use the TrackLineageInformation type, which

aggregates LineageRelation data, the correct lineage of individual track items remains fuzzy. The LineageRelationType differentiates the types Parent, Child, and Sibling, without providing clear definitions of Child and Sibling or their differences

respectively. 9.4.5 Common

A number of elements are commonly used by the types described above. All elements represent direct or as close as possible direct mappings of the classes described in STANAG 4676.

OGC 11-113r1

Figure 46: UML common types in STANAG 4676

9.4.6 Enumerations

The STANAG 4676 model defines a number of enumerations, which have been mapped directly to OGC Enumeration types.

OGC 11-113r1

Figure 47: UML model of STANAG 4676 enumerations

10 Bookmarking Model

Originally it was one of the objectives of OWS8 to define a bookmark model that would allow tracing from the detected moving object back to the original data. This objective has been met to some extent, but requires further research in order to work more efficiently and to support higher level of interoperability.

The bookmark model is illustrated in the figure below. It contains the identifier of the track and any number of Dwell and Frame elements. Those elements are provided by reference, not inline.

OGC 11-113r1

Figure 48: UML model of Bookmark

Instance of type Bookmark can get inserted into a WFS-T and then be used for further investigations in cases where the reliability of detected targets is questionable.

Nevertheless, it should be mentioned that the OGC portfolio of services does not contain any specific video data access services. Though both types, Frame and Dwell

data could be served by WCS and it seems that a seamless integration into the OGC concept of operations might be possible, it requires further investigation to provide a solid and efficient model.

OGC 11-113r1

Bibliography

[ISO2010] ISO (2010): Final text of ISO/CD 19156, Geographic information - Observations and measurements, as sent to the ISO Central

Secretariat for issuing as Draft International Standard, ISO/TC 211 Geographic information/Geomatics, 2010

[MISB0903.2] MISB (2011): MISB Recommended Practice 0903.2: Video Moving Target Indicator Local Data Set, MISB, 2011

[MISB2010] MISB (2010): TRM 1007: Surfing the MISP. A quick guide to the Motion Imagery Standards Profile, MISB, 2010

[MISB0102.9] MISB (), 'MISB RP 0102.9: Security Metadata Universal and Local Sets for Digital Motion Imagery', MISB, Technical report, MISB. [MISB0604.2] MISB (2011), 'MISB STD 0604.2: Time Stamping and Transport of

Compressed Motion Imagery and Metadata', MISB, Technical report, MISB.

[MISB0601.4] MISB (2010), 'MISB STD 0601.4: UAS Datalink Local Metadata Set', MISB, Technical report, MISB.

[MISB0604.1] MISB (2010), 'MISB STD 0604.1: Time Stamping and Transport of Compressed Motion Imagery and Metadata', MISB, Technical report, MISB.

[MISB0902.1] MISB (2010), 'MISB STD 0902.1: Motion Imagery Sensor Minimum Metadata Set', MISB, Technical report, MISB. [MISB0801.2] MISB (2009), 'MISB EG 0801.2: Profile 1: Photogrammetry

Metadata Set for Digital Motion Imagery', MISB, Technical report, MISB, currently under review (April 2011).

[MISB0104.5] MISB (2006), 'MISB EG 0104.5: Predator UAV Basic Universal Metadata Set', MISB, Technical report, MISB.

[MISB0603] MISB (2006), 'MISB EG 0603: Common Time Reference for Digital Motion Imagery Using Coordinated Universal Time (UTC)', MISB, Technical report, MISB.

[NATO2008] NATO (2008): NATO Ground Moving Target Indicator Format (GMTIF) STANAG 4607 Implementation Guide, NATO, 2008 [NATO2010] NATO (2010): STANAG 4607 JAS (Edition 3) - NATO Ground

Moving Target Indicator (GMTI) Format

NATO, 2010

[NATO4609v3] NATO (2009) STANAG 4609 JAIS (Edition 3) - NATO Digital Motion Imagery Standard, NATO, 2009

[OGC2011] Simonis, I. (2011): OWS-8 Analysis of OGC Standards for

Supporting Mobile Object Processing Implementation (Engineering Report). OGC 11-108