Level 0 Fusion. Level 0 fusion functions operate on sensor data to produce (extract) features using inference models. For imaging sensors, e.g., Electro Optical (EO), Infrared (IR), and LIDAR, object classification (labeling) is inferable using inference models developed by Deep Learning (DL) processes trained in the mission planning phase on targets and environments representative of those expected in the mission area, e.g., Air Tasking Order (ATO) area. The Level 0 fusion outputs are object/entity detections, image bounding boxes within the image frame(s), and broad classifications, e.g., “building”, “vehicle”. For the Electro-Magnetic (EM) sensors, sensor platforms may reconnoiter near to the targets to detect and triangulate intentional and non-intentional EM emissions from, e.g., Low Probability of Intercept (LPI) emissions, motor generators, and computer networks. The Level 0 fusion outputs are estimated waveform parameters, broad classifications, and Lines of Bearing (LOB). DF Level 0 often includes an ontology and knowledge base of a priori data based on in-depth exploitation of all knowledge derived from an Intelligence Preparation of the Operational Environment (IPOE) including target and confounder Characteristics and Performance (C&P), Order-of-Battle (OB), terrain data, ambient objects, man-made structures, avoidance areas, and other GEOspatial INTelligence (GEOINT). This data is structured into a formal ontology to support processing by DF functions.

Level 1 Fusion. Level 1 DF processes estimate properties of Events and Objects where Position, Identification, Classification, and Composition properties (PICC) are developed from extracted features using location and Bayesian inference algorithms. Even for bearings-only sensors that are air or space borne, object position may be estimable from sensor pointing angle intersection with the terrain or other GEOINT. As part of Level 1 fusion, Classification processes estimate that an object is a type of a class, e.g., SA-2 missile launcher, Spoon Rest acquisition radar. Classification is about “what”. Identification says the object is a member-of or belongs-to an organization. Identification is about “who”, involving estimation of attribution/ownership. Organization here is in a general sense and can include not just nationalities but also alliances or subdivisions or groupings thereof, e.g., NATO, or non-state actors such as drug cartels.

Level 2 Fusion. Level 2 fusion is concerned about linkages including objects and their parts, sometimes called Composition. An example is the Composition of an SA-2 SAM site consisting of parts, such as missile launchers, radars, generators, command and control trailers, arranged in some standard layouts. This can be represented as a graph where the nodes are the objects and the edges are the type of relationships between them, e.g., whole-part, distance-from. Similarly, the a priori data, structured into a formal ontology, can be formed into graphs. As well, graphical-based analysis views the fusion hypotheses as graphs with nodal and edge-wise relationships.  Directed Attributed Relational Graphs (DARG) and associated graph association methods have high potential for this type of Level 2 DF. The DARG analytical processes involve representation of layered data, cross-layer (graph) association, and associated evidence-to-situation information by graph-matching, In recent work, an inexact subgraph matching algorithm was developed as a variation of the subgraph isomorphism approach for situation assessment. DF enhances this procedure to represent inaccurate observations or data estimates, and inaccurate structural representations of a state of interest, thus accounting for the various uncertainties in multilayered data. Various probabilistic and other uncertainty representations, transformations between representations, and methods for establishing similarities between representations have been assessed on prior programs.

Level 3 Fusion.  DF Level 3 is concerned with prediction.  In this example, this means inferring the Tactics, Techniques, and Procedures (TTP) being used by the adversary and then the adversary’s possible and probable Courses of Action (CoA).

Level 4 Fusion. DF Level 4 is concerned with Information-optimizing resource management. Each C-ATA UAV’s DF estimation process is coupled with a sensor/source management process, an example of which is called Information Based Sensor/Source Manager (IBSM). This integrated approach to sensor management is based on an embedded scheme for trading off mission objectives with optimal sensor data acquisition, and is integrated into a complete the real-time DF processes.