DyKnow A Stream-Based Knowledge Processing Middleware Framework

Sammanfattning: As robotic systems become more and more advanced the need to integrate existing deliberative functionalities such as chronicle recognition, motion planning, task planning, and execution monitoring increases. To integrate such functionalities into a coherent system it is necessary to reconcile the different formalisms used by the functionalities to represent information and knowledge about the world. To construct and integrate these representations and maintain a correlation between them and the environment it is necessary to extract information and knowledge from data collected by sensors. However, deliberative functionalities tend to assume symbolic and crisp knowledge about the current state of the world while the information extracted from sensors often is noisy and incomplete quantitative data on a much lower level of abstraction. There is a wide gap between the information about the world normally acquired through sensing and the information that is assumed to be available for reasoning about the world.As physical autonomous systems grow in scope and complexity, bridging the gap in an ad-hoc manner becomes impractical and inefficient. Instead a principled and systematic approach to closing the sensereasoning gap is needed. At the same time, a systematic solution has to be sufficiently flexible to accommodate a wide range of components with highly varying demands. We therefore introduce the concept of knowledge processing middleware for a principled and systematic software framework for bridging the gap between sensing and reasoning in a physical agent. A set of requirements that all such middleware should satisfy is also described.A stream-based knowledge processing middleware framework called DyKnow is then presented. Due to the need for incremental refinement of information at different levels of abstraction, computations and processes within the stream-based knowledge processing framework are modeled as active and sustained knowledge processes working on and producing streams. DyKnow supports the generation of partial and context dependent stream-based representations of past, current, and potential future states at many levels of abstraction in a timely manner.To show the versatility and utility of DyKnow two symbolic reasoning engines are integrated into Dy-Know. The first reasoning engine is a metric temporal logical progression engine. Its integration is made possible by extending DyKnow with a state generation mechanism to generate state sequences over which temporal logical formulas can be progressed. The second reasoning engine is a chronicle recognition engine for recognizing complex events such as traffic situations. The integration is facilitated by extending DyKnow with support for anchoring symbolic object identifiers to sensor data in order to collect information about physical objects using the available sensors. By integrating these reasoning engines into DyKnow, they can be used by any knowledge processing application. Each integration therefore extends the capability of DyKnow and increases its applicability.To show that DyKnow also has a potential for multi-agent knowledge processing, an extension is presented which allows agents to federate parts of their local DyKnow instances to share information and knowledge.Finally, it is shown how DyKnow provides support for the functionalities on the different levels in the JDL Data Fusion Model, which is the de facto standard functional model for fusion applications. The focus is not on individual fusion techniques, but rather on an infrastructure that permits the use of many different fusion techniques in a unified framework.The main conclusion of this thesis is that the DyKnow knowledge processing middleware framework provides appropriate support for bridging the sense-reasoning gap in a physical agent. This conclusion is drawn from the fact that DyKnow has successfully been used to integrate different reasoning engines into complex unmanned aerial vehicle (UAV) applications and that it satisfies all the stated requirements for knowledge processing middleware to a significant degree.