Real-Time Search in Large Networks and Clouds

Sammanfattning: Networked systems, such as telecom networks and cloud infrastructures, hold and generate vast amounts of conguration and operational data, only a small portion of which is used today by management applications. The overall goal of this work is to make all this data available through a real-time search process named network search , where queries are invoked, without giving the location or the format of the data, similar to web search. Such a capability will simplify many management applications and enable new classes of realtime management solutions. The fundamental problems in network search relate to search in a vast and dynamic information space and the fact that the information is distributed across a very large system.The thesis contains several contributions towards engineering a network search system. We present a weakly-structured information model, which enables representation of heterogeneous network data, a keyword-based search language, which supports location- and schema-oblivious search queries, and a distributed search mechanism, which is based on an echo protocol and supports a range of matching and ranking options. The search is performed in a peer-to-peer fashion in a network of search nodes. Each search node maintains a local real-time database of locally sensed conguration and operational information. Many of the concepts we developed for network search are based on results from the elds of information retrieval, web search, and very large databases. The key feature of our solution is that the search process and the computation of the query results is performed on local data inside the network or the cloud. We have build a prototype of the system on a cloud testbed and developed applications that use network search functionality. The performance measurements suggest that it is feasible to engineer a network search system that processes queries at low latency and low overhead, and that can scale to a very large system in the order of 100,000 nodes.