The aim of the evaluation is two-fold - to demonstrate the OpenKnowledge system in action, illustrating that all parts of the system are capable of working cohesively in the desired manner; - to demonstrate that the technology provided by OpenKnowledge can make a positive difference to the management of such disaster scenarios.
We have the following evaluation hypotheses:General hypothesis: There exists some combination of interaction model sharing, ontology matching and trust assessment capable in a highly distributed and peer to peer architecture of rivalling the emergency response performance we would expect in a traditional centralised planning model; furthermore, the performance of the peer-to-peer system is more robust in the presence of failure of components.
This general hypothesis is insufficiently precise to be tested directly so we pursue a more specific hypothesis:
Specific hypothesis: In simulations using the OpenKnowledge kernel for coordination; Trentino GIS/flood data to represent world state; and locational information about people and resources, there exists some combination of OpenKnowledge interaction model sharing, ontology matching and trust assessment capable with a decentralised peer-to-peer system of coordination (relying on the ontology matching and trust mechanisms) of moving similar numbers of people to safe sites during a simulated flood event as we observe in simulations with a centralised system of coordination; furthermore the performance of the peertopeer system is more robust to the failure of sensors and breakdown of communication channels.
The evaluation was based on a simulation of an actual disaster. A simulator was built to replace the role of the real world: that is, it performed the following roles:
- keeping track of the state of the flood over time, through a sub-component, the flood sub-simulator, which simulates the flood and passes the information to the main simulator at each time step;
- keeping track of the locations of people/objects in the simulation;
- providing 'sensory experience' for simulated peers (e.g., people, sensors) by providing information of what they can see and feel in their location (e.g., other people, flood level, etc.)
- determining if peers can perform their desired actions in the world: for example, a peer can only move from A to B if there is a road between them and that road is not flooded. If it is possible, updating the state accordingly; if it is not possible, informing the peer of this (and thereby increasing their knowledge of the world).
- reporting the full state of the world to a visualiser at each time step, so that observers can see how the simulation is progressing.
The simulator uses actual Trentino GIS/flood data and invented but representative location of people and resources. The simulator is implemented as a peer in the network.
Peers within the simulation communicate with one another via the OpenKnowledge system just as they would during a real disaster. No peer has any special privileges that they would not have in the real world: for example, a coordination centre cannot see the full state of the world from the simulator; they have to reconstruct it from reports from sensors and peers on the ground, just as they would in real life.
The Flooding Scenario
We are concerned with the attempts of moving peers (such as firefighters, police and civilians) to reach a given destination (such as a safe area). The need to move and the chosen destination are determined through a command from an Emergency Controller: the top-level emergency-response personnel who are trying to manage the evacuation of people from the flooding area. The Emergency Controller determines who should move and where by analysing the situation as presented to it by the emergency monitoring system, which is getting its information through communicating water level sensors and weather forecasting services. When the moving peers have a destination, they decide how to get there by determining, through communication with a civil protection unit, which roads are blocked (the civil protection unit is communicating with reporters to find this out), and then providing the destination and the blocked roads to a route service, who will give them a suitable route.
The interaction of the peers is illustrated in the figure below:
We investigate two situations: firstly, a centralised approach, in order to demonstrate that the OpenKnowledge system can facilitate this as well or better than other systems designed for a centralised approach. Secondly, a decentralised approach, in order to demonstrate that the OpenKnowledge system can allow interaction to proceed smoothly even when a centralised approach is not possible.
An example, using our peers, is the presence or absence of the Civil Protection Unit. Its role is to poll all the reporters and keep a full picture of the states of the road, so that it can respond to any queries from moving peers. This is a centralised model. However, if the Civil Protection Unit breaks down, becomes disconnected or swamped, or for some other reason is unreachable, then, in the centralised model, the moving peers cannot get any information at all about the state of the roads. However, the flexibility of OpenKnowledge allows us to move seamlessly to a decentralised model, where the moving peers can directly query the reporter peers. There is no structural change needed for this: the moving peers simply uses the discovery service to locate these peers and a suitable IM for interaction.
Demonstrating the usefulness of OpenKnowledge in both a centralised and decentralised model is crucial in demonstrating what we perceive to be the benefit of using OpenKnowledge in such a scenario. Emergency responses tend to be fairly hierarchical and centralised by nature, and it is usually important to have a high-level of control. We are not proposing that this centralised control be completely removed: this could lead to chaos! However, the drawback of the centralised approach is that it is very brittle: if one key link is lost then the whole system can collapse. This is where the OpenKnowledge approach has a fundamental advantage over other approaches that facilitate the centralised model: it allows this seamless change from a centralised to decentralised system, meaning that the actors are empowered to continue interacting to the best of their ability even where the centralised plan has failed.
Full details of the scenario, together with a complete evaluation, can be found in Deliverable 6.8.
Back to Testbeds