Monitoring a real-time distributed system for fault detection and identification is an extremely challenging problem. Faults may manifest themselves at a different node than where the actual error occurred, and may also be dependent on a particular sequencing of events and thus not easily reproducible. In addition the fault may be the loss of connectivity to some segment of the system, rendering a monitoring agent running in one segment of the system unable to communicate with other segments.