ABSTRACT
Middle ware designed for use in Distributed Real-Time and Embedded (DRE) systems enable cost and development time reductions by providing simple communications abstractions and hiding operating system level networking API details from developers. While current middle ware technologies can hide many low level details, designers must provide a static configuration for the system’s underlying network in order to achieve required performance characteristics.
This has not been a problem for many types of DRE systems where the configuration of the system is relatively fixed from the factory (e.g., aircraft or naval vessels). However for truly open systems (i.e., systems where end users can add or subtract components at run time) the standard static network configuration approach cannot guarantee that required performance will be met because network resource demands are not fully known a priori. Open systems with stringent performance requirements need middle ware that can dynamically manage the underlying network configuration automatically in response to changing demands.
Fortunately, recent trends in networking have resulted in a wide variety of networking equipment that expose a standardized low-level interface to their configuration via the Open Flow protocol. In this paper we discuss how Open Flow can be leveraged by DRE middle ware to automatically provide performance guarantees. In order to make the discussion concrete, we describe the architecture of our prototype middle ware MIDAS as well as the details of one example network resource management strategy. We demonstrate the feasibility of our approach via performance assesment of a simple DRE application using our MIDAS and commerically available Open Flow hardware.
QUALITY OF SERVICE PARAMETERS
Fig. 1: Real-Time Publisher and Subscribers with QoS.
Figure 1 illustrates how MIDAS will match QoS between publishers, subscribers, and the underlying system. Subscriber A is admitted to the system because its required maxSep (20ms) is greater than or equal to the publisher’s (15ms). Additionally the middleware has determined it can guarantee the requested maximum latency. Subscriber B is not admitted because it requires a maxSep of 14ms which is smaller than the publisher’s.
APPROACH OVERVIEW
We achieve real-time guarantees on open networks built from COTS equipment by handing complete control over the network to the middleware via OpenFlow. Tight integration of the middleware with OpenFlow provides several benefits. First, it gives the middleware complete control over how data packets on the network are forwarded, prioritized, and ratelimited.
Second, many COTS switches can be made OpenFlow capable with a firmware update. This means that existing network deployments can be made OpenFlow capable. Third, in many OpenFlow switches all OpenFlow rule processing occurs at line rate. This means that the middleware can affect configuration changes in the network without any appreciable loss of network peformance.
MIDDLEWARE DESIGN
Fig. 2: Client Library.
The architecture of the client library is illustrated in Figure 2. If the application is a publisher, messages flow from the application to a local topic queue by way of the local topic manager. This allows the client library to perform a zero-copy transfer of data between publishers and subscriber that are running on the same host. Each local topic queue always has a special subscriber: the data-coding layer. The data-coding is responsible for serializing messages prior to transmission on the network.
Fig. 3: Global Resource Manager Architecture.
The GRM (Figure 3) is responsible for orchestrating all activity on the network to ensure that data is correctly propagated between publishers and subscibers. To accomplish this, the GRM must maintain configuration information about the network and implement the appropriate scheduling and network reconfiguration algorithms
Fig. 4: Example Network Graph.
Figure 4 illustrates a simple network graph. The network consists of two switches. These switches are connected via an uplink cable on each of their port 1. Each switch is connected to two clients (denoted by dotted circles).
FLOW SCHEDULING
A valid flow scheduler must perform three tasks: First, when a subscriber comes online and requests a subscription to an existing topic the flow scheduler must generate a candidate network configuration using OpenFlow configuration primitives. Second, the scheduler must analyze the new configuration and determine if it guarantees the timing constraints of all admitted flows plus the new one.
Third, if the new configuration is acceptable the scheduler must reconfigure the network carefully so no constraints of existing flows are violated during the reconfiguration. All three of these activities are non-trivial: Distributed scheduling is known to be NPHard if an optimal schedule is desired and there are no known exact schedulability tests for the general network setting. In light of these difficulties, we do not describe an optimal approach.
EXPERIMENTAL EVALUATION
Fig. 5: Network Layout.
We evalauted two aspects of MIDAS. First we wanted to see if the network scheduling used in the MIDAS improved the timing performance relative to that of a standard switch. Second, we wanted to see how robust the MIDAS timing guarantees are. In order to evaluate these two aspects we deployed the MIDAS on our OpenFlow test bench (Figure 5).
Fig. 7: MIDAS.
While it would be possible to reduce the message drop rate in the best effort setting by causing the senders to retransmit on failure, doing so would increase the effective latency of the message. Taking into account the 1 ms latency added by the JVM and TCP/IP stack, no messages violated the latency requirement when the MIDAS was managing the network configuration (All samples in Figure 7 are 4ms or less).
Fig. 8: Latency bounds for ST3 when PT1 and PT2 are malfunctioning.
Figure 8 contains the observed latencies when MIDAS was managing the network. Again, each point in the graph represents a single message. The y-axis is latency of that message, and its x-value is the moment the message was transmitted. Under MIDAS, no messages were dropped and all messages arrived earlier than their required latency bounds, 8ms.
RELATED WORK
To our knowledge the RTMB is the first example of a publish-subscribe middleware that uses OpenFlow to provide real-time guarantees on COTS Ethernet networks. Most research into middleware for distributed real-time systems can be divided into two categories. The first category involves research into how various CPU scheduling and determinism features can be used in middleware to effectively support the predictable execution of distributed tasks in a distributed environment. Examples of such work include TAO and the many middleware other real-time Corba middleware works such as FC-ORB, QuO.
CONCLUSION
The work in this paper represents a first step towards DRE middleware that can automatically provide strong real-time guarantees in a dynamic environment. We believe support for providing these guarantees in a dynamic environment will be critical for the success of newly emerging types of Cyber-Physical Systems. We described MIDAS, a prototype publish subscribe middleware which uses OpenFlow to manage the underlying network and provide guarantees for real-time QoS in order to illuminate some of the architectural and technical issues that apply to middleware designed for this environment.
Our initial evaluations showed that our prototype does enable more deterministic timing behavior. Even with a relatively high network load of 815 megabits per second all publishsubscribe network flows satisfied their millisecond-level timing requirements while on the normal Ethernet network latency constraints were violated and almost half the messages were dropped. The evaluations also showed that MIDAS’ ability to provide guarantees is robust: when we reconfigured two publishers as babbling idiots MIDAS prevented the remaining flow from deviating from its specified QoS constraints.
We believe the results in the paper encourage further research into OpenFlow and how it can be used to benefit DRE middleware. Such research topics include better scheduling and reconfiguration algorithms designed for use with the OpenFlow primitives, other types of QoS (beyond timing) that could benefit from full network control, and ways to detect and adapt to network faults dynamically.
Source: University of Pennsylvania
Authors: Andrew L. King | Sanjian Chen | Insup Lee
>> IoT based Real-Time Projects for B.E/B.Tech Students
>> 200+ IoT Led Projects for Final Year Students
>> IoT based Networking Projects for Engineering Students