Return to User Experiences

Return to the top

Maintenance of complex sensor network in smart industry using Data Distribution Service (DDS)

• Automotive
• Agriculture
• Manufacturing
• Oil & gas
• Smart Power grids & distribution
• Smart City

Figure 1: DDS - Sensor Maintenance

Return to the top

Here to cover a large and mixed audience let me touch on a few fundamentals quickly. Figure 2 presents the inforation in terms of the Automation Pyramid.¹⁾. Here is a picture of how industry automation works and the different levels of automation. Any industry starts with the field level and ends with the business logic. Now let us have a quick walk through on every level.

• In Industries like manufacturing, oil & gas, automotive, smart power grids & distribution, smart city and agriculture; “Automation” is no longer a choice

• To make the process intelligent and highly self reliant, the sensors play a major role in any industry; and thus it has the highest market share in automation (field layer)

• A brief about Sensor Network industry, a group of sensor are deployed along with actuators in the Field Layer. The field Layer collects data in real-time and exchange deterministic or non-deterministic data

  • The data collected by the central control system is used to take real-time decisions or provide insights, which can help improve production efficiency
  • Often, the granularity and continuity of data received from sensors impact the effectivity of the result significantly

• Sensor landscape - The Sensor Landscape is large. The following facts can help visualization of how big:

  • In large scale industries like manufacturing, oil and gas and power grids > 10000 to few million sensors
  • In automotive > 70 and < 2000 sensors per vehicle
  • In agriculture > 35 to < 5000 sensors per acre

Figure 2: Automation Pyramid

To begin with, the field level is the production flow that does the physical work and monitoring and at the Control the information from all then sensors is collected level and is used to make decision The supervisory layer that is the human layer or SCADA the information that is used to access the data to control the systems from one single location and plus it usually adds some graphical user interface for plant managers aspect. The Planning layer (Manufacturing Execution System (MES)) it monitors the entire manufacturing process in a plant or a factory The business logistics (Cloud) layer - it monitors or levels of from manufacturing to sales to purchase and to finance

So in this entire automation pyramid the complexity of managing the sensor nodes including the data transfer, control, configuration, and maintenance is the biggest challenge because this is where the data originates and every system reacts and responds based on the sensor data in every level of automation.

Also in comparison to the other automation levels, the field level has the highest market share aspect

So this is the populations nodes per industry and they are limited to.

Now you can imagine complexity of managing this bigger population of sensor nodes in terms of configuration, control and maintenance.

Figure 3: Sensor Network

Return to the top

These are the problems and use cases on managing the sensor nodes.

The use cases can range from

• Collecting low granularity data (high frequency– sub milliseconds range) for predictive maintenance from equipment's
• On-board real-time data in automotive, agriculture vehicle and smart cities
• Long term bulk data for creating predictive models of equipment maintenance such as weather patterns for agriculture , production planning in manufacturing, etc.

For this variety of use case scenarios, the problem statement, the major challenges in a Sensor Network are

• Scalability – since any sensor network is continuously growing,it needs an easier addition of new nodes in tot he existing network with minimal downtime and lower configuration overhead
• Replacement – any faulty Sensor Node in a running sensor network should be easily replaceable with no downtime
• Stored Data Redundancy – the data coming from different sensors should be maintained having multiple copies of the data should be distributed in order to assure that the network data fail safe
• Data Integrity and granularity – low granularity and continuity of the sensor data should be highly maintained without compromising the performance
• Data Security - The data privacy and operational safety must be assured without compromising the performance

Figure 4: Sensor network -Use case and Challenges

Return to the top

The strengths of the Data Distribution Service (DDS) standards and how they differ from other communication standards. DDS is an interoperable, platform independent, secure, scalable communication standard. It is our opinion that DDS is the right choice for these large scalable networks.

The following discussion is on the DDS features and concepts and how they allow to they align to these challenges.

• Data Distribution Service (DDS) is a Middleware protocol and Application Programming Interface (API) standard for Data-Centric connectivity. What data-sentricity means is that DDS can directly work on the state of the object in any DDS Node rather than using metadata implying actions on the data data or information about the changes the state.

• DDS conceptually sees a local store of data called the “global data space”. This physically implemented in a distributed way that are active in every DDS Node and prevents the need for centralized brokers. Consequently the Data Model (DM) itself resides in the “global data space”.

  • Distributed storage of information model

• Application can work on higher level instead worrying about

  • What is information?
  • How to format the information?
  • Keeping track of the connectivity of whether it is dead or alive
  • Checking if the last message sent or received, basically an acknowledgement
  • Decreases the application complexity the user application and the middleware are loosely coupled

• The cotrast of DDS as compared with other communication standards.

  • Location transparent –hides the Network Topology. DDS supports the Unicast and Multicast discovery (see RFC1112 - Host Extensions for IP Multicasting). If the DDS nodes are connected to the same hub or managed switch and the network topology does not cross the managed switch, and they use a default configuration, the nodes find each other using a Multicast form of discovery itself
  • Less or no deployment configuration since the applications are familiar with the global data space, configuration is not required because if nodes need the data is required they can access directly from the global data space rather than requesting the peer nodes directly.
  • Scalable from sensor to cloud, from the automation pyramid, the DDS Data bus will sit at any level of the Automation Pyramid and provide the data to the requested level. And here IT systems will not provide their own data system so the real-time and IT systems do not diverge and the system can be scaled
  • Quality of Service (QoS) Policies, DDS allows the applications define and share the user data with a controlled Quality of Service such as performance, scalability, reliability, durability, and security
  • DDS + Time-Sensitive Networking (TSN) can replace legacy field-bus over the years. TSN is a set of IEEE standards that aim to provide the time sensitive data transfusion over the Ethernet networks which works in Open Systems Interconnection (OSI) Model layers 1 and 2 (i.e., Transport Layer and Physical Layer) and DDS works in layers 3 to 6 (i.e., Session Layer, Network Layer, Data Link Layer, Presentation Layer) which means that QoS settings in the application take care of the data to use a real-time model in higher OSI layers and TSN can take care of timing at the lower OSI levels ensures that timing is guaranteed on the wire.

Figure 5: Data Distribution Service (DDS) Strengths

Return to the top

The Predictive Maintenance is a strategy using analytical and statistical techniques on past data to perform extrapolations about the future. The strategy is referred to as Predictive Analytics. The analytical methods are applied to maintenance issues fo predicting potential failures. The methods use patterns and anomalies, but is usually deployed when probability of imminent failure is high. This helps in allocating limited resources, maximizing device or equipment uptime, enhancing quality and supply chain processes. The end goal is to improve the overall satisfaction for all the stakeholders involved.

• Why predictive maintenance?

  • To avoid higher down times and improve production efficiency
  • To find equipment abnormalities
  • To clean and restore the components before performance degrades
  • to avoid total break-down or system failure in any industry

• For this to be effective, the condition of the equipment is monitored using arrays of sensors for collecting diagnostic data in real-time

• For continuous manufacturing any breakdowns that results in manufacturing slowdowns or shutdowns in business and service operation is costly. Therefore, intelligent maintenance (predictive and preventive) is an urgently need. For example, maintenance costs are estimated to range between 15%a and 40% of total production for any type of industry.

In Figure 6, the flow of data from sensors to Analytics is represented.

1. Data Sources - Sensors, Machines, Batch or stream data
2. Data Ingest and Storage - Real-time data ingest, Big-Data platform
3. Data Processing - Real-time processing, in-memory processing
4. Machine Learning - Machine Learning, Supervised / Unsupervised Learning
5. Analytics - Data visualization, SQL Analytics

Figure 6: Use case: Predictive maintenance

Return to the top

Within the chemical industry there is a need to mix chemicals in a controlled environment to assure the safety of the process and to avoid waste of materials and time (in other words, the efficiency of the process).

The overall process is to mix various and often expensive chemicals in multiple containers using an agitator. The agitators are moved in a prescribed rate of speed to ensure that the chemicals are thoroughly mixed and at a rate that allows the chemical reactions to work as intended. When a chemical process is initiated and a fault occurs, the initial chemical resources are wasted. Failures in this process are costly in terms of raw resources, loss of end-product and time. Potentially, in some processes the containers, the agitators, sensors themselves might need to be replaced. Therefore, any changes in the state of the chemical rate and speed of the chemical mixing might result in damage to the container, the agitators or drivetrain might idle the plant for days, weeks or months greatly damaging revenues. To mitigate this, the condition of the motor and the stirrer are monitored continuously in real-time. The data generated is used within the predictive models applied to all the automation layers to identify potential failures and apply the correct maintenance or repairs before the start of a new production. The data granularity, integrity, and the prevention of dataloss are essential for the predictive models to accurately predict failures and to recommend maintenance or replacement of parts.

As a result of all these factors, the Container Agitator in Chemical Industry is a good use case for DDS in helping maintenance operations.

The Challenge is to ensure the reliability of the chemical mixing process by reducing the risk of failure of components (i.e., containers, agitators, motors and sensors) reused within the chemical mixing process . The components need to maintained or replaced, when needed, in a timely way without jeopardizing the overall efficiency of the endeavor or adding unnecessary replacement overhead.

• Components:

• Containers

  • Volume of about – 2000 to 2500 cubic ft
  • Includes an agitator connected to an electric motor
• Agitators (i.e., stirrers) used to mix the chemicals at a prescribed rate to ensure a proper chemical reaction throughout the container. The Agitators are equipped with performance sensors that provide information to the operators
• Motor speed used to drive the agitators at the prescribed speed

• Monitoring

  • Live real-time sensor readings
  • Using predicative models at all levels of the automation pyramid (see Figure 2)
  • By operators making decisions about the condition of agitators, the motors and the drivetrains trying to identify potential failures before the start of a new production run

• Any changes in the state of the chemical rate and speed of the chemical mixing might result in damage to end products, containers, the agitators or drivetrains might idle the plant for days, weeks or months greatly damaging revenues.

• A network, where the data from the sensor can be collected at milli-seconds granularity without any dataloss, that can support real-time data exploitation for use in predictive models of the chemical mixing process initiate required maintenance be fore starting new production runs.

Figure 7: Use case: Predictive maintenance, Container Agitator in Chemical Industry

Return to the top

A problem confronting cities as they try to move and adapt to the Smart City Sensor driven concepts and initiatives is how to scale the Sensor Network to not just include more facilities but how to expand it to cover more city services within the sensor network ecosystem. For example, a city might need to expand its sensor network to add 1,000 homes a year, but they also want to add sensors for water, sewage, lighting, Electric Grid, natural gas, and communication grids (i.e., cable television, Phone and internet). In addition to these “traditional” city services, there is a desire to add traffic monitoring, emergency services, transit, parking, recreational facility usage and security services to the ecosystem.

As a result of all these factors, the Smart City scaling is good use case for Data Distribution Service (DDS) in helping the continuous scaling needs spanning many kinds of sensors and spanning years operation which also highlights the needs for interoperability.

• Since the ecosystems in this scenario are in a constant flux as the system grows to cover more functional domains, rapid deployment, scaling-up, maintenance of the Sensor Network is a real challenge

• Expanding the network can not add higher down-time or deployment time due to high configuration over head during extension, replacement of the nodes or application update in the network

• Flexibility of adding new sensor nodes while expanding the Sensor Network
• Replacing faulty sensor in the existing sensor network with new different vendor sensor nodes
• Less configuration overhead
• Use a redundant distributed model instead of centralized mode with single points of failure

Figure 8: Use case: Scalability Smart City

Return to the top

Data Distribution Service (DDS) addresses the following needs:

• Real-time

  • Fast, higher frequency exchange of data in sub-milliseconds
  • Data freshness and ensures no data corruption

• Redundancy

  • Multiple DDS Nodes can publish information about a particular observation
  • Multiple DDS Nodes can subscribe to information about a particular observation
  • Publisher / Subscriber model of communication where each DDS Node can publish collected raw data and where multiple consumers ( edge devices, analytics platforms) can collect the data. (See Figure 2). Subscribers to the published data can be operators monitoring the edge devices or can be anyone up the chain to the plant managers or owners that want to review the data from the edge to the platform analytics. This reduces the communication overhead of requesting the data from the individual nodes itself to all the other nodes in the network. This helps keep the processing power requirement and the costs on the node more global resulting in multiple data consumers on mobile phones, tablets or laptops allowing predictive models to run at all layers of the automation pyramid.

• Scalability

  • OMG: Extensible and Dynamic Topic Types for DDS (DDS-XTypes) provides a mechanism to add / remove /modify the Data Model (DM) of the each Sensor Node at runtime without the need for reconfiguring the entire network and the communications start in the middleware which automatically collects the configuration data of the configuration data from all the nodes creating a snapshot database from the latest live data from the network. The data can be accessed from the user application using a subscriber using a standard Application Programming Interface (API) Data Distribution Service (DDS). This allows the user application to remain agnostic to any node replacement in the network and the scaling of the network by updating the user application and processing the data without any coding of the application. And the sensor nodes can join or leave the network at any point in time. This means the nodes can dynamically discover that an update is needs without the need for many Domain Name Server (DNS).
  • DDS makes it easier for adding a node and deploying distributed application as the communication stack automatically collects data from all publishing nodes and creates a snapshot database of the latest live data from the network, which can be accessed from user application using standard APIs

• Maintainability

  • Strong Quality of Service (QoS) Policies ensure data is shared on-time, data freshness, and there is no data corruption in the collected dataset or group that is is critical for maintenance and to support predictive models. For maintenance use cases there is a requirement to get bulk data and low granularity data for updating predictive models and the QoS settings which highly enable the data integrity and making the data more latency sensitive.
  • Publisher / Subscriber
  • Extensible and Dynamic Topic Types for DDS (DDS-XTypes) allows for the integration of older and newer message content allowing for heterogeneous versions of sensors within the same ecosystem.

• Integratability

  • DDS Interoperability Wire Protocol (DDSI-RTPS) allows for data to be transferred in efficient, binary, cross vendor standardized messages. XTypes allows for the addition, remove, or modify elements in the data model of each sensor node during the runtime. This helps balance the tradeoffs of efficiency, vulnerability and compatibility of the sensor nodes over time. It provides a fast, higher frequency exchange of data in the range of sub-millisecond allowing the nodes to publish the raw data in real-time. It can run over Multicast. Data can be transported in frames or in streams using User Datagram Protocol (UDP) or Internet Protocol (IP). In the case of manufacturing use case, there is a need to get lower granular data and in Smart City use case, there is a need to get data in “more” real-time.
  • The users in multiple layers can access the data in the way they need allowing them to associate the data they need for background analytics and other roles such as plant managers, business users, or end customers require more customized data views. This customization of views is part of the Extensible and Dynamic Topic Types for DDS (DDS-XTypes) Specification.
  • The standard DDS Model has extended its usage to allow Web Object Model to monitor and control the Sensor Nodes from the live dashboard using the REST API defined in the Web-Enabled DDS (DDS-WEB) Specification for the managers sitting in the cloud allowing them to access the data using the|Web Enabled DDS API to access the Global Dataspace or the databus.
  • Interface Definition Language (IDL) allows for data to be expressed in language neutral way (not specific to C / C++ / C# / Java / Javascript, etc.)

• Security

  • DDS Security (DDS-SECURITY) allows to define and enforce configurations to handle the sensor data to be encrypted or transmitted confidentially or authenticity for reading the data without any additional overhead. In both the chemical and the Smart City use cases, security is a high priority. DDS-Security allows for the import and definition the configuration to handle the sensitive data to be more inclusive or transmitted confidently with authentication required to read the data without any overhead, while consuming data from any layer within the automation pyramid (See Figure 2). DDS-Security also supports multicast allowing for more efficient data transfer and without compromising performance. Using other security transport protocols like Transport layer security (TLS) or Datagram Transport Layer Security (DTLS) which do not have multicast support.

These are the features of DDS that are applicable in addressing the needs of the Chemical Mixing and the Smart City Use Cases previously discussed.

Figure 9: DDS Addresses the Needs

Return to the top

The DDS Enterprise Level Solution Architecture is depicted in Figure 10 is applicable to both the Chemical Manufacturing and the Smart City uses case already discussed. The architecture is mapped against the Five Layers of the Automation Pyramid (See Figure 2).

This is the solution architecture that has been proposed for these kind of complex use cases.

In the future, DDS will be the natural choice for the large scale network system in the real world.

Figure 10: DDS Enterprise level Solution Architecture