Meshdynamics
The Abstracted Network for Emerging Industrial Internet of Things
 The Abstracted Network for Enterprises and the Industrial Internet of Things.
 

Introduction


Widespread adoption of TCI/IP protocols over the last two decades appears on the surface to have created a lingua franca for computer networking. And with the emergence of IPv6 removing the addressing restrictions of earlier versions, it would appear that now every device in the world may easily be connected with a common protocol.

But three emerging factors are requiring a fresh look at this worldview. The first is the coming wave of sensors, actuators, and devices making up the Internet of Things (IOT). Although not yet widely recognized, it is beginning to be understood that a majority of these devices will be too small, too cheap, too dumb, and too copious to run the hegemonic IPv6 protocol. Instead, much simpler protocols will predominate (see below), which must somehow be incorporated into the IP networks of Enterprises and the Internet.

At the other end of the scale from these tiny devices are huge Enterprise networks, increasing movingly to the cloud for computing and communication resources. An important requirement of these Enterprises is the capacity to manage, control, and tune their networks using a variety of Software Defined Networking (SDN) technologies and protocols. These depend on computing resource at the edges of the network to manage the interactions.

The third factor is somewhat hidden from view, but will become more important as Enterprises seek to truly incorporate all the data from every kind of “thing.” There are millions of legacy machine-to-machine (M2M) networks operating everything from robotic assembly lines to process control facilities. These run on idiosyncratic older protocols that are often implemented on purpose-built Programmable Logic Controllers (PLC). Although these M2M networks have long existed as independent islands, increasingly Enterprises wish to pull these into a broader IoT strategy. But these legacy networks were often designed with no thought to wider communication and management and demand local control and response.

Concomitantly, any solution addressing these three factors must also be exceptionally tolerant of disruption, mobility and change. Enterprises placing “more eggs in one basket” will require that all data operation continue despite perturbations.


1. The Abstracted Network

   

Fig 1. The Abstracted Network emulates machine specific protocols within a (Scalable) SDN aware Fabric Enlarge

The solution to these seemingly contradictory requirements is an idea that is simple in concept, but demanding in implementation: The Abstracted Network. Conceptually, the MeshDynamics Abstracted Network replaces topologically independent networks with a single network made up of a new class of devices called propagators.

As seen in Figure 1 , traditional networks are still often separate today based on whether they are handling human-oriented traffic (smart phones, tablets, computers, etc.) or M2M traffic (sensors, actuators, robots, etc.). In some cases, such as shop floor environments, legacy protocols and response requirements may even have kept these devices as “islands” or “silos” disconnected from the rest of the Enterprise. These isolated networks have remained despite the rise of IP because of simplicity of the end devices or their peculiar communications and control requirements.

Besides the obvious networking inefficiencies, these isolated networks may not be managed and tuned via SDN techniques. More importantly, potentially important data flows and status indications are hidden from the primary Big Data servers managing the rest of the Enterprise’s business. Indeed, the potential power of the publish/discover/subscribe model for integrating IOT and legacy traffic into business intelligence processes is the leading benefit of the Abstracted Network concept.

The Abstracted Network model is based on MeshDynamics propagator node communications devices (blue rectangles in Figure 1) that take advantage of processing power and memory density advancements as well as exploiting emerging Open Source operating systems and communications protocols. Propagator nodes emulate the previously separate networks’ protocols, timing, and control interactions such that legacy and IOT devices need not incorporate higher level-protocols themselves but may still become part of the overall Enterprise operational structure.

The underlying technology making MeshDynamics Abstracted Networks possible is a sophisticated real-time database that creates a model of the logical network connections and requirements.  A variety of internal processes monitor and update this model based on network traffic flows to create an efficient virtual network structure no matter what the physical topology may be. This includes such details as latency, protocol translation, multicast pruning and forwarding, and even control loop management. Critically, as this Abstracted Network model is autonomously constructed, it may then be further refined, restructured, monitored, and tuned by the Enterprise using SDN techniques and protocols.

This is accomplished in two ways.  First, internal processes maintain deterministic performance between different network applications through a variety of scheduling and modeling algorithms within the propagator node network itself. This is part of the Meshdynamics mesh network framework. Secondly, applications agents onboard each of the propagator nodes may participate with Enterprise-wide SDN messaging to optimize network performance based on rules and other parameters. Physically, propagator nodes may incorporate a wide variety of wired and wireless interfaces to facilitate connection to many legacy and IOT devices As discussed below for the simplest IOT devices, propagator nodes permit publish/subscribe data flows even for the simplest or proprietary devices. Similarly, a variety of network connections may be used as links between propagator nodes themselves, the traditional Internet, and Big Data integrators. More


2. Disruption tolerance, mobility, and changeability

                      
              Fig. 2  Disruption tolerance between nodes is provided an adjacency database and routing table exchanges

With “more eggs in one basket”, survivability and non-stop operation of the Abstracted Network becomes vitally important. But because the interconnected propagators maintain databases for both their own topology and for the logical network configuration, disruption tolerance is provided inherently. As a first level of disruption tolerance, the propagator network maintains awareness of adjacent nodes and alternate paths, automatically making use of the best links as primary connections and maintaining awareness of any changes or failures. This allows near-instantaneous re-routing around failures. More.

 A by-product of this capability, propagator nodes may be in motion relative to one another, and to end devices. More.

Because the internal database of node adjacencies and link qualities is constantly updated, individual nodes may shift to new connections as old ones are fading, maintaining network operation. Disruption tolerance also extends to individual applications and families of connected devices.

The applications agents in each propagator node may be equipped with software that actually performs many functions on behalf of connected devices such as data collection, alarms and status, control loop management, even spoofing of network acknowledgement and requests. This may be maintained through a complete failure of the upstream connection and interruption of round-trip communications – the propagator node keeps the local operation intact until the upstream connection is restored.

The capabilities of the propagator node and the Abstracted Network Concept will be especially important for integrating the coming explosion of Internet of Things devices and their terse M2M communications schemes, discussed  below.


3. Publish, Subscribe and Discovery for the Edge

 

Fig 3: Applications Subscribe to Aggregation, Pruning Services and other Real Time Pub/Sub M2M “Shuttles”  Enlarge

The Abstracted Networks concept has one basic premise: If simple edge  devices aren't capable of deep protocol intelligence, it must reside somewhere and operate on behalf of the end devices. The major elements of that somewhere are the Level II Propagator nodes   As introduced above, these are like familiar networking equipment such as routers and access points, but they operate in a different way in with IOT and legacy devices. To explore the functions of the propagator nodes in more detail, we can examine their function in an IOT application – but of course this may be extended to legacy and IP networking as well.

Propagators listen for data "chirping" from any device. Based on a simple set of “markers” in the chirps (described below), propagator nodes decide how to broadcast these chirps to other propagator nodes and on to the higher-level Integrator (Big Data) subscribers. In order to scale to the immense size of the Internet of Things, these propagator nodes must be capable of a great deal of discovery. They must recognize other propagator nodes within range, set up simple routing tables of adjacencies, and discover likely paths to the appropriate integrators.  (The key is building a logical tree-like topology from physically meshed propagators. The topology algorithms have been tested and described here)

Typical applications on Propagator nodes serve as aggregation, pruning and If-this-then-that filters. They organically form “hubs”, similar to bus terminals along commonly used routes. M2M "Chirps" passing from-and-to end devices may be combined with other traffic for forwarding. Applications provide this networking on behalf of devices and integrators at levels "above" and "below" themselves within their hierarchy in a tree based framework. More.

Any of the standard networking protocols may be used, and propagator nodes will perform important translation functions between different networks (Bluetooth to  Wi-Fi, for example), essentially creating small data “flows” from M2M chirp data streams. Other trusted applications and agents, many residing inside the Propagators, coordinate the function and control of dumb small, cheap, and copious IOT devices through Software Defined Networking (SDN) paradigms for the edge.


4. Autonomous Industrial IOT Network Implementation (SPAWAR)



Fig 4: Disruption Tolerant, Semi-autonomous networks (SPAWAR)

MeshDynamics has been developing an open-source propagator platform for disruption tolerant networking for the US Navy and US Department of Energy. Propagator nodes support User Space Application Layer within an OpenWRT architecture for deep packet inspection, SDN based routing, Video, IFTTT (conditional “If This Then That” rules), etc. These propagator nodes provide autonomous, robust machine control with no assurance of internet connectivity through the built-in applications agents.

Nearly everything described above takes place autonomously and automatically within the Disruption Tolerant Mesh Network. Propagators and their applications even route around failures of links or nodes and deal with issues created by mobility of network elements. But some of the most important capabilities of propagators are distributed application intelligence agents that can allow higher level functions to “tune” the propagator network as part of an overall publication/discover/subscribe infrastructure and/or create application instantiations (e.g. machine controllers) within the nodes themselves. These permit connected devices to continue to operate when the broader network connection is lost. [More]

The end result is a Publish/Subscribe (Pub Sub) network that can be extended from Big Data servers all the way to the edge of the network while still maintaining a degree of responsive local autonomy. A variety of standards- based SDN protocols may be implemented on the distributed applications agents.

"MeshDynamics Scalable and Open Pub Sub enables us to rapidly integrate with Enterprise Class, OMG (Object Management Group)-approved, industry- standard messaging systems from RTI (Real-Time Innovations), PRISMTECH, OpenDDS, and others to provide assured real time end to end performance, even if we scale to billions of devices at the edge.”

Curtis Wright, Sr. Research Systems Engineer, Space and Navy Warfare Center.


5. "Smart Enterprise" Network Implementation (SHARP)

 


Fig. 5: “Smart” Networks supporting Real time Situation Assessment (SHARP)


Along with the ability to manage change and disruption, the unique multi- radio architecture for propagator nodes allows the extension of the network along many “hops” (node-to-node links) without additional wired connections. This “string of pearls” capability allows large outdoor and indoor "tunnels" to be connected as well as extending services to new remote locations at the next "edge".

These extended propagator node links are simply treated as another element of the Abstracted Network and may be managed by Enterprise SDN tools and techniques. The Meshdynamics mesh network fabric observes and accommodates the additional delays experienced by traffic traversing long strings of propagator nodes with minimal additional latency or jitter compared to other network connections. Propagator nodes also act as traditional switches or routers for IP traffic, while translating and packaging chirp and legacy traffic into IP packets for forwarding. Because the propagator nodes incorporate both chirp-based and traditional protocols, they are the natural point of integration for emerging IOT, legacy, and IP networks.

Sharp Corporation recently announced the QXC300 series of networking devices that acts as a traditional Wi-Fi  (meshed) Access Point and provides connectivity for conventional IOT devices such as cameras to deploy what they call “smart networks.” 

MeshDynamics’ propagator node software allows us to deploy WiFi networks today with minimal additional wiring and also incorporate emerging Internet of Things devices on the same infrastructure today and in the future.”
Mr. Arai Yuji, GM, Communication Division, Sharp Electronics, Japan.


6. What about security?

The security threat of billions of (conventional IP based) IOT devices is very real. IP based messaging (from Server to Device) simply won’t scale. IP is a sender-oriented form of messaging – thus, it mandates Encryption. That is a losing battle. Moore’s Law is slowing down and any way Metcalfe’s law is exponential e.g. O(n*n).

There is a good reason why Nature uses open, extensible, subscription-based (receiver-oriented) “messaging.” In the "chirp" world of IOT, security at the publisher, and assured delivery of any single message are passé. Chirps instead mirror nature with massive publish and subscribe networks based on “light” pollen.

As with nature’s pollen, pheromones, and birdsong, many may recognize that there is some data being published, but only the correct receiver will have the key to fully unlock the meaning. Chirp IOT is "female" (receiver-oriented) versus the "male" structure of IP (sender-oriented). When messaging is receiver-oriented, networks survive the relentless broadcast storms of spring. IP-based networks would collapse within days.

Further security is achieved through the applications agents in propagators. Secure data may be flowing through the propagator node network alongside open data, but is unintelligible without the encryption keys provided to the application agents. This is similar to receiver-oriented schemes found in nature, such as when air transports both proprietary (e.g., pollen) and open “signals” (e.g., birdsong). Individual propagators may be biased to transport or discard secured or open data.

One of the hidden security benefits of the chirp architecture is that there is no end-to-end direct connectivity to end devices – the propagator is always in the data path. With the potential for sophisticated security applications within the propagator, end devices are invisible to hackers and vandals. This approach is far simpler and cheaper than managing encryption and security at millions (or billions) of end devices – which further need not be burdened themselves with the processing power and memory needed for security applications. Small, dumb, cheap, copious – and secure. More


7. What about Standards?

The “Standards conundrum” suffers from the same misleading logic as requiring unique MAC IDs to address an IOT device. I alluded to this fallacy in my book where I describe how there are many John Smiths in the world, but the ones I have in my rolodex are sufficiently distinctive (to me, based on context) to be "uniquely" addressable. Local Uniqueness is enough. Nature concurs. In combination with receiver-oriented messaging, it is even exploited in how prolonged “broadcast” storms of spring disseminate pollen. The winds that carry the pollen are not “global” and time to live is inherently constrained. The same sort of broadcast over IP networks would be crippling, but each propagator effectively and automatically segments its local end devices from the network and vice-versa.

Propagators play a further role in managing standards and accelerating the proliferation of the IOT. Because each may contain applications agents tuned or defined by elements “higher up” in the network, they may serve as a translator for a wide variety of end devices. Chirp-based or IP-based, to name two, but also any variety of ad hoc, standards-based, or proprietary protocols found in older M2M networks. The propagator node effectively isolates and “spoofs” addressing, control timing, and other characteristics of the data stream. Again, addressing need not be globally unique – or even globally understood – application agents in the propagators host the necessary intelligence to handle all conversions.

Along with the possibility of a very wide array of physical interfaces on propagator nodes (wired, optical, and wireless, for example), these conversion capabilities ease standards issues and allow rapid migration of legacy networks to the IOT.
shuttle service schedules so no “passenger” misses a “flight”. The Abstracted Network manages all of this internal scheduling and re-scheduling transparently to the end devices and Big Data integrators.


Summary

Far from being a homogenous computing and networking environment, the next generation of Enterprise communications will actually require the integration of an even wider variety of protocols and devices as legacy and IOT applications are finally incorporated into the Enterprise network. Conversely, Enterprises will wish to extend their Software Defined Networking capabilities to encompass these “things” at the edge.

For Industrial IOT in particular, the future world of small, dumb, cheap, and copious sensors, actuators, and devices demands rethinking at both ends of the scale. At the far reaches of the network, simplified chirp based M2M protocols will minimize lifetime costs for the myriad end points of the Internet of Things.

At the same time, the concept of Abstracted Networks and powerful networking and applications tools concentrated in propagator node devices will allow unprecedented control and flexibility in creating huge Enterprise networks of diverse elements by extending industry-standard SDN tools and techniques.

Fully exploiting the power of the Internet of Things will grow from a total rethinking of network architectures.

MeshDynamics, in collaboration with our OEM partners, is exploring a machine-centric Abstracted Networks approach to managing “things”. Mesh Networks must also be future-proof: supporting legacy and emerging communication protocols. Fog resident Machine Controllers will require a time sensitive SDN-aware fabric. These real time mesh networks for industrial automation and  the Industrial Internet will be required to be Scalable, Synchronous, Secure, Future-Proofed and SDN-Aware .


About the author

The emerging Internet of Things architectural concepts and MeshDynamics wireless mesh networking propagator technology has been influenced by the Robotics and Machine Control background of founder Francis daCosta. (Early MeshDynamics nodes were installed on mobile robots.)  Francis previously founded Advanced Cybernetics Group, providing robot control system software for mission critical applications. These included local and supervisory real time machine-to-machine control. At MITRE, he served as an advisor to the United States Air Force Robotics and Automation Center of Excellence (RACE).

In 2012, Intel sponsored Francis’ book Rethinking the Internet Of Things  It was a  finalist for the 2014 Dr. Dobbs Jolt Award.


Related Links

The Abstracted Network Concept (Slides)
The Abstracted Network Concept (Paper and Slides)
Rethinking the Internet of Things: A Scalable Approach to ..