CoreDX DDS Architecture

Introduction

In this document, we present an overview of the CoreDX DDS internal architecture.

The Publish-Subscribe paradigm is well documented, and the Data Distribution Service Application Programming Interface (API) is standardized - you have confidence in this aspect of the middleware product. However, your understanding of internal implementation details can make or break your system.

With detailed information about the internals of the CoreDX DDS middleware, you can feel confident that you are making the right selection - one that will support the requirements of your system and fit into your architecture. A solid understanding of the middleware's internal architecture will help remove design uncertainty, highlight integration alternatives, and help you understand deployment scenarios.

There are, of course, many perspectives of the CoreDX DDS architecture, for example: the run-time architecture, the DDS entity hierarchy, the type system, data management, and the source code structure. Further, there are topics such as Dynamic Discovery and Content Filtering that deserve special attention as they can significantly impact system performance in large scale deployments. We will touch on all of these topics in this paper. If you are interested in further details, please contact us.

Overview

CoreDX DDS is developed in a modular fashion. Related functionality is isolated to provide easily severable code modules. This allows the middleware deployment to be tailored to the needs of the application, and supports small-footprint applications where code and memory size are of concern.

The Core of CoreDX DDS implements the 'C' language DDS API, the Real-Time Publish Subscribe (RTPS) network protocol. This includes the DDS Minimum Profile and the Ownership Profile QoS as documented in the DDS standard.

The ContentFilter module enhances the Core with the ability to filter data samples based on content. It provides a highly optimized (10's of thousands of filter operations (with thousands of SQL conditions per filter) per second) data filtering capability with robust SQL select syntax.

The CoreDX DDL Compiler is a development tool that generates code to support application specific data types. This tool is useful when data types are known at development time, and helps to support the type-safety of the DDS architecture.

The DynamicType module provides the ability for an application to produce and access DDS data without a-priori knowledge of the Data Type(s). This unique capability allows CoreDX DDS to be deployed into scenarios where data types are unknown or may change over time.

The language binding modules extend the reach of CoreDX DDS to C++, C#, and Java. The addition of new language bindings is straightforward.

DDS Entities

The DDS entities are defined by the DDS Standard. CoreDX DDS provides a standard compliant implementation of these entities. The entities provide the structure for the API, and define a hierarchical relationship between DDS components.

Type System

CoreDX DDS supports a full set of elemental data types (short, long, octet, boolean, float, double, string, etc) as well as complex (sequence, array, structure, and union) data types. The implementation compiles with the standardized wire-protocol encoding of data - Common Data Representation (CDR) - which provides a compact, efficient, and inter operable mechanism.

CoreDX DDS supports the OMG DDS Standard TypeSupport interface. This interface allows the application to define specific data types that will be known to the DDS middleware. The data type is specified in the Data Definition Language (DDL) which is a sub-set of the Interface Definition Language (IDL) commonly used by CORBA.

The DDS Standard mandates the ability to specify data types at development time. This is supported by the CoreDX DDL Compiler. The DDL Compiler generates source code based on a type definition. The generated source code provides a specialized Data Reader and Data Writer implementation that is tailored to the application specific data type.

Further, the CoreDX DDS middleware offers a more sophisticated type system that supports dynamic type generation and discovery. This extends the standard defined static type system with a more flexible DynamicType interface. With DynamicTypes, the application does not need to include any generated source code, which may be important for extremely space constrained applications that must deal with many different application defined data types.

The knowledge of type information allows the middleware to perform intelligent data management operations such as filtering based on data content or organizing data based on a 'key' value.

Data Management

A significant aspect of the Data Distribution Service is the management of data. Within DDS, data on a particular topic is grouped into instances based on key values within the data. Updates to an instance are referred to as samples. Each instance can have zero or more data samples associated with it.

DDS data instances have a well defined lifecycle. They can be created, updated, and disposed (or deleted). The CoreDX DDS middleware maintains the state information about all instances within a Topic. Instances can have one of three different states: ALIVE, NOT_ALIVE_NO_WRITERS, or NOT_ALIVE_DISPOSED. These states convey important information to the application and are also used by the middleware to manage data.

Data instances and samples are conveyed from DataWriters to DataReaders. In CoreDX DDS, the data is stored in Data Caches at the Writer and the Reader. The data in these Data Caches can be limited by various Quality of Service policies, and is removed based on several different events. The management of this data within the middleware is a complex topic with several different aspects that can be tailored by Quality of Service policies.

For an in-depth discussion, please see our white-paper: CoreDX DDS Instance and Sample Management.

Reliability

The Reliability Quality of Service is fundamental to the DDS architecture. This Quality of Service policy allows the application to indicate the level of effort required to guarantee that data is conveyed from a publisher to a subscriber. The CoreDX DDS middleware supports both levels of reliability as defined by the standard.

The Best Effort reliability setting indicates that the middleware should make an effort to get data from a Writer to a Reader; however, dropped data is acceptable. In contrast, the Reliable setting indicates that a Writer must ensure that data is received by a matched Reader. The reliable transmission of data requires an acknowledgement mechanism, and the ability to retransmit data if it is lost.

BEST EFFORT

The CoreDX DDS implementation of the Best Effort reliability offers extremely low overhead. When possible, Best Effort data is conveyed using UDP MULTICAST packets. This allows a single data transmission to reach multiple subscribers. Best Effort reliability exhibits the lowest network utilization, and is suitable for data that is frequently refreshed or of limited importance.

A Reader accepting Best Effort delivery of data may experience lost data samples; however, data samples will not be delivered 'out of order'.

RELIABLE

The CoreDX DDS implementation of Reliable data transmission provides an alternative that guarantees data transmission to Readers. In this case, the Writer sends Heartbeats to Readers, and Readers can reply with an ACKNACK message that serves a dual purpose: 1) to acknowledge certain data samples, and 2) to request retransmission of certain data samples (if necessary). CoreDX DDS offers an extremely efficient implementation with pre-cache buffering and tailoring of handshake timing through Quality of Service policies. When possible, the CoreDX DDS middleware will utilize UDP MULTICAST for data transmission - this offers the write once, reach multiple subscribers' benefit seen with Best Effort.

In the graphic to the right, we show the protocol messages required to recover from a dropped sample (sample number 2, in this case). The handshake messages (Heartbeat and Acknowledgment) are shown interleaved with the data messages. And, because CoreDX DDS allows the application to tailor the timing of these handshake messages, the network traffic can be optimized to the applications particular data publishing patterns.

Run-time Components

The CoreDX DDS publish subscribe middleware is a complex communications system. The flexibility of the DDS architecture, and the need to support a wide variety of platforms (hardware and software) requires a sophisticated software design and implementation. The engineering team at Twin Oaks Computing brings world class experience in software architecture and design coupled with a practical real-world perspective on software development. The result is a software architecture that meets the demanding requirements of high-performance data communication while maintaining a clear structure - making CoreDX DDS easy to maintain, extend and certify.

The CoreDX DDS middleware insulates the application from the details of interfacing with the Operating System and network stack. It provides Data Cache Management and Event Notification that can be customized by a robust set of Quality of Service policies. It includes support for Dynamic Discovery which enables the valuable loose coupling of the DDS publish subscribe software architecture. The run-time components pictured to the right work together to provide all of these features in an extremely small footprint. The static library code size is 500KBytes; the run-time memory requirements vary based on the number and nature of DDS entities required by the application, but can be well under 200KBytes.

Dynamic Discovery

The DDS wire protocol identifies a mechanism to support Dynamic Discovery of DDS entities within a domain. The Dynamic Discovery mechanism is a key component of a DDS publish-subscribe network, and adds significant flexibility to the deployment options of DDS. DDS enabled applications can easily adapt to changes in the system and network architecture, often requiring no code changes or re-compilation.

CoreDX DDS completely supports the standard Dynamic Discovery mechanism. This mechanism is a peer-to-peer architecture in which the information about existing DDS entities is exchanged between all DDS Domain Participants. Thus, each Domain Participant has a complete picture of all current DDS entities in the Domain. This mechanism is very effective for a relatively small network of entities. As the number of entities in the Domain grows, so does the amount of memory required to maintain this complete picture at each Participant. Eventually, the memory requirements of Peer-to-Peer Dynamic Discovery can become an issue that affects the scalability of standard DDS implementations.

CoreDX DDS Centralized Discovery addresses the scalability issue of the standard Peer-to-Peer Dynamic Discovery. It offers a standards compliant, highly scalable and flexible discovery architecture. With this capability, and its highly optimized DDS Entity management code, CoreDX DDS enabled applications can efficiently participate in DDS Domains containing tens of thousands of entities. Furthermore, a system can mix Peer-to-Peer discovery with Centralized Discovery to achieve unique DDS Domain architectures, all without sacrificing interoperability!

Content Filtering

Content Filters enable an application to filter data based on the content of the data. By implementing the filter in the middleware, the application is relieved of the burden of examining each data sample, and can focus on only relevant data samples. Further, by applying filters at the source of the data (the DataWriter), some applications can conserve signification network bandwidth by avoiding the network transmission of irrelevant data.

CoreDX DDS provides very efficient support for Content Filters. This support covers both static and dyanamic data types. The SQL-like filter expression language implemented by CoreDX DDS includes the powerful 'SQL IN' clause which provides a powerful capability to select data based on inclusion in a list.

Source Code Structure

The CoreDX DDS software is structured to match its modular architecture. This makes the source code easy to understand and maintain while supporting its key open architecture concepts. From a clear naming convention to a concise directory structure that groups related functionality, the source baseline is developer friendly.

The CoreDX DDS middleware supports multiple hardware platforms and operating systems. Therefore, the software must be written to support many different operating systems, compilers, and execution environments. It must accommodate the variations in the threading API's, as well as hardware differences such as Big-endian vs Little-endian data representation. Further, the build system must support many different build environments and compiler tool sets. CoreDX DDS includes a flexible build system that works under any modern operating system.

The CoreDX DDS Source Code is arranged in sub-directories based on functionality and requirements. Within sub-directories source code is arranged in multiple source and header files (C, C++, C#, and Java). Internal header files cover the ‘private’ API that is used within CoreDX DDS but not exposed to users. Public header files that document the API exposed to CoreDX DDS users are collected in a common area. The CoreDX DDS source code is written to be portable and compact without sacrificing readability.

Conclusion

The CoreDX DDS middleware includes many innovative technologies that are not to be found in other middleware implementations. This brief overview touched on some of the highlights of the CoreDX DDS software and provided some insight into its architecture and design. If you are interested in more details, or have any questions, please contact us to discuss.