Cloud Architecture
Virtual Summit

Enterprise-Grade Integration Across Cloud and On-Premise

REGISTER

April 21, 2022

Online Conference

Mobile and IoT devices at the edge are driving data creation and generating insights for enterprises at an accelerated rate.

According to an IDC model, this trend will continue as the global datasphere will roughly triple in size by 2025, with over 50 percent of that data coming from mobile and IoT devices, collectively labeled “the edge.”

The deluge of data is also causing more technology and processes to move away from inflexible and expensive on-premises environments to cloud and hybrid environments, modernizing the way enterprises store and manage their data. However, cloud can only address the burgeoning edge data growth so much.

To take full advantage of the growing amounts of data, enterprises need a way to manage it more efficiently across platforms, from the edge to the cloud and back. In specific, they need to process, store and optimize different types of data that’s coming from different sources with different levels of cleanliness and validity so they can connect it to internal applications and apply business process logic, increasingly aided by artificial intelligence and machine learning models.

As data volumes continue to grow at the network’s edge, enterprises will also need a set of tools to extract, cleanse, understand, process and analyze data and send it off to both human and machine downstream and upstream consumers.

In large part, the answers to these needs will be driven by advancements in three key areas: data management, integration and analytics at the edge. If this sounds familiar, it should: Its all similar to what has already taken place in the cloud.

Don’t Reinvent the Wheel. Start with Cloud Data Fabric

A data fabric enables data that is distributed across different areas to be accessed in real time in a unifying data layer, under the same management. It provides data management and integration capabilities to extract, understand, process and analyze any type of data from any source and then take that data and any subsequent products of it and make it consumable by multiple types of human and machine users. A data fabric is agnostic to deployment platforms, data process, data use, geographical locations and architectural approach. It facilitates the use of data as an enterprise asset and ensures various kinds of data can be successfully accessed, combined, and governed efficiently and effectively.

While data fabric technology is still in its early stages of understanding and adoption for IT and data decision makers, its benefits are many. A data fabric acts as both the plumbing and translator for data moving on and off different platforms, including data centers, the public cloud, private clouds and the many types of gateways and devices operating at the edge. Data that is distributed across different areas can be accessed easily and transparently – in real time in a unifying data layer, under the same management – through a data fabric.

Introducing Data Fabric at the Edge

So, as data fabrics demonstrate their value, it may be time to think about an ‘edge’ data fabric.

As mentioned earlier, data at the edge will soon account for 50% of all data. That is explosive growth – and the path remains higher.  Just five years ago, edge data was under 10 percent, and the trajectory is for it to be the majority five years from now.

Organizations looking to implement edge data fabric will need to take a crawl, walk, run approach. The cornerstone must be a data management platform that delivers on the interoperability, security, and local and distributed data persistence, governance, and management support for processing and analytics.

To handle the growing number of data requirements and environments edge devices encounter, an edge data fabric must perform several important functions including:

• Accessing many different interfaces (e.g. http, mttp, radio networks, manufacturing networks).
• Running on multiple POSIX-compliant operating environments.
• Working with key protocols and APIs, Including more recent ones with REST API.
• Providing JDBC/ODBC database connectivity for legacy applications and connections between databases.
• Handling streaming data through standards such as Spark and Kafka.

Transitioning to an Edge Data Fabric for Big ROI and Minimal Risk

Just because data at the edge is often siloed and hasn’t been fully mined doesn’t mean it isn’t tied to mission critical systems. Shipping systems, medical instrumentation, communications networks, POS systems, chemical processing plants, factory floors and many, many more systems are quickly becoming tied to the network as IoT.

The first place we will see the shift of data to the edge and the need to implement a data fabric will be in use cases driven by profit and risk reduction, and it will happen rapidly over the next two years as data sharing rules are implemented.  This will get it first tests in the retail sector, as these companies move to collect consumer data to build a better customer experience and possibly improve cross-sell and up-sell through product placement.

Depending on the product and the consumer’s desire to keep their data private, applications will need to locate that data on the edge device for use by the application locally. Datasets related to the products and services will need to be sent from the Cloud down to the Edge device, essentially reversing the order and relationship we see today.

Another place I expect to see this change from edge to cloud to the reverse direction is with increased access by patients to their medical data. In this case, it’s a matter of regulatory compliance; you will have the right not just to access your medical records, including diagnostic imaging, but also the right to share or block sharing that data with any care delivery provider or insurer.  If this is the case, then the central location for that data is with the consumer and therefore at the edge instead of in the provider or payer’s portal and thus no longer centralized in the cloud.

Main Drivers for Adoption of Edge Data Fabric

A major focus of an edge data fabric will be to securely get value from it. But what are the risks involved and why should we still press forward? Here are some major drivers:

• Privacy and security of the data are a concern. IoT security is lacking because these systems tend to be automated, and the software is embedded with default configurations such as IP addresses. Having a cornerstone edge data management platform that secures data at rest and in transit as the default can remove the burden on developers at the programmatic or integration level. This will especially be true if an edge data management platform is built to leverage modern OS, VM, and container security mechanisms
• Expanding Consumption past OT or even IT gurus. To derive real value from edge systems, business and IT users product managers associated with Operational technology (OT) and business and operations analysts associated with IT and the business need to be able to consume data from the edge at each phase of a lifecycle they are concerned with – whether it be products, services or operations. Data engineers and data modelers will be crucial as the data wranglers that establish reusable connections between source data and systems at the edge to ingest, enrich, process and analyze data as well as catalogs of those sources and assets produced. Without them, expansion will be sporadic, incomplete, and suboptimized.
• You can’t make a one-for-one transfer of cloud data fabric technology to the edge. Resources are constrained and a lot of data on edge devices will still be transient, residing in in-memory cache on devices and thrown away almost immediately; just capturing state in a near-local database is a major step forward. Using the same architecture for all types of data is critical but so is using it for metadata, cataloging and other governance operations as well.
• OT and IT must collaborate to get real value from the data. A data management platform capable of meeting these requirements can be used across the layers of devices, gateways, complex equipment and edge computing platforms to support not only the actual data, regardless of where it resides or it’s type and veracity. Where possible, look for small footprint, easy-to-use tools that can be used for multiple functions.

The majority of data processing used to take place in the data center but has shifted largely to the cloud in the last 10 years. In both cases, the processing happens on one side of a gateway, but under a different set of circumstances, as the data center was fixed and the cloud is fluid. Now, with so much of data processing taking place in the cloud and at the edge, we are at an inflection point that has made data fabric a necessity for enterprises moving forward.

An edge data fabric, and in particular embedded edge data integration and management software, will be essential to deriving full value from edge investments.

Lewis Carr is senior director of product marketing at Actian, and leads the company’s product management, marketing and solutions strategies and execution. Lewis brings expertise in analytics, big data, cloud and mobility to his current role.