When a developer or team of developers build a software application, an essential issue they grapple with is how the application will integrate various data sources (databases or web services) it will need to operate. Data virtualization is an elegant new technological approach that offers a powerful way to address this issue.
THE ESSENCE OF DATA VIRTUALIZATION
As they say, a picture is worth a thousand words so presented below is an image of how data virtualization works.
What this graph is showing is the fundamental principle of data virtualization. The data required from the various data sources is abstracted to one virtual data layer. This virtual data layer effectively becomes a single virtual database for applications to draw data from. Some of the attributes of data virtualization are as follows:
- Selected heterogeneous data sources can be easily combined into one virtual data layer and can be accessed via queries or web services
- Selected data can be modeled and a virtual data layer created in days
- There is no need to replicate or move data from its source system
- The virtual data layer integrates smoothly with existing front-end development approaches and tooling
With data virtualization, a front-end developer can build an application while only needing to integrate with the virtual data layer. This is distinct from traditional architectural approaches where special plumbing (code weaving through layers of middleware) is built to connect an application to each of its data sources. This approach works, but requires more time coding and greatly increases the complexity of back-end integration, especially as an application grows and scales.
For example, consider an application that draws data from a dozen different sources. If a single change to the application is needed, then there may be required modifications on each of the twelve unique sets of wiring that join the application to its data sources. This consumes developer time, as considerations about multiple layers of middleware integration (security, auditing, monitoring, business logic, etc.) have to be taken into account. The probability of bugs and coding mistakes goes exponential. An application can become brittle as even small changes can cause it to break, requiring debugging to get it working again.
THE MECHANICS OF DATA VIRTUALIZATION
(can be ignored if you are not technical)
The first step in Accur8’s approach to data virtualization is to model the data sources to a data repository. The location of tables, the type of joins, security and other key bits of information are mapped from each of the selected data sources to a metadata repository (essentially a warehouse that stores important knowledge about a company’s data). The resultant metadata repository contains a detailed map on how to locate all of the data that underlie the virtual data layer.
From here, Accur8’s query engine can now come into play. A front-end query for specific data is passed to our query engine that, in real time, augments it with information pulled from the data repository. This creates a transformed query that can retrieve data securely from the required back end data sources. The query engine also has the capability to stitch data retrieved from multiple data sources back into a single result set, which constitutes a working form for front end users. Our easy-to-use query language drives the query engine. The end result is a front-end developer can integrate their application to our virtual data layer and win easy access to all of the required disparate data sources without needing to deal with typical back end issues such as security, auditing, monitoring, etc.
THE NEED FOR DATA VIRTUALIZATION
- Applications can be built significantly quicker and less costly than traditional approaches.
These savings can be enormous for applications integrating many data sources
- A virtual data layer that incorporates multiple data sources can be built in days allowing front end developers to focus on the UX/UI almost immediately
- Data virtualization creates an agile/flexible development environment as modifications and new features can be implemented quickly, bypassing the complexity of back end systems. This means a developer can easily iterate an application easily to quickly hone in on the final UX/UI
- It is very easy to integrate additional data sources (internal/external) to an application as needed in the future
- Front-end developers become much more productive and fewer back-end developer resources are needed to build an application
- Applications can be easily built on top of legacy data
- It greatly increases the probability of a developer or development team building an application on time and on budget
Accur8 Integrate, our data integration tool powered by Data Virtualization, was used for a security application at a major European airport. The application involved a network of hardware devices, from multiple vendors, located throughout the airport that would scan cargo entering the country and identify hazardous materials. Accur8 Integrate was used to build a virtual data layer that integrated all of the data generated from each hardware device in only a few days. This included data such as PDFs, images, videos, license plate captures, etc. This virtual data layer contained all of the data scanned from all of the devices and became the cornerstone for the application’s analytics and reporting, complete with security and auditing, and permitted rapid development of a highly responsive front end user interface.
Accur8 Software is a leading data unification company, focused on high performance, scalable and accessibly priced integration technology and tools. We recognize that companies’ application environments are growing in complexity as they deploy more and more software applications to drive their businesses forward. This complexity hampers business performance because valuable data from across the company is not readily available to business users or systems. It forces IT staff to waste significant time and money dealing with the never-ending cycle of trying to integrate and share needed data across the organization.
The Accur8 Integration Engine is a data unification tool designe d to help companies address the issue of complex application environments. It provides a flexible, agile way to unify data across processes and applications without coding. This means being able to integrate data and applications together whether they are in-cloud, on-premise or separated by geographical distance. It allows companies to access data and have it flow across the organization to users and systems as needed. Its capabilities include data integration, application integration, master data management, and reporting and analytics. It can be deployed as a point solution to integrate data between two applications or as a tool to unify all of a company’s data and applications.
We have customers ranging from growth stage to Fortune 150.