In a world where people are demanding instantaneous service and next-day delivery, the concept of rapid or real-time data is becoming crucial.

Synthesis, one of South Africa’s top software companies, together with SEACOM and TransUnion, recently hosted an event in Johannesburg where experts from various industries deliberated the topic. Representatives from Synthesis, TransUnion, SEACOM, Standard Bank, and Imperial Logistics participated in the discussion.

Real-time data refers to the idea of having information instantaneously available that can help a business and its clients make decisions faster.

“It’s a decision I can make without having to ask anyone anything,” said Tom Wells, the Chief Disruption Officer at Synthesis. “As soon as I have to go and query some system, in order to make a decision, that’s no longer real-time.”

Whether in retail businesses, financial services, or logistics and supply chain management, the basic idea with real-time data is to give customers what they want as quickly as possible.

Fast is the new cheap

“Fast is the new cheap,” said the Global Head of Product at SEACOM, Robert Marston.

“That’s a saying that you heard in the telecommunications industry a few years ago,” Marston said. “We are living in a world where the average revenue from each user is dropping the whole time. Thanks to technologies evolving, there is always someone doing it faster and cheaper.”

The revelation was that rather than trying to be the cheapest option on the market the whole time, what people actually wanted was speed.

“Instant gratification. That’s the new norm in the digital age,” Marston said. “As a consumer you want something to happen right now. You don’t want to wait.”

As a recent TransUnion study shows today’s consumers demand a seamless onboarding experience and fast, personalised service. Loyalty is directly linked to top-notch service and accessibility, so it’s vital to ensure your processes aren’t cumbersome.

“People remember the experiences they’ve had. They may not complain or stop doing business with you immediately when they’ve had a bad experience  – instead, they will simply not give you their repeat business in future,” said Priven Moodley, Head of Consulting at TransUnion. “Customers want faster transactions and less friction – and here’s where real-time, high-quality becomes the key ingredient.”

Aside from the benefits that can come from using real-time data, such as fostering customer loyalty, the fact is that immediate service has become the new norm.

E-commerce platforms like Takealot and Amazon let people get their goods delivered within a few days of ordering. In the case of some items, like books, there is no longer any need to wait for them to be delivered unless you want a physical copy. You can simply buy an audiobook or ebook and start reading it immediately.

Services like Uber and Netflix have cemented this expectation of immediacy. Uber is also an example of giving the customer constant, real-time feedback on the status of the service they requested.

The value of Rapid Data

To build a real-time data system, Wells said that event-driven architectures work well.

“You break your system down and model it as a stream of events that occur over time. When someone makes a payment, takes out a mortgage, or steers left on a car—whatever it is, those things are just events,” he said.

You then build all your software to consume and interpret those events as they occur.

In addition to enabling rapid decision-making, real-time data that has been gathered can be used to gain greater insights into customer needs.

The data may also be used to train and invoke machine learning models. These could be used to make rapid decisions more accurately in future, or provide new ways to analyse and respond to customer behaviour.

Decentralised data

To process data in real-time and allow a customer or employee to make rapid decisions, systems need to be decentralised.

“There is a shift to turn Big Data systems into real-time predictive systems,” said Tom Wells, the Chief Disruption Officer at Synthesis. “That means taking those centralised Big Data architectures and trying to distribute them, making them more decentralised.”

There are several reasons that real-time data requires systems to be distributed. The first is if you need your application to work even when you have slow or unreliable connectivity. Bank and credit cards are an example of this. Depending on how a shop has set up its point-of-sale systems, clients can still pay with their cards even when a connection to the bank’s network is not available.

The second major reason is the physical limitations of how fast data can travel through computer networks.

“It’s almost mind boggling that you can move data from one side of the globe to the other in under a second. In the time it takes to say ‘one’, you can send a packet of data from Hawaii to a data centre Johannesburg,” said SEACOM’s Marston.

However, there are many factors on the Internet that you can’t control ranging from network congestion to cable faults.

“The closer the data is to the person using that data, the better,” said Marston. “That’s why we’re seeing AWS (Amazon) and Azure (Microsoft) setting up data centres locally.”

Data quality — finding the nuggets

One of the challenges of gathering real-time data is making sure that the quality of the data does not degrade compared to more traditional systems.

“Our business is focused on providing solutions to business problems using the highest quality of data. As part of our journey we are well along the path of speed of delivery of that data, without compromising its quality,” said TransUnion’s Moodley.

There are two types of data. Static data, or data at rest, is transactioned in real time. This means the data doesn’t change or need to be assessed, but is merely made available for consumption. Examples of this would be ID numbers, models and risk scores.

The second type of data is real-time data, which is data coming into a system, being immediately assessed, and being sent out again, with minimal time spent assessing that data. An example of real time data is income, card usage or weather. The rules, the process, and the decision making are within a very short timeframe.

Privacy and security of decentralised data

Adopting a technology architecture designed for real-time data does come with different privacy and security concerns.

If personal data is being used to build a profile for someone to assess their creditworthiness or combat fraud, you must comply with relevant regulations such as the National Credit and POPIA Acts in South Africa, GDPR in Europe, or CCPA in California.

An organisation will also have to think differently about information security if it wants to use real-time data.

One major consideration is the fact that to make data available rapidly, it needs to be physically close to the place where you will be using it. This could mean that data may be downloaded onto a user’s smartphone, or any other device that is physically near to where the data needs to be used.

The challenge is how to deal with highly personal data, which is essential for businesses like banks to be able to provide financial services. To ensure you keep people’s personal information safe, you cannot send out raw data: it has to be transformed into information and aggregated insights, that can then be used to make decisions, said Moodley.

Right place, right time, right channel

One way that real-time data may be used is to communicate offers and useful information to clients. For this to work, timing and context is everything.

Real-time data lets a business send out messages that are personalised for clients. These messages should be delivered to customers when it is relevant to them, rather than simply spamming them with offers.

Banks say that the most predictive data they have about clients is the behaviour on their transactional account and credit card, and their payment profiles at information and insight providers like TransUnion.

Wells said that this data may be used to drive behaviour instead of just predicting it.

The bank should be able to detect that you shop at a specific place at certain times. Shortly before you usually leave work, the bank can send you a message to inform you of convenient alternatives where you may earn more loyalty points for buying the same products or services.

“Imagine the bank notices that I regularly stop at a specific petrol station every week or so. It can make an educated guess that I stop there because it is conveniently located on my route between home and work,” said Wells.

If the bank has a different petrol station chain signed up to its rewards programme, it can look for one that is close to the one you regularly visit and recommend that you fill up there instead, helping you to easily increase the rewards you earn every month.

“If the bank demonstrates to me that it understands me as a consumer, understood my lifestyle, and wanted to promote better rewards — that is awesome,” Wells said “That’s why people use services like Google Assistant.”

The ability to rapidly access high quality real-time data dramatically changes the way people interact with companies and services, and how businesses respond to customers.

Rapid, real-time data can offer significant value, but only if it is delivered to the right place, at the right time, and through the right channel.