ALOOK INTO some of our

Our Work

We have crafted solutions for industry leaders, combining years of expertise with agility and innovation.

Stretching the limits of the Cloud with a Chatbot visualises data for the masses

Published • 06 March 2024

When customer experience transforms to the scale of citizen experience, superlative solutions are needed.

When Covid erupted in South Africa it was not only a question of how to distribute millions of grants to those in need. It was a question of how a system could manage a plenitude of data in real-time to provide true insight.

SASSA consulted the client working with its technology partner Synthesis to digitalise the grants. It was of national importance that the system managed millions of users securely and that SASSA could visualise the citizen experience and ensure it was not failing them at any point. Real-time telemetry was needed.

The Challenge

The chatbot is used when a person sends a Whatsapp or a Facebook message to the client with a citizen query.

With the grant, citizens sent a message to the client requesting the grant application and they then received a one-time pin (OTP) to ensure security and a unique form ID. They then completed the form on an easy-to-use and secure portal. Yet the data was astronomical with 3.5 million messages being processed a day in the first two weeks.

The client and Synthesis built the grant functionality in a mere six days given the urgency. In the beginning, the data was available but not collated and visualisable due to the speed at which the solution was needed. If something was failing in the citizen journey, the team could not identify the source.

A real-time dashboard of the data was needed to visualise system usage, any failures, chatbot response time and citizen sentiment.

It was critical that nothing interrupted the citizen journey to successful grant request. The client now manages customer satisfaction at a national level.

The Solution

“Initially, we used a Grafana dashboard with Influx DB as the metric store and Flink was populating it based on live data spooling into the multitude of different client sources. Flink sourced the streams out of all the different sources and joined them. The data was coming in at an unpresented rate and from numerous sources,” explains Tom Wells, Synthesis Chief Disruption Officer.

“We built this in the AWS Cloud to allow for massive scale and security. With this build, there was clarity.”

Flink was used to process and calculate the influx of data live. This was graphed and there is now precise visibility and instant clarity of any lags given the millions of users. When the grants were released, there were 59.8 million messages processed. Using Flink, the teams could detect delays in the chatbot response time as they occurred, and they could address them immediately.

“At such a pivotal and uncertain point in the pandemic, it was critical that we had site of how the chatbot was responding, its speed and the quality of service as this was an essential delivery for South African citizens,” explains Darren Bak, Synthesis Head of Intelligent Data.

Technology often needs to evolve with customer needs. In the second round of grants, Kinesis Data Analytics was used. The teams wanted to understand which users were actually interacting with the platform and where they were residing. They were able to add a second layer of data on top of what existed.

“We had to send an hourly report,” says Archana Arakkal, Synthesis Machine Learning Engineer, “Instead of having to manually conduct this, we now had a real-time dashboard. We could visualise heat spots in the different regions at a granular level. We could visualise what was happening in each ward. For example, if a ward was using the chatbot less than anticipated we knew that more public knowledge was needed in that area. This assisted with target marketing and this level of visibility is a winning factor whenever a company wants to increase the use of its offer.”

Technologies used: Apache Flink, AWS Kinesis Data Analytics, Amazon S3, Amazon API Gateway, Amazon Kinesis, AWS Glue, Amazon Lambda, Amazon Dynamo DB, Amazon Redshift, Amazon Athena

The Results

There was now a graph with real-time visibility of the data which provided profound insight into the situation. The client had precise knowledge of the citizen journey and could pinpoint where the journey needed to be altered so citizens could successfully complete their grant application. This included changes in the application form itself as an example. Any hinderance to completion was detected and amended.

“The data insights allowed us to change the whole journey where needed to ensure user satisfaction. We had profound user journey knowledge. Government entities knew exactly where they needed to change their processes,” says Arakkal.

AWS needed to increase the limits of AWS Lex, which processes messages, to handle this volume. It was an unprecedented situation. The client had to manage an essential customer journey at a scale never seen before and the platform connecting government to citizens rose to the challenge.