ALOOK INTO some of our

Our Work

We have crafted solutions for industry leaders, combining years of expertise with agility and innovation.

The ML story: how the world’s largest first-party data company and Synthesis transformed from non-believers to believers

Published • 23 July 2023

In 2020 PCG (pre-ChatGPT), the world’s largest first-party data platform had an idea.

But there was a problem, and the company didn’t believe it was possible. Even its solution partner, Synthesis, which is renowned for innovation, couldn’t guarantee it could bring the idea to life. But fortuitously both parties were willing to dabble with what they then penned as science fiction.

This is the story of how two brave companies went against the status quo and transformed from non-believers to believers.

The Challenge

The best insights come from the source. Over its tenure, the company has uncovered 4 billion real insights.

As an industry giant, completing over 100 million surveys, the company has discovered, curated, and designed ways to get the greatest and most accurate responses from surveys. This science lives in its complex survey programme.

The questions are placed into a Word document and then programmers manually code these questions into the system to include its best practices for flow control – such as skip logic and repeated questions – and non-leading questioning.

The company’s potentially impossible idea was that with Machine Learning (ML), the word documents could “code themselves” into the programme using its millions of learnings, making their survey process that much faster.

The company knew there were other ways to speed up survey development time, but it didn’t want to pursue the status quo so it contacted Synthesis.

Convincing the non-believers

Synthesis’ Intelligent Data team was transparent that this would be a journey of experimentation. It had extensive experience with ML, but this pushed the technology’s boundaries.

The idea was so cutting-edge that Synthesis’ ML expert, Marais Neethling, had to convince himself to agree to take on the challenge. He was reluctant to commit to a project that he couldn’t promise he could deliver.

“Our specialty is experimenting with bleeding-edge technology and using it to solve the most complex problems, learning as the technology unfolds, but this was different”, explains Darren Bak, Synthesis Head of Intelligent Data, “We couldn’t guarantee an outcome and we never commit to a project we cannot deliver but when the company said they were willing to take that risk then we were onboard. We were in it together.”

“We couldn’t say with 100% certainty that it would work but we believed it would. We both agreed to suspend our disbelief and to commit to experimentation,” explains Bak.

An uncertain journey begins

“We decided to pursue a proof of concept (POC), a proof of value, and a proof of technology,” explains Bak, “We started a prototyping process to determine whether the idea would be feasible and achievable.”

“At first, we didn’t necessarily know what the end product would look like or what basic features it would need. The best way to start was obvious – by empathising with the company, understanding their problem, and then looking at a solution using Artificial Intelligence (AI) to automate the process,” says Neethling.

Synthesis used generative AI that was still in beta testing. Bak says: “It was invite-only at the time. GPT3 had just come out and GPT2 was only known by Natural Language Processing (NLP) specialists. Not many people even knew about OpenAI, the company behind GPT3. ChatGPT doesn’t exist yet. It hadn’t seen the light of day and we were creating solutions with a rich understanding of natural language using the precursor to ChatGPT. It was real bleeding-edge technology.”

In nine weeks, the first POC was ready, and the results were there. “There was a spark,” explains Bak, “When Marais first presented, I remember thinking this is going to happen. This is one of the most conceptually exciting projects we have ever worked on.”

“I have to credit the company. They gave us the time and leeway to explore different approaches. It was a team effort where we worked together every step of the way. Their openness and deep desire to improve their customer journey made this possible,” says Neethling.

The Solution

The final solution proved the company’s idea was not only possible but exceptional. The successful outcome was a pipeline that could take the human-written survey in English and convert it into a structured, programmed survey platform. This included the programming of the outline and the logic so the flow of the surveys would change depending on the answers of the respondents.

“What that pipeline is really doing is interpreting, reading, and understanding the word document and survey science the same way that a human would create something,” explains Neethling.

ML now converts Word documents into computer code which automatically gets uploaded into the system. Customers simply write their Word document, upload it and drag and drop it into the system. The survey is instantly programmed for them, improving their experience and saving programmers’ time.

The company then created the interface for the programme and the companies worked together exchanging code to bring the idea to life.

The Result

The idea was no longer science fiction. Word documents were successfully coded into the company’s survey platform.

In testing the efficacy of the pipeline, over 100 surveys were put through the system between the end of December 2022 and early January 2023, showing an aggregated 23% reduction in time overall per project, with simpler surveys achieving a 50% reduction.

Programmers would have time for other value-packed work and customers would enjoy a faster process.

A willingness to explore the art of the possible, expertise, and timing made this possible:

“If the company came to us in 2018 with the same idea, I would have said it was wishful thinking. In 2020, I was testing a precursor to ChatGPT and using it as a playground for transforming surveys into structured forms. I saw its potential,” explains Neethling, “This potential convinced me to agree to the project even though we couldn’t promise the outcome. If the company was prepared to do scary stuff, then so were we.”

When asked why they pursued this extraordinary idea, the company representative said: “We thought that if our ideas worked even remotely well, it would be better than any in-the-box solution. It would help remove burdens from our customers and internal programmers. It was better to take the risk.

Innovation was there, and it paid off. We didn’t want to follow what everyone else was doing and it was a victory. Now, we are believers. This is real. It is not all unicorns and rainbows, and we couldn’t have asked for a better partner.”

Today the idea has been transformed into a reality and has opened a world of future possibilities for the company where ML can continue to improve its already outstanding surveys.

“This is only the beginning of what we can do with this technology and our partnership made this possible,” concludes Bak.