As overwhelming as technology can be, it is generally predictable. Often, the challenges developers face boil down to misconfigured code, that can be adjusted over time, humorously referred to as PEBCAK: Problem Exists Between Keyboard and Chair. Alternatively, software providers can release patches or introduce new features. On the other hand, people are far less simple, even in professional environments. This is eloquently encapsulated by my astute colleague Hanno Brink:
“We are normal people with professional facades.”
People can have good days and bad days, exhibiting varying behaviours in both cases. They also have numerous underlying motivations, individual biases, and different knowledge foundations explains Gabriel Eisenberg, Senior Data Engineer at Synthesis Software Technologies. These factors result in inevitable gaps that can be challenging to bridge. To do so requires clear communication, understanding, adjustments of one’s own behaviours and paying attention to the other person.
In my experience as a Technical Lead, I found it rather surprising that people encompassed about 50% of the challenges and technology accounted for the remaining 50%. Conversely, as a Sales Engineer, I found that 90% of my problems came down to people, leaving only 10% as technical issues. Even as a developer, the human element cannot be evaded, with an estimated 20-30% of challenges being people related.
While one might assume that presenting a robust technical solution would result in easy acceptance and progression in the building process, it’s not that straightforward. There’s a continual need to convince others of the solution’s validity, align perspectives, address miscommunication openly, build both relationships and trust over time, and collaborate closely with the client and team members.
People play a pivotal role in these processes.
So, is the solution to avoid people entirely? While tempting for the stereotypical introverted engineer, it’s likely impractical and would lead to isolation. Moreover, the most intriguing and challenging technical problems and solutions often emerge from collaboration.
Where people matter in business
As a consultant, one will always be interacting with a client, both at the stakeholder level and at the user level. One needs to be considerate of who one is speaking to. From the broader business context to the nuanced dynamics of interpersonal communication. Each facet plays a role in shaping successful outcomes for projects.
Business context
I have been reading a fantastic book by Joe Reis and Matt Housley called, “Fundamentals of Data Engineering”. While the below extract is framed for data engineers, I believe that it generalises for technologists working across industries:
“Without a framework for managing data, data engineers are simply technicians operating in a vacuum. Data engineers need a broader perspective of data’s utility across the organisation, from the source systems to the C-suite, and everywhere in between.”
Upon reading these two sentences, what struck me was how this related to my experience working in a platform team responsible for the development of an enterprise scale data lake. As a relatively small but competent team, we were overwhelmed with the demands of the many teams and stakeholders who wanted their data to be on the data lake or wanted to consume from it. In trying to make the data available and conform to consistent formats despite the inevitable, seemingly endless variations and problems, we could not take the time to understand the context of the data, its usage and the nuances of its business rules.
Given the time, engaging with the business stakeholders and individuals would allow for a better understanding of their requirements. It helps to understand what problem we are solving by building a particular system, and what it means for the business itself. The “why” is critical, explains Gabriel. Understanding this and the monetary impact will underly the decisions being made regarding the system and can also be used to frame design decisions and arguments.
Seeing it from others’ perspectives
What are the motivations of people you are working with and why do they need to achieve the tasks they’re putting forward to you? By empathising and understanding where they are coming from, you can obtain context about the broader business and stressors.
Open and regular communication
Open and regular communication or feedback will ensure opportunities for earlier intervention, if needed, and will help manage the expectations of stakeholders. Managing expectations can make or break a project. This is one of the biggest lessons I have learnt. Being upfront with your stakeholders builds personal trust and manages expectations.
Engagement protocol and establishing boundaries
It is important to set boundaries and lay out terms of engagement. This protocol will prevent you and your team from being overwhelmed by ad-hoc requests and will help prioritise requirements and manage client expectations.
Meet them where they are
The reality is that you don’t necessarily know the context of the people you are talking to. Job titles can be misleading and force assumptions into interactions. It helps to first understand if the person you are speaking to is technical or not. Secondly, what level of information do they need to know given the situation.
Gabriel reflects on a mistake when first engaging as a Technical Lead, “I made the mistake of speaking to a business stakeholder at my technical level. This led to miscommunication where we would end up talking past each other. After a month or so of this back and forth where we didn’t make much progress, and with some guidance from a colleague, I pivoted my approach and language to meet the stakeholders needs. This streamlined our interactions and made for a far more fruitful and enjoyable project. This lesson is something I have applied ever since learning it, and while I sometimes find myself reverting to more technical language, I do my best to check in with the other person to ensure that they are following what I am saying.”
Lower the barrier to entry
Technology can be intimidating, especially to non-technical people. For any developers reading this article, consider the first time you come across a new tech stack or tool. That feeling of intimidation or imposter syndrome might be -ever-present for someone who is non-technical. For your solution to be used, it needs to be simple with relevant automations.
Gabriel shares real-life examples of this below:
In one of my projects, we built a JSON configuration layer to deploy data pipelines and their associated infrastructure with Terraform (one of my favourite tools). For us, it was relatively simple as all we had to do was populate the JSON config file, follow the Git workflow (Gitflow) that we had defined, and everything would be deployed for us via CI/CD. Surely our users would easily adopt this pattern? People struggled with this. As a platform team, we made assumptions about the capabilities of our users and did not engage with them to understand where they were. Many people were unfamiliar with git which complicated many of our deployments. Reflecting on this, we probably should have created a simple user interface for our users with helpful prompts and automations to fulfil the required Gitflow.
In another project, an on-premises feature store was migrated to AWS. As Spark fans, we opted to rebuild the SQL scripts in PySpark with AWS Glue. We assumed that since Python is extremely popular, the barrier to entry was low for users to build their own custom scripts. Many people were unfamiliar with Python and Spark, and we should have made their lives easier by sticking to SQL. Perhaps we could have opted for a tool like DBT to keep this as simple as possible – with appropriate governance and data discovery to prevent DBT from spiralling out of control.
Allow for regular user feedback
For proper adoption of your solution, you need to be guided by your user. This can be guided by a persona – the actual person who most accurately represents your userbase and whose opinions should guide your development. For example, perhaps they prefer a blue button over a green one and therefore, we should favour their feedback. One needs to build for what our users need versus what we think they need. Beyond this, with regular feedback you are more likely to hit your targets.
Keep it simple stupid (KISS)
Use simple language and sentences, especially in documentation. Complex language will prevent people from understanding what you have built and may prevent it from being used. Images and flowcharts will often explain more than words.
Trust is earned and not given with clients
Trust with clients is earned through honesty, ethics, and genuine commitment to their best interests within the agreed scope. Improvements that are out of scope can be placed on a backlog which shows commitment to improving clients’ environments.
Trust can also be built by showing clients that you take care of the work that you do. My colleague, Nikeel Sathoo, summed this up perfectly. Nikeel suggested that rather than diving into developing a solution, for example; on a call with clients, take a step back from the interaction, do your research for a reasonable amount of time and then come back with a solution. Not only will this ensure that your approach is strong, but it will show the client that you want to get the right solution for them, rather than a generic one that might address some of the immediate concerns but may not scale to others that are less obvious.
Gabriel concludes by stating that navigating the nuances of project delivery within a client domain can be incredibly challenging, but with a communication-driven approach, this can be made much simpler.
