It’s increasingly challenging for organizations to maintain a comprehensive set of skills for their roles. We live in a fast-changing world with an evolving social landscape, the emergence of remote work, and rapidly advancing technology. With thousands of roles to keep on top of, succeeding in this endeavor is a key driver of success for talent growth, work transformation, diversity, equity, and inclusion.
Many HR technology vendors offer to address this problem by turning to Artificial Intelligence and Big Data. For example, they may provide a service which uses Artificial Intelligence to look at the content of an organization’s roles, and crowd-source skills based on a cloud of similar roles found in Big Data. While this does present a welcome opportunity for automation, it should come with warning labels.
Defining Big Data
Before we get into the opportunities and challenges that the usage of Big Data applications can bring, let us look at defining this intimidating term. Oxford Languages define it as “extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.” Using the above example of HR tech vendors, given how quickly job role names and descriptions evolve, it is easy to see why leveraging computers and coding is highly beneficial to keep databases up to date with the latest additions and definitions.
No doubt any leader searching for organizations to imitate would look for the most successful organizations who embody the talent growth, diversity, etc., which they aim to achieve. Artificial Intelligence, however, has no such qualms. In fact, the outstanding companies we might hope to be modelled after, tend to be in the minority.
Common industry problems are, after all, common to many organizations, and in Big Data the mediocre and poor-performing organizations are always overrepresented compared to the leaders.
The same goes for leadership in innovation and work transformation. Taking a statistical average of roles and skills from Big Data leads to the recycling of tired norms.
These problems are exacerbated when historical roles are brought into the Big Data pool, because the large quantity of historical data will always statistically outweigh recent advancements.
Let’s investigate the Big Data story for a Senior Software Engineer role
Historically, numerous social drivers have led to a prevalent but flawed profile for this role.
- In the mid-1960s a seriously flawed, but highly circulated study commissioned by System Development Corp. concluded that the people most likely to succeed in the field “don’t like people”, “liked solving puzzles”, and “dislike activities involving close personal interaction”. Both psychologists and almost all the participants involved were men, and the conclusions were highly speculative, however it had a strong influence which is still felt in the stereotype of the role today.
- Sexism, and especially toxic campus culture has alienated women attempting to enter the field for decades.
- Early approaches to software development were based on the rigid requirements of mechanical engineering. While the flaws of this analogy have been amply demonstrated, the approach is still prevalent in much of the industry.
Advancement in our understanding of what drives success in the software engineering field has led to the rise of iterative software development, which emphasizes close, cross-functional collaboration, and a process of continuous feedback. Engineers must attend first and foremost to the user and their human experience of the software. Clearly, people skills like leadership and stakeholder management are key.
Fuel50 conducted a Big Data analysis of Software Engineer roles across Europe, and found:
- Leadership skills are less commonly mentioned than 9 other soft skills. It is statistically insignificant compared to other priorities like, being “self-motivated”, “problem solving”, being “success driven” and even “curiosity”.
- Any skills to do with stakeholder management are listed 10th among just professional skills, below even dubious needs like “finance”.
These important soft skills are further buried below a long shopping list of technical skills, so if you use unsupervised, Big Data-driven AI to crowd-source a list of skills for a Senior Software Engineer, the most crucial people skills will have no chance of featuring.
So, relying on Big Data alone risks amplifying outdated and often problematic norms in your organization and undermining the very indicators of success which you set out to improve.
So, what should the savvy purchaser of a Career Architecture system do, to avoid the pitfalls of Big Data?
When considering adopting any Big Data-driven product for your organization, always ask how human experts collaborate with the Artificial Intelligence to ensure that the harmful biases in Big Data will be addressed.
Fuel50’s workforce psychologists and expert AI trainers recognize that bias can arise not only from the wording or “form” of skills, but also the way they are combined into one role. Our trainers monitor for systematic bias and our AI-driven content and profiles are generated in a way that ensures they actively promote diversity, inclusion, and appropriate soft skills, tailored to the business functions and your organization’s values.
Our human experts work collaboratively with Artificially Intelligent agents to find and review trending skills in Big Data, while ensuring problematic norms are avoided to prevent these from influencing Fuel50’s cloud.
This enables to provide the automation benefits of Artificial Intelligence and Big Data, along with industry-leading role profile design and excellence in diversity, equity, and inclusion.
When considering adopting any Big Data-driven product for your organization, always ask about how human experts collaborate with Artificial Intelligence to ensure that the harmful biases in Big Data will be addressed. This will help mitigate the negative impact of bias and will support HR teams and programs that include diversity, equity, and inclusion in their development.
