NLP: Promoting the inclusion of women in the digital sector

By doing so, Microsoft is illustrating the potential of Artificial Intelligence to better the common good! Taking part in the program, I had the pleasure of assisting the social enterprise “Social Builder” in the Diversity & Inclusion category. I’d like to tell you about this enriching experience. 

Perfecting the chatbot user experience 

Social Builder is a social enterprise championing gender equality in the digital world. Its aim is to help women change careers and find employment, through guidance, training and integration actions. The tool used by Social Builder to pursue this ambition is Adabot, its virtual assistant. 

Launched in October 2018 on Facebook Messenger, it connects female jobseekers and women looking to change careers to the digital ecosystem in their area. Adabot is a genuine virtual orientation coach and guides users throughout their career. However, the way the bot worked was not enabling Social Builder to effectively achieve its aims, which include making Adabot a decisive tool to guide women towards careers in the digital sector.  

Several obstacles were identified:  

• Mandatory authentication via an external account at the start of the process  
• Lack - or even absence - of customized answers 
• Lack of data collection - even though this would help to enrich the user experience 

The whole idea of the Share AI project is to implement artificial intelligence solutions that will significantly improve the user experience using Adabot. 
 

Technological solutions deployed 

One of the primary aims is to convert Adabot into a proprietary intelligent bot; in other words, a bot that is native to the Social Builder website. Then solving the issues identified upstream could already make the bot much more intelligent, accurate, compliant and upgradeable. Finally, to become a genuine artificial intelligence aide, Adabot needs to understand the user’s intention, irrespective of sentence complexity, and be able to ask follow-up questions to eliminate any ambiguity or simply find out more about the user. It needs memory to reuse key information throughout the conversation, for context or customization purposes, such as getting the conversation back on track if the user asks irrelevant questions. To achieve a certain linguistic level, Adabot must be able to learn from its users.  

With this in mind, we decided that NLP (Natural Language Processing) would allow both analysis of users’ feelings, based on their answers, and better understanding of their intentions. The aim is to determine their experience and offer them options allowing them to get on board more easily with the recommended career. The key issue is making the bot more intelligent by continuously improving the quality of its answers, and to give it the ability to offer users custom coaching. NLP is the obvious choice to improve Adabot! 

Our objectives are to improve the bot’s retention rate and provide users with more reliable career assistance by improving performance in terms of:  

Access to and use of the system:  

• Relevance of the answers provided  
• Better understanding of user intentions  
• Custom assistance 
   

NLP: Usage context, benefits and limits 

With the advent of deep learning, NLP has been used for various tasks, such as understanding, ranking, translating and predicting (or generating) text, among many others. It enables bots to understand the semantics of the language used, text structures and spoken sentences.  

This makes it possible to extract useful information from a large volume of text data. Among other things, it can help identify recurrent issues with a product or service based on user reviews and then deduce performance indicators, such as the customer satisfaction rate and experience.  

We first identified several technology obstacles to eliminate: 

• Access to the bot requires authentication via an off-platform account  
• The existing bot’s decision tree is static  
• The data is not easily actionable to improve the bot  
• There is no interface between the bot and Social Builder’s CRM 

To eliminate the first obstacle, Social Builder transferred the bot to another platform, called Vizir. In order to ensure GDPR compliance, users can now choose whether to consent to the history of their conversations with the bot being saved. Using the saved history, we analyzed conversations in order to measure the bot’s performance and determine the causes of cancellation or any issues encountered by users.  

Performance is measured by producing indicators. Adabot already had several KPIs, including the number of new users, duration of conversations and a satisfaction survey. These indicators are still not enough for effective measurement. To take account of users who do not answer satisfaction surveys, we chose to focus our analysis on users’ words. By analyzing feelings based on conversations, we can now measure:  

• Bounce rate, which reveals the percentage of users who visited the website without consulting the bot 
• User experience, which detects whether users are interested in or indifferent to digital professions after their discussion with the bot, based on the words they used 
• Interest in events or training courses thanks to the bot, to find out whether a user was convinced to sign up after their conversation 
• Interest in digital professions thanks to the bot, to see whether the conversation led the user to consider working in the digital sector  

We therefore used the AI Builder in Microsoft’s Power Platform solution to train a feeling analysis model, with the aim of classifying user opinions into three categories: Positive, Neutral, and Negative. 

• The first group indicates that the user was satisfied with her conversation with the bot and got all the information she needed. 
• The second group includes all users who were hesitant or doubtful following their conversation with the bot. These users still need to be convinced via a follow-up phone call, with one of Social Builder’s advisors for example. 
• The final category gives us an overview of users disappointed by the bot, either because they did not get the information they needed or because they do not identify with the situations set out by the bot. These users’ conversations are further analyzed to pinpoint any possible bottlenecks that made their conversation sterile. This will then enable Social Builder to improve the bot’s conversations and/or recommend new careers tailored to these wide-ranging situations. 

Our work with Social Builder on Adabot is still ongoing and new ideas are likely to emerge with the integration of NLP, such as translation, in order to include non-French-speaking users. NLP should enable Adabot to have a real conversation with users, while an individual’s conversational data could provide invaluable information, by understanding trends and better interpreting users’ feelings. The natural language processing model proved essential to analyzing conversations and identifying a set of indicators capable of shedding light on how the conversation went.