Best practices for selecting an AI chatbot platform

If you’re interested in creating a chatbot solution for your workplace, you will often find yourself considering primarily customer-facing platforms. In this guide we help you find evaluation criteria that help you decide which solution is best to create your internal AI with.

1. Hive bot vs. one-off vs. bot platform

In order to pick a platform or vendor, the first question is regarding to the scope of your project. You will probably fall within one of three categories:

Category A: Hive bot vs worker bot

If you want one central chatbot that users reach out to, regardless of the nature of their question

Master bots are kind of like a concierge in a fancy hotel. Whether it’s theater tickets, dinner reservations or helping you with your luggage, everything can be arranged…

In the same breath, a master bot can help you with whatever you need; whether you want to know “How much leave do I have?” or say “I need a contract template”.

It understands the intent of your request and always knows a correct resource to refer you to; be it a document, a link or a specialized chatbot that deals with your question.

Master bots can either incorporate tools to create specialized sub-bots (for example by importing a Knowledge base) or, alternatively, hand conversations off to bots that are made with other platforms.

Category B: Singular bot

A single chatbot designed to answer questions centered around a single vertical

These one-off chatbots are frequently used for QnA-style chatbots for a specific topic, say leave policies. They are pretty easy to create, as you can draw from a wide range of customer-facing platforms. (Easy-to-follow examples would be here or here.)

This choice is great for companies that are just starting to experiment with bots. However, if you want to incorporate real-life data into responses you may find yourself needing development skills pretty quickly.

Also, while nothing is stopping you from creating just many of these one-off bots, you will have to manage things like user authentication, analytics, and more, all separately for each bot. To avoid your maintenance costs being driven up by this at a larger scale, there is Category C.

Category C: Bot platform

One solution that teams in the organization can use to create a multitude of bots

This is the ‘toolkit’ approach whereby IT picks it’s own platform and provides it to other teams in the organization to make their own bots with.

This allows IT to manage the security and deployment; whilst development can be left to the business side where people have the knowledge about processes, but not often the tech skills to create bots from scratch.

Bot platforms often have easy-to-use AIs that you can create ‘conversation flows’ with and let you manipulate the AI engine through a GUI, too.

_{Editor’s tip:}
You may find that many bot platforms may try and convince you they’re also a master bot product provided you get all your bot needs fulfilled by them.
However, in a future of chatbots there are inevitably going to be chatbots from 3rd parties, e.g. a CRM solution that you acquire may already feature a decent chatbot. So you should check how agnostic your chosen platform is when it comes to handing conversations off to 3rd party chatbots.

Whether you have a decentralized bot platform or not, there are some pretty considerable questions you need to try and solve for your project to become ‘production ready’:

1. Quality: who checks that the bot understands enough of the queries users have and who ensures it gets retrained as people use it? (See 6.)
2. Discoverability: how will the average user become aware that there now is a “Find a legal expert” bot or a “Make a meeting room reservation” bot?
3. Search vs chatbot: when should users consult a bot, and when the Enterprise search or Intranet search? Or should chatbots appear in search results directly?
4. Maintenance: if bots are drawing on APIs to serve real-time data, who will be responsible to debug and fix the APIs when they will inevitably require maintenance? (See 8.)
5. Deployment: will users need to navigate to certain Intranet pages for each bot or will there be integrations for Microsoft Teams or Slack? (See 7.)
6. Governance: if a bot says you can stay at home if it snows, does that count? Who checks when that what bots are saying is correct?

2. NLP considerations

Natural Language Processing is the process by which a user’s question gets processed and matched with a bank of available intents. This process is often done in the cloud. If you allow voice inputs, these are also mostly transcribed in the cloud as well.

If your organization is concerned with keeping things, that employees ask, from leaving the company network, then this will be an important criteria for your choice.

_{Editor’s tip:}
If privacy is a top concern for your organization you should consider Botpress (NLP happens on-site) and for secure voice input a Google Pixel 4 (transcription doesn’t rely on an Internet connection)

Apart from security, NLP systems within a bot platform can also have strengths and weaknesses of their own which you should weigh up with what you need:

• Accuracy: how good is the platform you’re evaluating and how much training data does it require before results are reasonably well distinguished?
• Re-training: are failed user requests collected in a report? And can this data be used to improve the NLP engine, for example by adding them to the AI model? (See 6.)
• Localization: can the AI engine be trained in languages other than English? And can it be trained in multiple languages simultaneously? Depending on that, you might have to create a new chatbot for each use case ✕ language.

3. Conversational vs. Scripted AI

Imagine a customer orders pizza:

Here is what a conversational AI would respond:

But here is what a scripted AI would say:

A conversational AI can – more or less flexibly – infer important characteristics, like the size and toppings the customer selected.

A scripted AI, however, sticks to strict conversational flows. That means, if you haven’t accounted for an option to be entered at the current stage, the AI engine will ignore it, or worse, fail.

In reality, very few AIs are truly conversational. Nevertheless, the platform you select should be able to skip certain questions based on already existing information about the user, e.g. filling out the user’s address based on their profile.

4. User personalization

Not everybody in the organization who asks the chatbot the same question, should get the same answer.

Take “business class travel” as an example: Whether or not someone will be eligible to their pre-departure beverage will often depend on the employee’s seniority level. Your chosen platform should already know the user’s level and not have to ask them for it.

If a platform doesn’t have business logic that can account for such cases, then oftentimes companies decide to just dump the entire policy into the chatbot and let the user figure it out.

That’s a bad idea, at least if you were hoping users are going to come back to this chatbot in the future.

Bots that don’t reduce friction compared to the ‘old’ way things got done, aren’t sticky enough to retrain people’s behavior.

So choosing a platform that allows for easy personalization is absolutely critical to creating a successful AI chatbot for your office.

5. Backup plans

What if the bot is getting confused and needs backup?

Chatbots will be tripped up from time to time, so your bot platform should be able to hand off tricky conversations to a real-live agent.

Ideally, this happens seamlessly mid-conversation, so that the agent can read the transcript thus far (therefore preventing the user from having to repeat themselves).

Prompting a hand-off should happen either:

• as part of a conversation flow (certain paths automatically invoke a real agent)
• if the bot registers an error (say an API times out)
• when the user prompts for it (they may feel they got stuck)

6. AI learning and analytics

The kinds of AI you encounter at the workplace typically are ‘supervised’ Machine Learning algorithms. These aren’t able to intuitively create new connections or self-improve.

Thus, in order to allow the AI to get smarter with time, it will require the help of humans that sift through all manners of reports:

• Usage analytics: what has been requested how often, what requests stall mid-conversation, etc.?
• Failed requests: for what requests did the chatbot come up blank?
• Assistance requests: what topics were users engaging in that made them patch in an agent?
• Modes of use: is the chatbot used a lot on the desktop vs. on mobiles? (If it’s used on mobile a lot, but subsequent forms or bookmarks aren’t mobile-friendly, this could lead to friction in the UX).
• API performance: in which situations were responses delayed because an API that’s part of the fulfillment flow was slow to respond?

Then of course there is the even more valuable category of user feedback. Ideally your platform allows users to vote on every answer the chatbot gives using little 👍/👎 buttons in the UI, alongside an optional comments field.

But if that’s not possible, then another way would be to ask users to fill in a short survey after the chatbot flow has finished.

Your feedback will be one of two categories:

• Positive
  Whenever users indicate something was helpful that is a big, big sign that your chatbot has made something easier compared to how it was. The more positive feedback you get the higher the chances are that user are ‘rewiring’ their behavior to seek out the chatbot again for future queries.
• Negative
  Conversely, if there is any negative feedback either the information didn’t reduce the friction enough or was just plain wrong. Either way, these issues need addressing quickly to avoid users turning away from the chatbot in the future.

7. Channels and deployment

There are obviously many channels and chat applications that various platforms may be compatible with:

• Webchat plugin (for web pages and Intranets)
• Slack
• Microsoft Teams
• Cisco Webex
• Mobile web or native app
• Browser extension
• Desktop application
• Alexa
• Google Assistant

You don’t need your chatbot to be available on all of those, but those that your organization cares about, should be supported.

_{2 Examples:}
1. Let’s say you’re planning a chatbot for your SharePoint users, then you should check whether a web chat plugin is available.
2. Meanwhile, if your organization uses Slack you should check your tool supports Slack’s native Block Kit, while you don’t need to pay attention to how well they integrate with other chat tools.

Then there is a question of deployment: Is your organization submitting custom apps/skills/chatbots to each platform you want to use the chatbot with?

Submissions to ‘app stores’ of chat platform, e.g. Microsoft Teams, can be quite arduous as companies need to support OAuth protocols, meet privacy requirements, provide contact details, as well as pass a ‘user experience review’ from each platform they want to be listed in.

This could spell some upfront work for you. On the other hand, most SaaS solutions will simply feature their own centralized chatbot for most channels. That’s just a generic chatbot from the vendor you can readily add to the instance of your chat tool (say the central Microsoft Teams bot from Digital Assistant).

However the catch is that all the branding will be rigidly showing the SaaS tool’s name with next to no option to modify the experience for your users; just in case you do in fact need that flexibility. (It’s because the vendor similarly had to go through the tough review process to get their chatbot listed.)

_{Editor’s tip:}
Some platforms actually let you go either way, by providing their central chatbot, but also providing endpoints you can consume with your custom chatbot, say for Microsoft Teams.

8. Use cases

What do you actually want to help your users with? Will you make mostly QnA based chatbots where people ask questions and get answers?

Or do you want to offer precise answers drawing from data from your CRM, ERP, Outlook, Helpdesk, etc.?

We made a whole guide about how to best select use cases for your chatbot based on attributing a score. But in a nutshell, you need at least some use cases that are not just generic, canned “copy-and-paste jobs” or bookmarks to places elsewhere in the Intranet.

Drawing on APIs to give instant, personalized answers is a huge factor in how useful your chatbot is going to be perceived. Imagine you could reduce ‘looking up your PTO allowance’ from a 3 minute process to a 10 second process, and user’s are going to love your chatbot.

So that this can be accomplished, you will need the possibility to execute your own scripts and API requests. Some platforms make this easier with out-of-the-box APIs or an open platform to write your own connectors; while other platforms are little more bare bones.

9. Project support

You may be a total AI pro, or may just be getting your feet wet in the field; but either way your project could well benefit from a platform where you can:

• Draw on expertise from the vendor’s developers
• Get help with connecting and debugging APIs, OAuth tokens, etc.
• Receive technical support for error messages

How much developer support, or professional services, you reasonably will require, should inform what platform you decide to go with.

Summary

Picking the right chatbot platform isn’t an easy ask, so if you’re just getting started any consumer-facing platform will be good enough to make first experiences.

But if you’re thinking of going all in, you should have a deeper understanding of your Digital Workplace ‘landscape’ and evaluate vendors based on how well they align with your vision of a Digital Workplace.

The above steps should be a good approach to evaluating chatbot platforms and how suitable each one will be to your needs. Do you have any other areas you think others should watch out for? Then please leave a comment below.