6 secrets of successful chatbot strategies

What do NASA, Capital One and Verizon have in common? They’ve all built chatbots. And despite the organizations’ obvious differences, their early chatbot development processes have proved to be relatively similar, providing valuable insight into how organizations should build and onboard bots .

“There are no bot standards,” says Nathan Shedroff, executive director of Seed Vault, an open source developers’ community currently developing chatbot guidelines. “Imagine it’s ’95 and it’s the very beginning of the web, but there’s no HTML. There’s no standard that anyone can learn and start building websites,” he says. Every coder is on their own — not just for building, but for implementation, too.

Here, IT leaders from NASA, Capital One and Verizon — who each received a CIO 100 Award in IT Excellence — discuss the critical chatbot development and deployment problems. The following six tips will help your organization onboard its first chatbot.

Narrow your target audience and need

Before coding that first line, ask if you even need a chatbot. What bots do and what the public thinks they do aren’t always the same, so the business unit requesting one might not understand what they’re asking for. To certain end users, chatbots do everything from take lunch polls in Slack to rig presidential elections. “Sometimes folks … have very grand visions about all the many things that the intelligent assistant they’re building is going to be able to do,” says Ken Dodelin, vice president of conversational AI products at Capital One.

So instead of treating chatbots as a catch-all, closely examine the need. “We don’t build them in a vacuum,” says Tom Soderstrom, IT chief technology and innovation officer for NASA’s Jet Propulsion Laboratory. “On the contrary, it’s really to solve [a user’s] problem or to make them more productive.” At NASA, for example, one chatbot helps Deep Space Network engineers locate antennas more quickly. But at Capital One, customers use a bot to check bank balances. “What’s the goal here?” Soderstrom asks. Chatbots are interfaces that people use to navigate data. If that’s not your goal, a bot’s not the right tool. Whatever you want to do, Dodelin says, “When you can go more narrow, you can have more success.”

Narrow purpose doesn’t always mean narrow audience, though. At Verizon, Vice President of Digital Ashok Kumar says the company’s My Verizon App and My Fios App chatbots are two instances of the same core engine. This can cause natural language training problems, he explains, as certain words like “device” may have different meanings for Verizon mobile versus Fios television customers. But the business need — to make it easier for people to troubleshoot that device — is the same. There’s no reason the same bot can’t address both.

As Soderstrom explains, “An intelligent digital assistant is something that helps the person, so there may be a different one for a scientist, a different one for an engineer, a different one for a business person, [and] a different one for marketing.” To make building out for multiple audiences easier, NASA uses a proprietary template and a consistent user interface whenever it can.

Choose the right platform

Once your target user and target problem have been determined, you’ll need to pick the platform where your users prefer to connect. This could mean developing a chatbot that responds to multiple forms, from speech to text message to the web. Note what’s best for your target audience may not be what’s best for your team.

NASA developers, for example, test in Slack, but “we also create a website so [users] can go straight to the URL because not everybody uses Slack,” Soderstrom says. NASA changes platform from bot to bot as needed. In all, the Jet Propulsion Laboratory alone has roughly a dozen chatbots “accessible by speech, by typing, by texting, [or] by website,” he explains. “We want them to be able to get the answer from wherever they are.”

The bot connecting citizen scientists to information about Mars, for example, is actually an Alexa app. Voice assistants typically aren’t thought of as chatbots, which stereotypically engage through written language, but Soderstrom considers the technologies one in the same: “The idea there is to make it as easy as possible for the humans interact with. And if you can only speak to it, that’s your only option. If you can speak or text or type then you have many different options — whatever is easier for the human.”

Define what determines success

Next let’s talk numbers: What return on investment (ROI) do C-suite or other managers expect from your chatbot project? A bot’s financial value can be remarkably difficult to prove. Capital One piloted its balance checking chatbot over text in March 2017 and added an Alexa skill one year later. But Dodelin still doesn’t have HR efficiency or customer profit numbers to share, hoping those will be ready by fall. The metrics that matter more right now, he explains, are the ratings customers give the bot online, whether continued training makes the chatbot better, and whether the marketing department is comfortable promoting it: “How good do we feel about the experience? Are we at a high enough success rate to really lean into it?”

Engagement is also the measure of success at NASA, where Soderstrom says chatbots are a great way to get “end user return on attention, not return on investment because there really is very little investment.” NASA builds quickly: The Deep Space Network bot was designed, demoed, built, and tested over two days. Calculating ROI, he continues, “would take way too long and it would become too bureaucratic. The advice is to devise this structure to look for moments of engagement with the end user community, and when you see one, jump on it really quickly and try it.”

Optimize your build process for business value

When that moment comes, Soderstrom says don’t delay — just go for it: “If we try to up front say, ‘We’re going to make this thing perfect. We’re going to think through all the data that that thing is ever going to access and then we’ll build a big requirements document and we will build it and we’ll pop out a year later and everybody will be super happy,’ it doesn’t work that way.” Instead, he says build fast, fail fast, adding that NASA treats each chatbot project “as a small startup. They just go off and do it as quickly as they can.”

Granted, many of NASA’s bots are employee-facing projects that users or developers request to solve internal business problems. For customer-facing bots, Verizon’s Kumar recommends a longer approach: Development for Verizon’s Mix and Match bot — which guides consumers through a company data plan of the same name — took months. Developers trained the chatbot while Verizon developed the actual consumer plan. Then marketing got involved. “Normally when you build a product, you work in parallel to build a website,” Kumar explains, as the plan, the chatbot, and marketing collateral were simultaneously released.

Verizon projects also take longer because Kumar’s team builds them principally from scratch. “There are people who are core AI developers, but then there are engineers who are designing the actual conversation,” he explains, as well as conversation designers and “teams of people who retrain the bot” after people enter new expressions.

Soderstrom’s team is much smaller. “We have just a few people who are really experts at cranking these out,” he says, and all of these engineers have other work to do. As a result, NASA does the chatbot design work itself but uses Amazon Lex for the natural language understanding component.

Consider security

There’s at least one area, though, where Soderstrom insists on taking time internally: Security. As Seed Vault’s Shedroff points out, “None of this is gonna move forward unless we can trust these systems.” So Soderstrom loops in NASA cybersecurity after concept but before development. “Cybersecurity is always part of these things because they can shut it down at the end,” he explains.

Granted, as a government agency, NASA has no choice. “We have to show compliance to certain rules and regulations that private industry doesn’t have to,” Soderstrom explains, “so when we experiment with these things, we keep that in mind.” If a chatbot “isn’t secure enough or if it isn’t compliant,” he adds, NASA can’t use it. Similarly, Dodelin says that as a bank, Capital One presented its bot to regulators before release.

Even if you’re in a less-controlled industry, Soderstrom says, “You still need to be secure because if all of a sudden you leak everybody’s information, you’re out of business.” So think about authentication, where data will be stored, and if training is necessary to keep users from entering personally identifiable information (PII) that they shouldn’t. Also consider limiting the bot’s functionality as a way to keep it safer: For example, Capital One’s bot lets customers check balances — not transfer funds.

Tweak and train continually

After solving security and the who what hows of the business side, there are the better-known concerns developers typically associate with chatbots, like ongoing training or bias elimination — not just in how the bot handles data but also in how it’s marketed. For example, Capital One’s bot is intentionally gender-neutral to avoid stereotypes about assistants. Bias can be regional, too, as any Southerner who’s ever used a voice app well knows. Different parts of the country use different expressions, as do people from disparate social classes or educational backgrounds, men versus women. This is part of what Kumar’s training team deals with at Verizon, why it’s so important, he says, for developers to consistently add new terms. “That’s the constant training which happens,” he says, describing chatbot development as “an ongoing process. It never really ends.”