Sunday, April 29th 2018
Do you develop bots for Cisco Webex Teams (formerly Cisco Spark)? Do you ever wish that there was a way to create a set of regression tests to ensure that your bot behaves consistently given the same input? Or maybe you wish that there was a way to speed up your iterative development/debug process when you are working on a bot response to a complex set of input? Or perhaps, you just wished you could do some bot development work when you were in a place that didn’t have good internet connectivity?
If so, I’m happy to let you know that the Webex API Emulator, originally created by Stève Sfartz, API Evangelist at Cisco DevNet, can help make your wishes come true. The emulator is a simple node.js project which you can download from GitHub and run locally. It loads a tokens.json configuration file that tells it about the users that can interact with it. This file, which is editable by you, allows you to create “authorization tokens” and “personIds” for your bot, your tester and any other “people” that might be added to spaces as part of your tests. Think of it as your own, local, common identity server.
Let’s start making those wishes come true by dealing with the last wish first, the idea of doing bot development when you are offline. Once the emulator is up and running you can use a tool like Postman to send requests to the Webex API Emulator to do things like create spaces, add a user (or bot) to a space, or post a message to the space. The emulator also supports the webhooks API so if you configure your bot to talk to your locally running instance of the emulator, your bot will get notified when it is added to a space or when a message is sent to it. Once you can can send a message to your bot via the emulator, you can open up your favorite editor or debugger, and you can start stepping through your bot’s responses to these commands, building and debugging responses to these inputs, all without an internet connection.
We’ve glossed over a few things here, but we’ll get back to it. Assume you are now able to send commands to your bot via Postman and see the Webex API requests that your bot makes in response, all offline. Handy for working offline, right? But once you get used to this, you may start to realize that running in this mode simplifies your development motion even when you DO have an internet connection. Imagine your bot needs to have a particular response in certain specific, potentially complex, circumstances. For example if you send your bot a message like “who from my company is in this room”, you might want it to list people from your company but also be sure that it skips people from other companies. You’d want to make sure that the bot behaves properly in circumstances where there are no people from your company, some people from your company, no people from other companies and some people from other companies. To test this you’d ideally create a several spaces and add several combinations of people to each of those space, and then type in your requests in all those spaces. Doing this manually in the Webex Teams app can be time consuming and it might not make your test subjects very happy to keep getting added to different spaces as you test different variations!
With Postman and the emulator, you can make the API calls to create the spaces, others to add each of the users, and another set to send in the commands. These can then be saved in a postman collection and run with a couple of keystrokes. Your bot didn’t respond as you expected? Simply stop the emulator and your bot and restart them (in that order!). This brings you back to a pristine environment where no spaces have been created yet. Alternately, have another set of commands to delete the spaces and run your test again as often as you like, until it works perfectly for all the variations that you care to test for!
Hmm, as we start building up these sets of API calls to emulate user input that we want our bot to respond to, we are starting to get close to what some might call a regression test. There is still one problem though, a true regression test should also validate the responses from your bot. Luckily the Webex API Emulator also supports “bot test mode”. In this mode we can instruct the emulator to look for bot responses to certain input and provide that back in its responses to us.
Now, let’s get a flavor for how bot test mode works.
Before we could run any tests we needed to start our Webex API Emulator in “bot test mode” and start our bot using an API token specified in the emulator’s tokens.json file. When we’ve got the the emulator and our bot running properly we’ll have some terminal windows that look something like this:
Once those services are running we can start sending in our test input. The first couple of test cases don’t generate any bot reaction, they simply set our state.
Now we are ready to send a message to the bot and check if it responds as we expect. Here’s what that request looks like in postman:
This is a fairly standard POST to the /messages endpoint with a body which specifies the roomId of the previously created space (stored in a postman environment variable), and includes some markdown which mentions our bot and sends the command “/leave”. (Note that all of the environment variables used in these test cases are explained in more detail in the bot-test-framework-example readme). Now let’s look at what makes this request a true “regression test”.
Here are the headers for this request:
The Authorization and Content-Type headers should look familiar to anyone who has used postman to send requests to the Webex API, but there is a new header “X-Bot-Responses” as well. When the emulator sees a request with this header, it understands that we expect the bot under test to make two subsequent requests in response to this command. It will “intercept” the original response to this request and “hold” it until it gets the two bot requests and then build out a consolidated response that looks something like this:
This response contains two objects:
Finally, notice the “Test Results” circled in the image above. For each step in our test suite, we created Postman test cases to validate that the API response was as expected, and when appropriate, that the bot responded as expected as well. We won’t delve too deeply into how these tests work, but a full set of example postman tests, including the test cases are provided in the bot-test-framework-example.
While there are a lot of moving parts here to make this work, we think it can ultimately speed up your bot development process. I’ve been using this for a few months now and the regression tests have helped me discover bugs in my bots that I would have depended on users finding for me without it. I also love being able to work on a bot while on an airplane with sketchy internet connectivity!
If you decide to give the Webex API Emulator and bot-test-framework-example a try, we’d love to hear how it goes. git Join the Ask Webex API Emulator space on Webex Teams and give us feedback. Right now this is just another Webex API developer community supported project, and we hope this post makes it clear that tools like this only happen with an active and engaged developer community who make contributions like Stève, Victor and Nick have done. Who knows though, if usage of the emulator really takes off, we may someday be able to turn it into a full fledged service provided by Cisco.
Let us know what you think!
-- JP Shipherd, Sr. Product Manager