What I Learned by Developing a Top Alexa Skill.
Top 1% of Skills
There are over 160,000 Alexa skills available worldwide. Amazon does not publish rankings or engagement scores, but reviews are visible on the site. Voicebot estimates that less than 1% of skills receive more than 100 ratings.
My Skill
Tarot Reader is a game / educational tool. Users can have their Alexa explain specific tarot cards while they practice along with their own deck, or they can ask Alexa to do a reading for them.
Tarot Reader currently has:
- 2356 reviews
- a 4.2 out of 5 rating (highest ranking for all Tarot games. Next one is 4 out of 5).
- the top (non-sponsored) result spot for “tarot” in the Alexa Skill Store.
What I Learned
Voice Is Still King
Four months ago (Sept 2024), I migrated my skill from voice to multi-modal voice and touch screen. I was eager to see what impact this would have on usage.
Less than 50% of my traffic comes from a screen-enabled device. The number one device accessing my skill is still the Echo Dot. In fact, the gap between Echo Dot and Echo Show grew despite adding screen support for the Show!
After four months, usage is up 33%⬆. This seems to be driven less by Echo Show adoption and more by growing and retaining Alexa’s driving more user engagement per customer.
Customers are up 12%⬆. Utterances per session are up from 4.5 to 5.8 (29%⬆), and daily sessions per user are up from 1.5 to 2.1 (40%⬆)
Support Routines
75% of my traffic comes from users who have added the skill to one of their routines, which drives higher user retention and engagement.
75% of the users who directly ask Alexa to launch the skill are new customers. Direct invocation seems to be more for people trying out the skill and casual users.
Tarot Reader’s 6-rate fluctuates between 36% to 41%, which shows a high level of stickiness supported by routines.
Interaction Model Matters
While migrating to the new multi-modal models, I used Alexa’s developer tools to help improve my utterances, which are the verbal snippets that Alexa turns into commands for my skill.
The combination of NLP profiling, using slots and aliases, and the new conversational models improved my interactions and reduced the medium to low-quality confidence utterances from 1.3% to 0.06% (54%⬇), which means half as many people getting the dreaded “I’m sorry, I didn’t understand” response.
Pay attention to feedback
A large part of improving my interaction model was based on user feedback.
Customer reviews are helpful, but even with over 2,000 reviews, the feedback is sparse. Do the obvious, read your reviews, and address issues.
Less obvious is your utterance data. Alexa keeps a log of the phrases users ask your skill. I discovered reasonable user responses that I hadn’t accounted for, synonyms I added to slots or intents, and phrases that indicated where something wasn’t making sense to the user and where I could do a better job.
Leverage the Developer tools from AWS
Use the Alexa Skill Coach to discover areas of improvement. As AWS adds more features to Alexa and their skill models, the coach will point out which ones can improve your skill. It spots where you are not using best practices and gives you a practical set of action items.
Thanks
Finally, a large part of my skill’s success is that users engage with it and seem to like it. I appreciate all the feedback and am thrilled that out of 160,000+ skills, so many people have made mine part of their routines.
As an engineer, seeing people use your code is probably one of the best feelings you can have.
As for the future, only Tarot Reader knows 😂
