get in touch

Our MAU was flat for three months, and the roadmap wanted more AI

For about three months, our monthly active users barely moved. Not down, not up. Just flat.

At Envision we built AI-powered tools for people who are blind or have low vision: an app and smartglasses that turn the visual world into speech. AI was moving fast, and so were we. So when growth stalled, the instinct in the room was clear: ship the next AI feature. Something new to talk about, something new to try.

I wasn't sure that was the problem. So before we agreed on the next big thing, I went looking for where people were actually getting stuck.

# Data isn't always there. When it is, use it.

I'm a user-centered product manager. Most of my decisions start with talking to people, and in a startup that is often all you have. But we did have usage data, and a flat line is a question worth asking it: are fewer people coming in, or are the people who come in not finding enough reason to stay?

For the numbers I used Mixpanel and PostHog. I built funnels in both to see how people actually moved through the app: where they started, which features they opened, and where they dropped off. With that, I looked at three things side by side:

  1. Usage per feature. Which features do people open, and how often?
  2. Repeat usage. Of the people who tried a feature, how many came back to it the week after, and the month after?
  3. What users and developers told us. Conversations with users, and with the engineers who saw the support issues and edge cases up close.

None of these on its own tells the full story. Together, they did.

# Two patterns that didn't match the roadmap

A core feature was used constantly, and asked about constantly. Scan Text let you take a photo of a document. We ran it through OCR and opened the text in a reader, so you could listen to a letter, a menu or a form. It was one of the most used parts of the product, and it was very accurate, because it ran on a powerful model online. But that also meant it only worked with an internet connection. And in conversations, the same request kept coming back: can I use Scan Text offline? High usage hid the problem. People used it because they needed to, not because it fit every moment they needed it.

The newer AI features got a first try, and then not much more. People were curious. They tried them once or twice. But repeat usage was low compared to the core features.

Put those two on one picture and the roadmap looks different:

Usage versus satisfaction, illustrativeA two by two chart. The horizontal axis is how often a feature is used, the vertical axis is how satisfied users are. Scan Text sits in the high usage, low satisfaction corner. The new AI features sit in the low usage area.how often it's used →how much people like it →keep it healthyfix this firstnice, but nicherethink or waitScan Textnew AI features
Illustrative, not real data: how the two patterns looked next to each other.

The thing most people depend on was the thing we were least likely to touch, because it wasn't new.

# Saying "not yet" to more AI

Some of the team wanted to keep pushing new AI capabilities. I understand why. AI was what made Envision special, it was moving quickly, and honestly, I love AI too. But a new capability doesn't help much if people keep running into the same wall with the one they already rely on.

So I didn't argue against AI. I argued about order. I brought the usage and repeat-usage patterns together with what we heard from users, and made a simple case:

  • the people we already have are telling us what's in the way, in data and in their own words
  • a new feature that gets tried once doesn't fix a flat line
  • a core feature that works in more places, every day, for the people who rely on it might

It took a few conversations. In the end we agreed to put improving the core experience ahead of the next round of AI features.

# The easiest feature on the roadmap

Here's the funny part. We already had an offline text feature. Instant Text read text out loud in real time, straight from the camera, without an internet connection. Scan Text and Instant Text used similar technology. So my question was simple: if we can do it offline for Instant Text, why can't we do it for Scan Text?

Two Envision app screens side by side. Left, on pink: Scan Text, with the headline 'Read complex documents and letters', a photo of an open book being read out loud. Right, on yellow: Instant Text, with the headline 'Turn any text into speech, instantly', the camera pointed at a 'Neighbourhood watch area' sign that is read out loud.
Scan Text (left) read full documents with an online model. Instant Text (right) read short text live from the camera, offline.

We had never built an offline Scan Text for a reason. The on-device recognition wasn't as accurate as the online model, and we assumed people wouldn't accept a less accurate version. But nobody had actually asked for perfect accuracy offline. They asked for Scan Text that works when there's no connection.

So I pushed for a "fake" offline Scan Text. When you're offline, you get the same Scan Text flow: take a photo, get the text in the reader. Under the hood it uses the on-device technology we already had. It was literally a capability we already owned, repackaged in a slightly different UI. Not a big build at all.

From the very first iteration, people were stoked. Not because it was technically impressive, but because they felt heard. They had asked for this again and again.

We tracked satisfaction in two ways: thumbs up and thumbs down in the app, and a survey we sent before and after the release.

  • Thumbs up versus thumbs down went from 80/20 to 85/15.
  • Customer satisfaction in the survey went from 4.3 to 4.6 out of 5, between September 2022 and December 2022.
In-app thumbs up versus thumbs downBefore offline Scan Text, 80 percent of ratings were thumbs up and 20 percent thumbs down. After, 85 percent up and 15 percent down.In-app thumbs up vs. downBeforeBefore: 80% thumbs upBefore: 20% thumbs down80% up20%AfterAfter: 85% thumbs upAfter: 15% thumbs down85% up15%share of all ratings Customer satisfaction survey, out of 5Average satisfaction was 4.3 out of 5 in the September 2022 survey and 4.6 out of 5 in the December 2022 survey.Satisfaction survey (out of 5)Sep 2022Sep 2022: 4.3 out of 54.3Dec 2022Dec 2022: 4.6 out of 54.605
Before and after offline Scan Text. The survey ran in September 2022 and December 2022.

# Perfect world, real world

In a perfect world, it works like a textbook. You have clean data, you find the insight in it, you ship an improvement, and then you isolate exactly that one change and measure how much it moved the needle.

In the real world, especially in startups and scale-ups, that's almost impossible. There's rarely enough traffic, time or tooling to isolate one change from everything else going on. The numbers above are real, but they're not a lab result. Other things changed in the same period too.

So you look at different things and see if they point the same way. Sometimes that's a survey or a thumbs up button. But most of the time it's simply talking to people: users, colleagues inside the company who know the product and its problems inside out, and people at other companies or stakeholders who've seen similar problems. None of it is perfect proof. Together, it's usually enough to make a good decision.

# What I took from it

  • High usage isn't the same as satisfaction. People use things they need even when they're frustrated by them. Usage tells you where it matters; conversations tell you whether it's working.
  • Repeat usage is a more honest metric than launches. A feature that gets tried once is interest. A feature that gets used again is value.
  • Data is a way to disagree well. It moved the conversation from "I think" versus "I think" to a shared picture of where users were stuck.
  • Data doesn't replace talking to people. The numbers showed where to look. The users explained why.
  • Don't assume what users won't accept. We never built offline Scan Text because we thought lower accuracy would disappoint people. They just wanted it to work when they were offline.
  • The easiest features can have the most impact. This was one of the smallest things we shipped, and one of the most appreciated.

In startups, scale-ups and bigger companies alike, data isn't always available, clean, or complete. But wherever you have some, it's worth using it to find the bottleneck before deciding what to build next. Sometimes the most impactful thing on the roadmap is the boring one.