[PM] Can we make Yelp better?

For this post, I’m going to discuss a feature I’d like to be added on the Android version of Yelp.
From what I understand, the Android team on Yelp is short-staffed so updates are rolled out somewhat slowly.

Disclosure: I’m not affiliated with Yelp.
BIG SIDE NOTE: This is not how a PM process works. This is just me brain-storming so there’s a lot of babbling. An important skillset for a PM is to be extremely concise.

What should I build?

Normally, the rule of thumb for product managers is that we don’t build features that we want. We speak to customers, users, businesses, and our stakeholders (which sometimes it could be internal employees if we were building internal features or tools). We gather data and analyze metrics to verify what we need. We combine quantitative and qualitative data to build and contribute to the company’s vision.

The main idea of this post is to figure out what the app needs, validate the feature, create it and roll it out.
From my experience an APM and PM, the traditional methodology of product management actually works well, so I’m going to follow them.

Below are the steps I’ll be taking

Competition and Market Analysis
Understanding Customers (Interviews, Personas)
Metrics (which metrics do we look at)
Epics & User Stories
Wireframing (UI/UX)
Prioritization and Roadmap

Competitive Analysis

Yelp has some competition for online platform that provides restaurant reviews.
However, from Yelp’s SEC filings, we can see that non-restaurant reviews are also a huge part of Yelp.

If we include those, the competition would be endless (For example, we’d have to include websites like Groupon and Facebook).
I don’t work at Yelp so I don’t have enough data to do a comprehensive competitive analysis.

For the purpose of this post, I’ll stick with Google Local Guide as a valid competitor in the restaurant review segment. (Foursquare, Zagat, and TripAdvisor did come to my mind, but I decided not to since their strategies change too often).

Categories	Yelp	Google (Google Maps)
Product Core: – Who is in charge of making the product – How good is the product team	Yelp has a dedicated Android Team. From what I know, the team is small.	Most likely has a pretty big team working on it. Lots of product managers, software engineers, etc
Size of the user base	10 million+ playstore downloads	1 billion playstore downloads. (this isn’t a fair comparison because Google Maps is a multi-purpose app)
Design	Steadily improving. Design is not top tier, but average.	Google Maps is a well-designed app but the restaurant guide integration is subpar.
Brand: -How highly regard is the brand?	The Yelp brand for restaurant ratings is huge in certain parts of North America.	Google has an extremely strong brand and marketing team and they have been recently promoting their local guide system.
Speed: -How quickly can changes be made	Small team but company is smaller than Google so changes are made at a decent pace.	Large team but company is huge so changes are frequently made but usually in smaller increments.

This competitive analysis shows that Google is catching up fast but somewhat in a different direction.

Google’s strategy is simple: billions of people use Google Maps so why not integrate the system into it?

It makes sense and it’s brilliant: people will look up a restaurant using Google Maps to find directions but when they do, they see the rating of the restaurant. The rating is useful to the user (provides value for the customer).

However, unlike Yelp, a user won’t open Google Maps to simply view a rating of a restaurant (or rate a restaurant for that matter).

Google has the almighty Google search for that! And rightfully, Google Search is a dangerous competition to Yelp.

So doing a competitive analysis with Google Maps, a pseudo direct competitor, isn’t 100% valid.
However, by analyzing the two different mobile apps, we were able to see that Google has multiple ways to provide their rating system to users. Strategy analysis is key in product management!

Another danger is that Google can technically manipulate their search ranks to push Yelp listings lower. Since we’re only concerned about the mobile app for this post, this isn’t too relevant but it’s a (dangerous) food for thought.

So what can Yelp do better using the resources they have to have an edge over Google? What do the customers think?

Understanding Customers

“The community is the product and the product is the community”

Yelp’s elite squad is an incentive program that rewards “elite” members for being active on Yelp.

For more details, this article highlights what the program is about and how it started.

I’ve been a Yelp elite for a few years and made some friends through the elite events gratuitously hosted by Yelp.

This gave me an opportunity to conduct “customer” interviews.

Some may say that it’s a mistake to interview elites because elites are all power users. Understandably, customer interviews should never be based on ONLY power users.

In fact, if your goal is to generate activation/retention metrics, power users should usually be excluded.

However, not every elite member is a power user. Some newer elite members actually did not understand how the app functioned and only knew how to write reviews and look-up restaurants. Considering Yelp has a lot more features, this was shocking! (Here, I’m assuming the traditional definition of power user – a user that has the “desire to make the most intensive use of computer programs.”

And yes, there’s some confirmation bias here, and I’m overvaluing my own experience, but there’s enough people I’ve observed over the years at elite events so I think there’s some statistical significance. So I think interviewing elite members was justifiable. It was my only option anyway…

In addition, I will look into some social media to check some feedback.

I gathered data from:

500 Playstore reviews
100 posts from Quora
100 posts from Reddit

As mentioned above, I also interviewed elites, using Google forms, and asked feedback from:

200 Elite members that use an Android devices

This took about 2 weeks (elite members were slow to respond to my Google form due to the most people being unfamiliar with the platform. In hindsight, asking by email would’ve been better). Anyway, the result was well worth it.

Note that this was a pseudo-exploratory and pseudo-satisfaction oriented interview.

Below is the summarized result:

The result was astonishing. The data suggests that most complaints were around keywords like “friends” and “friend” activity.
I also created a word cloud and generated it from the additional comments section:

This shows that SOCIAL is huge for users and is a common complaint. We need to find a way to somehow improve the social aspect on Yelp. What kind of feature would it be? Perhaps allow users to engage with each other more socially? But how?

This data will significantly help a product manger come up with an idea to improve some metrics.

Metrics

So what metrics do Yelp care about? An accurate answer would require some internal data. However, we can make an intellectual (data-driven) guess.
Most people are aware that Yelp’s sales team is very aggressive to small businesses. That’s because their business model is B2B (yes I’m aware that, Eat24 has some B2C aspect but that’s irrelevant).

In terms of their restaurant rating business, Yelp mostly wants to improve engagement, retention and task success.

For engagement, Yelp’s goal, for the B2C sector, is to maximize the number of reviews, photo uploads and in general, social activities (liking photos, complimenting others). This is easily quantifiable. Retention is easily measured by measuring people that come back to the site to read reviews, and do some sort of activity mentioned above. And lastly, task success is the average time to open the app and check a rating for the website. Yelp wants to improve this time.

I omitted adoption because Yelp already has a large user-base. In this case, adoption would be a user signing up on Yelp to write a review. This isn’t a good metric to measure because A LOT of people sign-up on Yelp to write one review (usually where they have a horrible experience and give someplace a one-star) and never review again. I’m sure Yelp has the exact data of people with 1 review and 1 star review, but I’ve personally seen a lot. (Side note: There’s also probably a lot of accounts with one five-star review and most of these people are probably business owners haha).

For the usually measured happiness metric, it’s not relevant but we can check the playstore and NPS rating if we have to, but Yelp never really bothers me to rate the app so I’m guessing Yelp actually doesn’t care too much about this metric.

Anyway, the above metrics do not directly increase Yelp’s revenue. But it increases value of their platform, and in turn, increases their the chance of making sales.
Our change will measure these metrics. Yelp uses Amazon (A/B Testing, EMR, Redshift), Google Analytics, Optimizely, so everything is trackable.

Epics & User Stories

FINALLY. We will get to the feature. It’s my fault for not disclosing it earlier (and in the usual process, you definitely should, especially after speaking to customers).
An epic would be some sort of grouping of features or functionalities that we want to build.

The epic for this case study would be: “Improve the social aspect of the Android app of Yelp”
Usually an epic is accompanied by a summary of features we want to build, product requirements, design requirements (sketches, prototypes) and engineer requirements (technology).

Since I’m not working on this feature specifically, I will omit these but if you’re an aspiring PM, you should definitely learn about the above.

Now the user stories are important for this, and I used Xtensio to create a persona. I will keep it brief.

Users want to be able to go to events so that they can meet new people that I can dine out with!

User Stories:
1. As a user, I want to know when events are being planned so that I can check and attend them
2. As a user, I want to know who is attending the events so that I can make my decision to attend the event myself.
3. As a user, I want to know when certain people are attending the events so that I know which events are worthwhile.
4. As a user, I want to know what kind of users are attending certain events so that I can meet similar people.

This user persona helps us relate to the user. In general, the perception was that non-Elite users want to find ways to engage more socially on Yelp. Elite events are a great way to meet people but what about people who aren’t elites? How do they meet people?

I think we can utilize events better. Yelp has events that are hosted by members that aren’t for elites (user-generated content). Some are just as fun as elite events! But did you know it even exists? We think the visibility of this menu is way too low and it’s not marketed well enough.

So what if we improve events in a way that’s more socially engaging? People could be afraid to attend certain events if they’re unfamiliar with the attendants.
As we created the user persona above, in the end, I think the best way is improve “Events” and the metrics surrounding it that we discussed earlier.

Some features that would fit into the user story would be:

Add notifications when there are events near the user.
- Allow the user to control what events will trigger the notification, modify the distance ,etc
Instead of “interested,” change the wording to “confirmed.” (I hate “interested” or “maybe” RSVPs on Facebook. If you’re not sure, don’t RSVP. If the event requires an early headcount, shouldn’t you make the decision to make it easier for the host anyway?)
Allow users to add some optional comments when RSVPing – make the attendees more personable to others. Right now, there are a lot of users with 1~2 reviews using the events feature, and a lot of them don’t even have a picture.
Use the existing “Follow” feature and add “Events” as part of the notification on the mobile app.

Wireframing (UI/UX)

A lot of the features to be built are not extremely challenging in the perspective of an engineer and designer.

However, I think “Events” could possibly use a revamp in design because the current design is not very noticeable for the average user. A lot of earlier problems presented by users actually are solved by the “Events” page but there’s a lack of awareness. (Tip here: when talking to designers, present the problem and not the solution. They are usually much better at designing than you are).

Since this is just my personal blog, and I’m not a designer, I’m going to omit the re-design of the Events page. (SORRY)

You can also always use tools like Balsamiq and InVision to wireframe / prototype.
For this case, I could also just use something simple like Axure or Sketch.

I would also work alongside a designer in revamping the events page to something with more visibility, following best UX practices.
Side note: one of my favorite apps is “HotelTonight” and I actually draw a lot of design inspiration from it.

Prioritization and Roadmap

Does this add business value? I think not too much. Does it add user value? I think it does by a lot. Cost? I’m not too sure exactly but I would say it would take about 2 weeks to fully implement and test so I would roughly estimate it to be about $50,000.

That’s definitely enough to prioritize this feature, but if there’s something that has a higher (user value+business value)/(cost) then this could drop down a bit in terms of priority.
Is this a “Must” feature? Probably not. But I think it’s a “Should” feature which makes it pretty significant.

As standard Yelp practice, I’d probably roll this feature out slowly, using A/B testing to carefully track metrics and usage. Yelp doesn’t usually advertise new features like this too much but if the revamp of the design does happen, they should post all around social media, and their own blog and mention the reasoning behind the changes. (And of course press release).

In the long term, we can track how many users are actually using events and see if it increases revenue (data science).

That’s it.

Thank you reading this post. A lot of it is me just brainstorming and babbling my thought-process. You could say it’s a stream of consciousness.
I love Yelp as a product, and I know I diverged a lot from the original path but I truly believe that Yelp needs to improve its product to stay competitive against others, especially Google.

It’s almost silly. I could’ve just said “Yelp should improve events like this” from the start. But this was also supposed to be educational for aspiring PMs. And as PMs, you need to truly understand the “why” of everything. why why why why why why!!!

Anyway, Google is scary. Beware Yelp and good luck.