Following the insightful article by The Next Web (Google Play Store Review Botnet) that uncovered a particular burst in irrelevant positive reviews for some huge apps, the yellowHEAD App Store Optimization team set out to investigate further and explore the implications for ASO.
What was reported in The Next Web?
Between mid-March and mid-April, there was a huge spike in what seems to be fake 5-star reviews in certain huge apps, including WhatsApp, Gmail, Facebook and Facebook Messenger. These reviews are extremely suspicious since they address topics that are completely unrelated to communication apps, such as games. Moreover, several reviews mention Kindle, which is also very odd since those were left on Google Play. The Next Web hypothesized that those were caused by bots in an attempt to bypass the Google Play reviews filter.
We felt that there’s a bit more information here that was yet to be discovered and decided to have another look for ourselves. This blog post will include some additional findings we have seen that might shed some more light on this mystery.
Here’s what we found…
1. MORE THAN A ‘GAME’
The Next Web’s article mainly looks at the frequency of the word GAME in the reviews, illustrating just how irrelevant the reviews are for apps like WhatsApp, Chrome and other tools that are definitely not games (hereon in we’ll refer to these as ‘The Communication Apps’). We took this a step further and checked a more expanded list of game-related keywords like SLOTS, BINGO, POKER, etc. We found quite a high frequency overall for these words as well.
- Each of the above keywords in reviews follow similar patterns in frequency across the apps – i.e. they coincide in time period
- These reviews were mainly present among big apps in the Communication/Social Media categories, though many others did not experience the phenomena (e.g. Waze, Anti-Virus, Spotify, Netflix, etc.)
TWITTER – Frequency of “SLOTS” in Written Reviews
TWITTER – Frequency of “BINGO” in Written Reviews
INSTAGRAM – Frequency of “SLOTS” in Written Reviews
INSTAGRAM – Frequency of “BINGO” in Written Reviews
GMAIL – Frequency of “SLOTS” in Written Reviews
GMAIL – Frequency of “BINGO” in Written Reviews
2. BRAND NAMES MENTIONED IN REVIEWS – APPS WE KNOW DON’T USE BLACK-HAT
Initially, TNW reported that the fake reviews mentioned the irrelevant word “games” and therefore were both suspicious and generic. While the reviews are in fact very suspicious, they are often far from being generic. Among these reviews, we also found a high-incidence of app brand names being mentioned. What surprised us was that some of the brands mentioned are yellowHEAD ASO clients, so we know for a fact that they do not practice any black hat techniques – neither review incentivizing nor the use of review ‘bots’. [Hereon in, we’ll refer to the games mentioned in suspicious reviews as ‘The Gaming Apps’]
3. NO CHANGE IN THE GAMING APPS’ REVIEWS
We didn’t see any significant change in reviews for the Gaming Apps during the time period where we saw the increase in the Communication Apps’ reviews. This strengthens our assumption that the suspicious reviews were not initiated by the Gaming Apps.
4. LACK OF REPETITION WITHIN GOOGLE PLAY
We also didn’t notice any repetition of reviews from the Gaming Apps and the Communications Apps. This leads us to believe that the reviews were coming from elsewhere and were not a result of any involvement from the Gaming Apps mentioned.
So where are these reviews coming from? Are they bot-creations as The Next Web theorized? Drumroll please…
5. DEFINITE REPETITION OF REVIEWS FROM AMAZON
Where we saw mentions of specific Gaming Apps in the reviews, we checked them in the Amazon Store (since The Next Web noticed such a high incidence of ‘kindle’ mentions). And BINGO! We found exact matches of the recent reviews in Google Play for Communication Apps like Twitter, Gmail, WhatsApp, Facebook, Instagram, Skype and Messenger that already appeared in reviews for the Gaming Apps in the Amazon store.
- The dates are quite distant and in some cases we even saw that a four-star review from 2 years ago on Amazon for a Gaming App, had become a very recent five-star review on Google Play for a Communication App.
- Though we haven’t yet checked, our guess is that we’ll see other reviews repeated from the iTunes App Store as well, since there were also – yet fewer – mentions of iPhone and iOS in reviews. Possibly from other sources as well.
Below are some examples of reviews for a Communication App on Google Play that we also found for a Gaming App in Amazon:
Review of Twitter on Google Play – April 7, 2017
Review of Wizard of Oz Slots on Amazon – September 28, 2015
Review of Gmail app on Google Play – March 29, 2017
Review of Bingo Bash on Amazon – December 22, 2013
Review of WhatsApp on Google Play – April 1, 2017
Review of Zynga Poker on Amazon – January 28, 2012
Review of Facebook Messenger on Google Play – March 29, 2017
Review of DoubleDown on Amazon – January 12, 2013
Review of Instagram on Google Play – April 2, 2017
Review of BlackJack myVegas on Amazon – September 29, 2015
Review of Facebook on Google Play – April 4, 2017
Review of Konami on Amazon – October 4, 2016
Because of the consistency in the trends in reviews across several Communication Apps, for specific keywords mentioned and in the frequency by dates, we doubt the Gaming or the Communication Apps themselves had much of a role in the matter. If that were the case, all the Communication Apps would have all had to sync with each other to produce the above patterns in frequency.
THE ASO IMPACT (or “LACK THEREOF’)
The Communication Apps’ star ratings aside, we saw an opportunity to really investigate the long-held ASO theory that keywords mentioned in positive reviews will help boost the app’s ranking for that keyword.
- Was there any change in ranking for the ‘new’ keywords mentioned in reviews for these Communication Apps during the high-frequency time period?
- Was there any effect on keyword rankings for the Gaming Apps mentioned in those 5-star reviews?
Since it’s generally assumed that reviews do affect keyword rankings for phrases mentioned there, we expected to see positive jumps for both 1 & 2 above.
However, after taking a careful look at keyword rankings, we found no effect in ranking trends for neither the Communication nor the Gaming Apps.
Current WhatsApp Rankings for Gaming Keywords
Current Gmail Rankings for Gaming Keywords
Current Skype Rankings for Gaming Keywords
It seems that keyword ‘stuffing’ in reviews as a black hat technique is quite fruitless after all. Perhaps the context for the Communication Apps was so far off that the algorithm recognizes the irrelevancy.
What is still undeniable is the importance of reviews and ratings for conversions. However, so long as the user is swayed by the content of the reviews, not just the rating, they would anyway not take the game-related reviews to heart. It’s as if a black hat technique as such would be more useful without any written comment at all – just a 5-star rating would not leave any clues.
All things considered, because there was minimal benefit to be had with the events described above, we believe the whole phenomenon was either inadvertent – a glitch in the reviewing system – OR, to take The Next Web’s theory further – the result of a bunch of very advanced, yet quite poorly-instructed bots.
What’s your take?
Want to hear more from our App Store Optimization experts? Meet us at MAU!
back to blog