This will be the third and final installment of our three part series regarding Google’s new smartphone crawler. In parts 1 and 2 we discussed how the new bot works, its impact on which types of sites will be affected and how it will affect them, and how to generate mobile redirect so that it is correctly indexed and working properly. Here, in this final installment, we will be discussing the general problems mobile platforms have when search indexing and how you can prevent them.
If you think indexing is going to be a seamless, predictable, and faultless process, stop now. This is not like doing SEO for desktop sites where there are fewer pages and fewer possibilities for things to go wrong. Ignoring risk-factors and mis-indexing will undermine your SEO efforts and will leave all forms of your site, mobile and desktop, vulnerable when Google decides to change its algorithm again.
To keep you and your company on track to reach its SEO goals and prevent mis-indexing, you should read and follow these guidelines.
The first thing you want to know is that Google hates duplicate content. They never have and they never will. It seems like the general feeling is that Google wants to avoid, or at least reduce, the need to index entire mobile pages. That’s why they are indexing and caching mobile redirects. Just by bringing mobile into the picture, you are creating possible confusion with Google where something could be misinterpreted as a duplicate.
To combat this problem, it’s important not to be sneaky. An example of a sneaky way to create duplicate content would be with something called DUST, or Duplicate URL Same Text. This occurs whenever more than one type of a URL will resolve in the address bar but the browser shows the same exact page. An easier way to comprehend this concept is by page rendering with or without the “www” in the URL. However, DUST can also be seen when there are various versions of a home or category level page.
It’s not uncommon at all for mobile platforms to control servers and databases that generate mobile content. They’re also notorious for creating a lot of DUST. The thing to remember is that a lot of mobile platforms do not really have a full understanding of SEO. Or they think they know more about SEO than they really do. So they try to set their servers to be as flexible as possible with what page requests they can create and create as many different variations of a URL as possible. However, platforms should be instead focusing on setting up the servers to 301 redirect any version of the URL that is not the canonical “chosen” to redirect the “chosen” version of the URL. You can also use this to set up your servers to avoid DUST.
Based on what we know, we do think there could be a chance that Google’s new smartphone crawler could be too literal at first, relying primarily on the redirects that are already in place without assessing other signals or algorithmic occurrences. Also, it’s probable that the crawler will have a high sensitivity to mistakes that are on a site or the redirect. Knowing this, it’s important to avoid 404 errors and misdirects.
Having mistakes on your site will hinder crawling and indexing and will position your site in a bad light. It is also possible that this could affect your desktop site too. This is why it’s important to habitually check up on your content for indexed 404 errors in Webmaster Tools. This is especially important if you creating dynamic mobile pages or using a hosted mobile solution to create your mobile pages. The easiest way to find and fix 404s is by setting up a Webmaster Tools account for your mobile content. This will eliminate the need to subtract desktop figures to create meaningful information, so you can just see the mistakes and errors related to the mobile content.
Note that a lot of 404s in mobile platforms are created by incorrectly expired mobile content. But remember to also check for 404 errors caused by a lack of capitalization normalization and trailing slash rules set up on the server. It is possible for one URL to be working, but if the same URL has a capital letter in it, or has the presence or absence of a trailing slash; it will be seen as missing. However, you can set up your server to automatically normalize URLs to remove capital letters.
It doesn’t matter if you’re getting 404s or you’re just being redirected to the home page, both are signs you have a problem. But don’t worry; this is common when conducting mobile SEO, especially when mobile platforms are in control of the server. If you’re working with an external mobile vendor, you might want to check all this out for yourself. The error-based redirect to your home page could be mis-indexed as the mobile redirect.
Mobile platforms usually don’t archive mobile pages for long periods of time, especially if the particular site is generating new and unique content on a consistent basis. However, they probably don’t have a mechanism to correctly expire the content in a manner that is good for SEO. So a similar situation could occur with a 404 error on an expired page. Usually, mobile platforms will just remove the content and leave the 404 error. But this won’t make your mobile site look good at all since while you’re creating new mobile content, there are also 404 errors being generated at the same rate.
So one has to wonder, what would happen if Google began to take these 404 errors seriously? Ideally, Google will learn to connect the 404 errors on the mobile pages to your desktop pages, but they haven’t yet. So, hopefully, right now Google isn’t letting the 404 errors hurt your site’s credibility or ranking. But, to be on the safe side, you shouldn’t risk it. Eliminate 404 errors and you will be in much better shape.
Remember when optimizing your mobile content that the smartest thing to do is play it safe. Keep your content and server settings as tight and concise as possible. Check your URLs and errors consistently to avoid risking mis-indexing. The more pages and redirects you add, the higher risk you run of being mis-indexed.
The above guidelines, along with parts 1 and 2 of this series regarding Google’s new smartphone bot, should help you better understand and take advantage of this new bot. Make sure you are paying attention to detail regularly and you should be fine. Remember, SEO can be tricky, especially tricky on mobile devices, so make sure you play it safe and play it smart.