Page not indexed in WordPress? Beware of rel=canonical issue for dynamic links

The Story of My LifeRecently I stumbled upon an SEO problem on a huge site that had a lot of it's pages simply disappear from Google index.

I used SEO Doctor to check all the usual suspects (robots.txt, index meta tag, even the scarcely known x-robots-noindex HTTP header) but everything seemed to be in check yet the pages were not indexed. It was starting to drive me crazy.

Then suddenly I discovered the problem.

Since WordPress 2.9 (I believe) rel canonical meta tag was introduced and added automatically for all dynamic pages.

So if you have any non-standard WordPress pages (created through a plugin or in other dynamic means) it may be possible that it they also contain rel canonical tag added by WordPress, which would prevent the particular pages from being indexed.

If that is the case solution is simple. Just add this to your theme functions.php:

remove_action('wp_head', 'rel_canonical');

Learned my lesson the hard way so you don't have to! And also the next version of SEO Doctor (1.4) will feature a warning if canonical links are detected on the page.


More like this:


Posted in: SEO
TAGS:, , , , , , , , , , , , , , ,
Both comments and trackbacks are currently closed.

35 Comments

  1. Mar 17th, 2013 8:27 PM

    i cant understand sir please give me details of "robots" my index page is some time show is not indexable

  2. Sep 15th, 2012 10:16 AM

    Rel Canonical in not prevent to index dynamic pages, it solve the problem of Dublicate title or content and tell search engines as similar content so it will indexed as similar pages..

  3. Aug 10th, 2012 11:49 PM

    I believe this post needs to be amended or removed -- it implies that canonical tags are bad and prevent content from being indexed, which is not true. A canonical tag prevents a *URL* from being indexed, as tag indicates that the page duplicates content of another page.

    • Aug 10th, 2012 11:51 PM

      Yep they are bad when are set against your control, as in this case with dynamic pages that you want indexed.

  4. Jun 28th, 2012 1:09 PM

    FYI - If you use the Yoast/Wordpress SEO it will turn canonical URL's on even if you've edited your core wordpress to turn them off!

    Unlike many other SEO plugins, WordPress SEO does not have an option to turn canonical URL's on/off so you have to edit the functions.php file to get rid of them.

    • Jun 29th, 2012 7:04 AM

      Hi Yoshi :)
      I do use yoast's seo - how do we turn this stuff off - most everyone says yoast is the best for seo - where in the functions.php would I turn it off . . . there is such little information "out there" about canonical what the heck is it good for & why do we need it?
      Thank you in advance for your reply :)

      Just Breathe . . . Breathe Possibility
      Linda Lou

  5. Jun 23rd, 2012 7:08 PM

    You mean to say that the link rel="canonical" will cause an issue?

    • Jun 23rd, 2012 11:39 PM

      Yes, it does - I did go into my theme's epanel & it did correct the issue - what I was finally able to figure out is that wordpress itself generates rel=canonical & themes themselves do the same - fortunately the themes I use have a place to check or uncheck rel=canonical which I disabled & now SEO Dr. shows the issue as corrected.

      However, now I have another problem I haven't been able to figure out yet - SEO Dr. is now showing that I have two H1 tags on every page - but what's interesting is on one particular page I only have H2 tags yet is still telling me I have 2 H1's - so somewhere between WordPress & the themes an H1 tag is being duplicated!

      If anyone has an answer to this fix I would be gratefully appreciative.

      Nothing is more important than feeling good . . . Just Breathe . . . Breathe Possibility
      Linda Lou

      • Jun 24th, 2012 6:53 AM

        Hi Linda, I still see some big sites like techcrunch still using that meta so I don't know why are you recommending to omit that?

        • Jun 24th, 2012 7:21 AM

          I'm not necessarily recommending to omit anything, it's just that SEO Dr. is showing those two items duplicated. Also, not necessarily being a person who knows everything (just enough to be dangerous I guess) this error as shown using SEO Dr. concerned me enough to want to figure out how to fix it . . . It's a little worrisome to me to see a "page not indexable" and "more than one H1 tag" for SEO reasons. As you know Google is slapping the hell out of people with their algorhythms & if you want any chance whatsoever to rank you have to pay attention . . . from what I understand. Forewarned is forearmed right?

          Nothing is more important than feeling good . . . Just Breathe . . . Breathe Possibility
          Linda Lou

          • Jun 24th, 2012 7:29 AM

            Frowm what I can see from your post's code you only have one H1 tag, have you checked your theme? My assumption is that probably there's something between your theme and plugin(no sure which plugin) that is causing this duplicate H1 then your functions.php rewrites everything then as a result you only have 1 H1. Does it makes sense?

            • Jun 24th, 2012 7:36 AM

              ooopppsss I thought SEO doctor is also a plugin..

              • Jun 24th, 2012 7:50 AM

                Hi Linda, I installed SEO Doctor on my FF and it shows you only have 1 H1 instance.

          • Jun 24th, 2012 8:44 AM

            Sorry I got confused... your site is http://lindaloustarr.com/? I checked it and it looks like that it doesn't have H1 tag at all

            • Jun 24th, 2012 8:50 AM

              Check out http://lindaloustarr.com/about-linda-lou & you'll see that it's showing 2 h1 tags - there are several page, posts, theme home page like this (I married 4 different themes together for this site).
              Nothing is more important than feeling good . . . Just Breathe . . . Breathe Possibility
              Linda Lou

  6. Jun 11th, 2012 2:13 AM

    I tried the fix you suggested & it didn't help - on wp home pages for http://lindaloustarr.com - lindaloustarr.com/review or lindaloustarr.com/blog - what is interesting is I have 4 wp installs & the one at lindaloustarr.com/sanctuary does not render the rel=canonical issue - I have searched & searched for 3 days now & cannot find a fix for this - can someone please, pretty please, help :)
    Thank You

  7. Jun 10th, 2012 1:40 AM

    Ok, I've spent two days now trying to figure out why some of my home pages won't index. First, I use 4 different themes to get the feel I need for my site. Using SEO Dr I discovered that three of those themes are showing not indexable because the page has canonical url set to . . . I looked at them through view source & the three that are the issue are showing dual header information - the one that does not have the index problem http://lindaloustarr.com/sanctuary does not have a dual header issue when looking at it through view source. The ones that do have the issue are the main site http://lindaloustarr.com - http://lindaloustarr.com/review & http://lindaloustarr.com/blog -

    I have searched, read, searched, read for two days trying to figure this out on my own, signed up for google webmaster, tried the fix in the functions.php (both parent & child)
    # Remove WordPress' canonical links
    remove_action('wp_head', 'rel_canonical');

    added this to htaccess file in main root

    # Begin 301
    RewriteEngine On

    RewriteCond %{HTTP_HOST} !^lindaloustarr\.com [NC]
    RewriteRule (.*) http://lindaloustarr.com/$1 [R=301,L]

    submitted sitemaps on all .com/folders as above & nothing has changed on the three with issues.

    I found that wp robots txt was hidden so I installed wp robots txt plugin so I could access & fix there as well to be sure it stated user-agent allow . . .

    I am humbly, graciously, & gratefully asking/pleading for some assistance with this - pretty please :)
    Linda Lou

  8. Apr 10th, 2012 9:40 PM

    I'm using the tool doctor reports that my site not because these indexing code conical but he's Index, google can help me

  9. Apr 4th, 2012 4:04 PM

    This is my first time i visit here. I found so many useful stuff in your website especially its discussion. From the a lot of comments on your articles, I guess Im not the only one receiving the many satisfaction right here! keep up a good job.
    Build your credit

  10. Mar 29th, 2012 2:02 PM

    I've found that in wordpress especially the canonical tag loops on category/tag/homepage pages too. This means that Google is being told twice or even more times that a post/page content is 'owned' by many other pages in many other places hence Vladimir's awesome solution.

  11. Mar 29th, 2012 1:30 PM

    It's really important to keep in mind that the Canonical tag on it's identical page is fine; Googe ignores this circle.

    The more important thing to keep in mind is if you're blocking crawlers in your Robots.txt but heavily linking to these blocked areas on your site e.g. an RSS Feed then you are linking to what Google see's as a black hole. Not contructive in comparison to canonicalised pages.

  12. Jan 7th, 2012 7:57 PM

    After yahoo backlinks disapeared seo doctor seems not working now properlly

  13. Oct 12th, 2011 6:44 PM

    nice , i had the same issue in my wordpress at http://www.cloud-hosting.ca
    thanks for the fix. worked perfectly.

  14. Sep 29th, 2011 4:34 PM

    looking here
    http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html
    and the specific example of wikia.com it sounds like it's ok to have a page with a canonical link reference pointing to itself. Which is perfect when there are mutliple domains/paths to a page, and clearly the correct one when used will create this 'loop' -but google says ok.

  15. Tadas
    Aug 30th, 2011 10:06 PM

    Had the same problem on http://www.fixis.lt/lt/Kompiuteriu-supirkimas.html kompiuteriu supirkimas, And thanks to this tip now is everything just fine. Thank you!!

  16. Jul 18th, 2011 1:56 AM

    I have problem on my WordPress said:
    Page is not indexable because This page has a canonical URL set to [http:// ..
    HTML:
    HTTP: X-Robots-Tag:noindex
    robot.txt: User-agent: *Disallow:/
    HTML:

    Link my site: http://www.pembuatanwebsitesurabaya.com.
    I have tried to follow instruction above, but, no luck. Any assistant to help?

  17. Jun 11th, 2011 3:32 AM

    My site - http://facebookbackgroundsandlayouts.com had this very same issue, however I believe I have fixed it now thanks to your useful advice :-)

    I will comment back when I discover if it is fixed or not, thanks!

  18. Mar 3rd, 2011 5:29 PM

    I never comment on site but I had to come back and say THANK YOU. I helped develop a real estate site built on wordpress. All the property pages' content was dynamic based on the url. Months had gone by and none of the properties would show up in google, yahoo,etc. I just stumbled upon your post and it fixed the issue. It was the damn canonical links!

    • Mar 8th, 2011 9:26 AM

      You are welcome Greg.

  19. Sean
    Nov 19th, 2010 10:00 PM

    Brilliant! Worked like a charm. Thank you, Vladimir!

  20. Oct 18th, 2010 5:06 AM

    I am having an issue on a fresh install of WP 3.0.1 using a theme from REIhomesales.com. I have yet to be indexed and started googleing to find reasons why this could be and I believe this is why.

    My page http://www.fasthomebuyersa.com/our-solutions/are-you-in-foreclosure-in-san-antonio-tx/ lists a rel='canonical' href='http://www.fasthomebuyersa.com/our-solutions/are-you-in-foreclosure-in-san-antonio-tx/' which is obviously the identical url.

    So when google trys to spider this url does it in essence tell google don't bother indexing this content because it is identical to the content of the canonical url, if google goes there, they are told to go to...which is like an infinite loop?

    ???

    Clint.

    • Jun 30th, 2011 6:07 AM

      I was wondering the same thing and would love for this to be clarified.

      otherwise, thanks for the really helpful hint

  21. Oct 10th, 2010 6:16 AM

    i cant understand what is rel.. pl simplify

  22. Oct 3rd, 2010 1:59 PM

    Hey BP I think what Vladimir is trying to say is not to de-index what has been indexed by Google but if you are facing a similar situation just make sure that WP is properly tagged on canonical attribute.

  23. Oct 3rd, 2010 3:57 AM

    I am having a hard time following your line of thought. If a page -any page- has the right canonical, why would it be de-indexed? It makes no sense. Google said they will strictly follow canonical metatags. I am missing something for sure.

  • ManageWP