Google not listing site, but no errors and content appears rich

manpreet Best Answer 2 years ago

We have inherited a website of around 30,000 pages (actual content), each with a unique title and rich content. Whatever we try, Google seems to not like listing the new site and visits have dropped by 80% (vs. old site & domain).

The website was redeveloped recently and changed domain at that point too, which hasn't helped work out what is happening and this marked the drop in visitors. The old domain was registered Oct 2005, the new in 2009, so both have some age to them. In Webmaster tools recently submitted a notice that the site address had changed, possibly too soon to see any effect of this (7th Dec).

The older CMS was hard to redirect from, so have a very large .htaccess file (1MB), is there a limit to how large this file should be with redirects? I could perhaps code something in PHP to handle the 30,000 redirects programatically, but the URL's in the old were pretty strange using comma separation and other symbols. I have used header checkers and the correct 301's are being returned.

We also submitted a sitemap with 25,000 pages via Webmaster tools, of which it listed 11! There were no errors and as I say, the page content is rich with descriptive titles.

Google can see 68,000 pages in Webmaster tools, but the actual listed in only 175, so the problem seems quite significant and the others remain 'unselected'. The curve of the 'unselected' seems to reflect the efforts we have put in to have the site list, yet they seem not to be indexed.

Site: http://bit.ly/VKYClf (The older site was the same name but hyphenated)

I have researched a lot, but all steps so far have been fruitless and the pages listed hands around the 170 mark.

Can you think of any specific steps worth taking to identify any factors preventing the site from listing?

Thanks in advance and happy to provide more information on anything.

EDIT: In case it helps anyone else, the website is built of Wordpress but uses custom vars to generate lots of pages on the fly... since WP 2.9 a canonical tag was added, but the two were not playing well together and they were pointing to anything WP could find with that ID... have now removed and hopefully things are moving forward

manpreet 2 years ago

It's going to take awhile to get fully indexed if you just submitted 25000 url's, it may take months for that.

I would recommend you log into google webmasters, go to health (fetch as Google) you can give individual url's to crawl, Google should respond within 24 hours. If your indexed page still does not show up you have major problems that I would not be able to assist with. You get 500 fetches a week. If you want your site indexed right away it may be your only shot.