Just come across an interesting article originally posted back in August that highlights an interesting point about Drupal core's robots.txt file potentially blocking content on a Drupal site from being indexed. I'd not heard of this issue before and it's got me wondering whether this is the cause of a problem that we've recently been trying to get to the bottom of. Must investigate further and see if I can improve our Drupal sites' indexing on search engines...
Comments
Comments are now closed.