Google: URLs Excluded By Robots.txt Aren't Removed Until URLs Are Individually Reprocessed

Google's John Mueller posted a clarification on how and when Google processes the removal requests, or exclusion requests, you make in your robots.txt. The action is not taken when Google discovers the change in your robots.txt, but rather after first the robots.txt is processed and then the specific URLs that are impacted are individually reprocessed by Google Search.

from Search Engine Roundtable https://ift.tt/DUgxl3h

No comments:

Post a Comment

Leave Post