Google has just pushed some updates to the reduce the Google crawl rate documentation. Most of the changes are just organizational to the content on the page but under the "Urgently reduce crawler traffic (for emergencies)" section, Google added new information on how to use Retry-After HTTP header.
from Search Engine Roundtable https://ift.tt/aLHFb8Q
No comments:
Post a Comment
Leave Post