Google: Very Few Robots.txt Files Are Over 500KB

Gary Illyes shared a nice little tidbit on LinkedIn about robots.txt files. He said that only a tiny number of robots.txt files are over 500 kilobytes. I mean, most robots.txt files have a few lines of text, so this makes sense but still, it is a nice tidbit of knowledge.

from Search Engine Roundtable https://ift.tt/GY7ncJm

No comments:

Post a Comment

Leave Post