Gary Illyes and Chery Prommawin released their second video in the Non-Canonical Cabinet series; this one is named What can you actually put in a robots.txt? Here they discuss what is, and is not, supported by robots.txt files. And while Google may support features or directives, other search engines or AI crawlers might not.
from Search Engine Roundtable https://ift.tt/i39Zb1C
No comments:
Post a Comment
Leave Post