# Too many requests... slowdown. # AhrefsBot http://ahrefs.com/robot/ User-agent: AhrefsBot Crawl-Delay: 60 # SemrushBot https://www.semrush.com/bot/ User-agent: SiteAuditBot Crawl-delay: 60 # Bingbot http://www.bing.com/bingbot.htm User-agent: Bingbot Crawl-delay: 60 # GPTBot https://platform.openai.com/docs/bots User-agent: GPTBot Crawl-delay: 5 User-agent: YandexBot Crawl-Delay: 60 User-agent: DataForSeoBot Crawl-delay: 60 # Dont want Huawei AI, asks a lot User-agent: PetalBot Disallow: / #General Sitemap: https://www.jcvignoli.com/sitemap.xml #Blog Sitemap: https://www.jcvignoli.com/blog/sitemap.xml Sitemap: https://www.jcvignoli.com/blog/sitemap-news.xml #Tonio Sitemap: https://www.jcvignoli.com/tonio/wp-sitemap.xml #Celestia Sitemap: https://www.jcvignoli.com/celestia/sitemap.xml #La Puerta Sitemap: https://la-puerta-de-la-amazonia.jcvignoli.com/sitemap.xml #Wikileaks Sitemap: https://wikileaks.jcvignoli.com/sitemap.xml