Don't have Telegram yet? Try it now!
https://404.su/blog/pravilnyy-robots-txt-dlya-lendinga/
Правильный robots.txt для сайта без CMS