https://easyschema.com/blog/robots-txt-what-is-it-and-how-does-it-works/