Sometimes you have content on your site that you don`t want Google to index. Personally I wanted to hide my WordPress Blog from indexing since this blog is created with NEXT.JS and the WordPress REST API. I want to use WP as an API and not a blog.
Turns out, it`s quite simple. Go to the root folder on your webserver. This can be done with SSH, SFTP or even trough your server-providers control-panel. Add a new file called robots.txt. Then add the following lines.
User-agent: * Disallow: /wp-*/ Disallow: /name-of-other-folder/ Disallow: /myimage.jpg
And that`s it:) Google and other search-engines will not include the given files and folders anymore.