Hide Folders and Files With Robots.txt

Sometimes you have content on your site that you don`t want Google to index. Personally I wanted to hide my WordPress Blog from indexing since this blog is created with NEXT.JS and the WordPress REST API. I want to use WP as an API and not a blog.

Turns out, it`s quite simple. Go to the root folder on your webserver. This can be done with SSH, SFTP or even trough your server-providers control-panel. Add a new file called robots.txt. Then add the following lines.

User-agent: *
Disallow: /wp-*/
Disallow: /name-of-other-folder/
Disallow: /myimage.jpg

And that`s it:) Google and other search-engines will not include the given files and folders anymore.

Other posts:

Adding Meta Tags to Next.js With Headless WordPress.

3 simple CSS hacks