Johnny91 Every time I update the config file (license applied), I change some settings to the file and then reupload. and then the website asks to enter the license every time again.
Is it supposed to be that way?
No, it's not supposed to be that way. It is not normal. The only explanation, would be if you somehow corrupted the config or license key, or the server or browser is somehow caching response data (not normal either). I would have to diagnose with a real link.
Johnny91 Also, I have a robots.txt file to discourage search engines to index my site.
For your info, there is already a meta tag in the Files document that tells search engines to not index anything.
<meta name="robots" content="noindex,nofollow">
It's not guaranteed that I keep the meta tag in future release, so you might want to add your own robots.txt
.
Johnny91 'files_exclude' => 'robots.txt',
The files_exclude config option uses regex, so you would want to add it like this:
'files_exclude' => '/^robots\.txt$/',