NetTalk Central
NetTalk Web Server => Web Server - Ask For Help => Topic started by: Robert Iliuta on October 08, 2014, 10:35:37 AM
-
Hallo,
I would like to secure / hide my web app and also I don't want to be founded on google or any other search engines.... What could be the best to do this?
This app also run with an SSL certificate and require authentication. It will be much better if I run on different port then 443? also NT take consideration if I put a txt file "robots.txt" with specific text inside?
ex: User-agent: *
Disallow: /
Also I would like to include meta tags in my header like:
<meta name="robots" content="noindex">
<meta name="googlebot" content="noindex">
how can I insert this lines?
Thank you for any tips, suggestions!
Robert
-
Hi Robert,
>> I would like to hide my web app
your app can't be both "visible" to valid users, and invisible to other people. If someone knows the URL they can certainly get there.
>> I would like to secure my web app
SSL and Logins work well to restrict access to authorized people.
>> It will be much better if I run on different port then 443?
yes, to some extent it will be harder to "randomly find" if it's on another port. Especially if there are _no_ links to the address on other sites.
>> also NT take consideration if I put a txt file "robots.txt" with specific text inside?
robots.txt is for search engines. They request the file and either observe, or ignore, the contents therein. The server doesn't process the file. Obviously all the main search engines respect the robots.txt, so havng one there effectivly excludes you from the search. (Although the spider can't log in anyway, so there's really not any real practical difference.)
in WebHandler, in SetCustomHTMLHeaders method,
self.metaHeaders = '<meta name="robots" content="noindex"><13,10><meta name="googlebot" content="noindex">')
Cheers
Bruce
-
Hallo Bruce,
>>yes, to some extent it will be harder to "randomly find" if it's on another port. Especially if there are _no_ links to the address on other sites.
Thanks.
>>in WebHandler, in SetCustomHTMLHeaders method,
>>self.metaHeaders = '<meta name="robots" content="noindex"><13,10><meta name="googlebot" content="noindex">')
cool! works perfect.
Thank you Bruce!
Robert
-
>> Obviously all the main search engines respect the robots.txt,
Yes, the honest ones do, but there are a lot of spiders, that ignore these settings.
It would be nice to have a blocking-list, something like the Apaches .htaccess, in Nettalk too. Then adding a "honeypot.html" to robots.txt and finally putting each IP-Address, that crawls to this honeypot, gets an honorable place on the blocking-list.
Just as an idea.