3 Commits

Author	SHA1	Message	Date
ic0nic	c918a9fe2e	Incorrect structure ... The structure for robots.txt is incorrect for some/most search engines.	2013-11-01 01:22:16 +01:00
ic0nic	50d4fb9bf3	Allow discovery ... To allow search engines to at least find the website.	2013-01-29 19:33:31 +00:00
Greg Knaddison	f37303d858	For server-load and privacy reasons, disallow robots.	2012-08-10 18:21:26 +02:00