94. Give the syntax of the robot.txt file.
The /robots. txt file is a publicly available: just add /robots. txt to the end of any root domain to see that website's directives (if that site has a robots. txt file!). This means that anyone can see what pages you do or don't want to be crawled, so don't use them to hide private user information.
