When user-agent and allow or disallow are together, it really is regarded as just one entry in a file where in fact the action will simply be reproduced to the user agent that is specified. If you want to direct this to numerous user-agents, list an asterisk (*).
You will then need to ensure that your particular robots.txt file is conserved. Make certain you perform some after to ensure internet crawlers should be able to find and recognize your file.
Save the robots.txt file being a text file.
Position the file inside the directory that is highest-level of internet site ( or in the main for the domain).