See here an example of Magento robots.txt that improves SEO. Including the solution to let Google reindex the changed robots.txt