CMLABS Launches The Seventh SEO tool: Robot.txt Generator

Written by cmlabs, Dec 15 2021

Image 1. Cmlabs Robot.txt Generator Tools


Jakarta - cmlabs released a new SEO tool known as Robot.txt Generator Tool. This tool is based on a website application. The main goal of the tool is to support the SEO activities which create robots.txt file for webmasters that tells search engine crawlers which URLs the crawler can access to their site.

The robots.txt generator tool provides several features. Users can choose for default robot access between allow and disallow. Users can choose crawl delay up to 120 seconds. Users can also add a sitemap if they have one. Users can add as many directives as they need for adding specific user-agents and specific categories. Users can add directives without any limit. In the process, the system will generate more user-agents based on how many users add directives inside robots.txt.

In this tool feature, the user will be given a preview of what users have inputted on the previous form. For example, if a user has added five directives with different user-agents and different paths, six user-agents will be shown in the preview section below form. By default, if you change user-agent both allowed and disallowed, one user-agent will be added on preview with root (/) as the default path. If you add crawl delay, the crawl-delay property will be shown in the preview too.

To get all the features above, you must visit the page, and generate your robots.txt file for your website. Don't forget to explore more tools related to SEO and webmasters that will help your SEO activities at and good luck. To find more about cmlabs products and services, you can directly visit and if you want to know more about cmlabs App Lite please contact the marketing team at (+62) 822-600-600-71 or send us an email at

Last edited 2021-12-15 11:10:02

Super Admin

Super Admin

Note: We have attached some of the most common questions asked by users below, along with their answers. To use the cmlabs Keyword Ranking Tracker application, you don't need to request for a quote from marketing. Please click login to the application.

WDYT, you like my article?