Crawling can be used to gather information about all the pages your application creates. For instance you can visit every page to make sure none has an error.
Grell is very easy to use. You only need to provide the start page and Grell will yield to your code with each page crawled. In this example we print out the pages giving a 404 error, this code will not crawl till we finish every page in Google:
Grell adds two IDs to each page: ‘id’ and ‘parent_id’. With this information is very easy to re-create the crawling tree to, for instance, create visualizations. Grell also supports filtering with whitelist, blacklist, custom filtering, retries, etc.
For more information on all the possibilities of Grell, please visit the Grell repository and read the thorough documentation.