This website crawler program allows you to crawl URLs and fetch key onsite elements to analyze onsite SEO.

The SEO Spider is a powerful and flexible site crawler, able to crawl both small and extensive websites efficiently while allowing you to analyze the results in real time. It gathers critical onsite data to allow SEOs to make informed decisions.

This software application is developed with Java to provide users with a simple means of gathering SEO information about any given site, generating multiple reports, and exporting the information to the HDD.
Features of Screaming Frog SEO Spider

Find Broken Links, Errors & Redirects
Analyse Page Titles & Meta Data
Review Meta Robots & Directives
Audit hreflang Attributes
Discover Duplicate Pages
Generate XML Sitemaps
Site Visualisations
Crawl Limit
Scheduling
Crawl Configuration
Save Crawls and re-upload
Custom Source Code Search
Custom Extraction
Google Analytics Integration
Search Console Integration
Link Metrics Integration
Rendering (JavaScript)
Custom robots.txt