Screaming Frog Tutorial: How to Audit Any Website in 30 Minutes

What Screaming Frog Actually Does (and Why You Need It)

Screaming Frog crawls your website like Google does and reports back every technical problem it finds: broken links, duplicate content, missing meta tags, crawl errors, redirect chains, page speed issues, and about 50 other things that kill your rankings.

Here's the honest take: if you're serious about SEO, you need a crawler. Not eventually. Now. Most of the clients I audit have no idea they have 200 broken internal links or 40 pages with duplicate title tags. Screaming Frog finds this stuff in minutes instead of you manually checking 500 pages.

If you don't want to read this: Buy the desktop version for $99/year (one-time), crawl your site, export the broken links and duplicate content reports, and fix the top 20 issues. You'll see traffic improvements within 30 days. The rest of this guide walks you through it step by step.

Quick Reality Check: Desktop vs. Cloud vs. Free Tools

Screaming Frog has three versions:

• Free desktop version — crawls up to 500 URLs per project. Good enough for small sites under 500 pages. Fully functional, no watermarks, no BS.
• Desktop Pro ($99/year) — unlimited crawls, API access, scheduling. This is what I use and recommend.
• Cloud version ($99/month) — crawls run on their servers, integrates with Google Search Console and Analytics natively. Overkill for most people.

For most small to mid-market businesses, the desktop Pro version is the move. You pay once, you own it, you use it as much as you want. The cloud version makes sense if you're running audits on 50+ sites per month and want scheduled automatic crawls. I do that for agency work, so the cloud pays for itself. You probably don't.

Free alternatives: Sitebulb (14-day free trial but then $99 one-time), Google Search Console (free but limited), Ahrefs (expensive but good), SEMrush (expensive). None of these are better than Screaming Frog for the price. GSC is good as a supplementary tool, but it only shows crawl issues Google encounters—not all issues on your site.

The 30-Minute Audit Workflow I Use With Clients

Step 1: Set Up Your Crawl (2 minutes)

Open Screaming Frog. Paste your domain in the URL field. Before you hit “Start,” go to Configuration > Spider > Advanced.

Change these settings:

• Crawl Depth: 2 (unless you're auditing a huge site with deep category structures, 2 is fine)
• Threads: 5 (faster crawl without overloading your server)
• User-Agent: “Screaming Frog SEO Spider/15.5”
• Respect Robots.txt: Yes (you want to see what Google sees)

Hit Start. Go get coffee. For a 1,000-page site, this takes 5-10 minutes.

Step 2: Check the Overview Dashboard (1 minute)

When the crawl finishes, you see the Overview tab. This gives you the high-level picture: total pages crawled, response codes breakdown, indexability, crawl errors.

Look at the red numbers. Anything in red is a problem:

• 4xx errors (404s): broken pages or links
• 5xx errors: server issues, contact your host
• Redirect chains: Page A redirects to Page B redirects to Page C. Google wastes crawl budget on this.
• Client-side redirects (JavaScript redirects): worthless for SEO

Write down these numbers. You'll use them for the client report.

Step 3: Export Broken Links and Redirects (3 minutes)

Go to Bulk Export > Response Codes > 4xx Client Error. Export to CSV. This shows every broken internal link and every page that links to it.

Why this matters: if a user clicks a broken link on your site, they leave. If Google crawls a broken link, that's wasted crawl budget. Fix these first.

Then go to Bulk Export > Redirects > All Redirects. Look for redirect chains (A→B→C). These should redirect straight to C instead of bouncing around.

Step 4: Find Duplicate Content Issues (5 minutes)

Go to the Duplicate tab. Filter by Title and Meta Description.

Here's what you're looking for:

• Duplicate title tags: Two pages with identical titles dilute ranking power. One page should own that keyword.
• Duplicate meta descriptions: Confuses Google about what each page is about. Less critical than titles, but still fix it.
• Duplicate content (body text): If two pages have nearly identical content, you need to either merge them, noindex one, or canonicalize.

Export these. Flag the ones that matter (usually the top 10-20).

Step 5: Check Page Titles and Meta Descriptions (5 minutes)

Go to the Page Titles tab. Filter for:

• Missing titles: Every page needs one
• Duplicate titles: (You already did this, but double-check)
• Titles over 60 characters: They get cut off in search results. Google doesn't care, but users do.
• Very short titles (under 30 characters): You're leaving SEO value on the table

Then Meta Descriptions tab:

• Missing descriptions: Every page should have one (even if Google rewrites it sometimes)
• Over 160 characters: Gets truncated

Pro tip: Use the bulk editing feature in Screaming Frog to update these at scale. Export, edit in a spreadsheet, re-import. Saves hours.

Step 6: Look at Headings (H1, H2, H3) (3 minutes)

Go to Headings. Check for:

• Missing H1s: Every page should have exactly one
• Multiple H1s: Only one per page. Multiple H1s confuse search engines about what the page is about.
• H2 and H3 orphans: Headings should be hierarchical (H1 > H2 > H3). If you jump from H1 to H3, fix the structure.

This ties into broader technical SEO best practices. Clean heading structure helps Google understand your content and helps users scan your page.

Step 7: Check Images and Alt Text (2 minutes)

Go to Images. Filter for Missing Alt Text. Every image should have a descriptive alt attribute (for accessibility and image search).

You don't need to rewrite 500 alts today. But flag the important ones: hero images, product images, infographics.

Step 8: Review Indexability (2 minutes)

Go to the Crawlability tab. Check for:

• Noindex pages: You should only be noindexing pages you don't want to rank (like duplicate pages, thin content, login pages). If you're noindexing important pages by accident, that's a problem.
• Pages blocked by robots.txt: Same deal. Make sure important pages aren't blocked.
• Pages with nofollow: Usually fine for internal links, but sometimes you'll find internal links you want Google to follow are nofollow by accident.

This is part of site crawlability. If Google can't crawl it or you've told Google to ignore it, it won't rank.

What I Actually Fix Based on Budget and Site Type

If you have 1-2 hours and $0 budget: Fix the broken links (4xx errors), add missing H1 tags, fix missing title tags and meta descriptions. Done. That's 80% of the value.

If you're an e-commerce site: Broken links and duplicate content are killing your crawl budget. Also check for parameter-based duplicate content (UTM parameters, session IDs). Screaming Frog shows this in the Duplicate Content tab.

If you're a local business (law firm, HVAC, dentist): You probably have 20-50 pages. Make sure you don't have duplicate titles/descriptions on service pages and locations pages. That's the #1 issue I see.

If you're running a blog or news site: Watch out for thin content pages and near-duplicate content across categories. Screaming Frog shows pages with very little text, which often rank poorly.

The Reports You Should Export and Share

When you pitch fixes to a client or stakeholder, export these three reports:

1. Broken Links Report (Response Codes > 4xx): Shows impact. “We have 47 broken internal links, including 12 on your homepage footer.”
2. Duplicate Content Report: “23 pages share identical title tags. We recommend consolidating these into 8 unique titles.”
3. Missing Critical Elements (combined from Title, Meta Description, H1 tabs): “156 pages are missing H1 tags. This confuses search engines.”

These three reports tell a story and give you a prioritized action list. Don't dump all 15 reports on someone. Pick the ones with actual business impact.

The Gotchas and Limitations I've Hit

JavaScript-heavy sites: Screaming Frog crawls the HTML it receives. If your site loads content with JavaScript, Screaming Frog might not see it. The desktop version has a JavaScript rendering option (paid feature in cloud version), but it's not perfect. Use it as a supplementary check.

Large sites over 500K pages: Screaming Frog's cloud version has limits. If you're crawling 500K+ pages monthly, you might need a dedicated crawler like Botify or DeepCrawl. But if you're running a typical business site, this isn't you.

Session-based content: If your site requires login, Screaming Frog can handle it (Configuration > Spider > Authentication), but it's finicky. Sometimes it works, sometimes it doesn't. Usually not worth the headache; just crawl what's publicly visible.

False positives on redirects: Sometimes Screaming Frog reports a redirect when it's actually just a 200 OK page that happens to set a Location header. Check manually before you “fix” it.

What I Use and Why

I run audits on 20-30 client sites per month. I use the desktop Pro version ($99/year) for quick one-off audits and the cloud version ($99/month) for ongoing monitoring. The desktop version pays for itself on the first two clients. The cloud version I use because I'm lazy and like automated weekly crawls.

For smaller operations, the free desktop version is genuinely enough. Use it once a quarter. Fix the top issues. Move on.

Your Next Move

Download Screaming Frog (free version) today. Crawl your site. Export the broken links and duplicate title reports. Pick the top 10 issues and fix them this week. You'll see traffic improvements within 30 days if your site has technical problems (most do).

If you're managing multiple sites or want automated crawls, buy the Pro version for $99/year. It's the best one-time tool investment in SEO.

The sites that rank well aren't always the ones with the best content. They're the ones with clean technical foundations that Google can actually crawl, understand, and rank. Screaming Frog finds the cracks in your foundation so you can fix them.