Allison Chaney
Apr 05 2018
The Screaming Frog SEO Spider is a tool that can help you organize and manage your SEO related tasks for your website. You can find a download of this tool on the Screaming Frog website. We love this tool because it has a ridiculous name, which reminds us not to take ourselves too seriously when SEO work starts to feel tedious. But the real value of the tool is not in it’s name, it’s in the feature that allow you to fetch and analyze key data that’s related to on-site SEO.
We’ve created a video tutorial on how to use some of the key features in Screaming Frog, and you can view that over on our Boot Camp Institute site in the Digital Marketing Tools training. The tool crawls your website, just as a search engine spider would, and collects all kinds of meaningful data. Here are some of the data that you can analyze, and what you would do with that data:
- Clean, search friendly URLs. You can easily scan a list of all of the page addresses (URLs) of your website and note if they are readable by humans with keywords in the URLs as this is better for SEO than long URLs that have a bunch of characters and doesn’t make any sense to a human. For example, if you were searching for wakeboard accessories, which URL would you click? (probably the first one right?)
a) examplesite1.com/boating/wakeboard-accessories
b) examplesite2.com/product142.php?id=09234j&page=4
- Check your site for 404 (not found) errors. This tool will tell you if they found any broken pages while crawling through your site. You don’t want the search engine or your users to click a link within your site and land on an error page. This is a quick way to check your site without clicking every link on every page of your site.
- Title & Meta Description Tags. This is a great tool to check your site for any title tags or meta description tags that fall outside of SEO best practices. You can check for pages that are missing title tags, have duplicate tags, or have tags that are too long or too short.
- Heading Tags. Heading tags are like the headline or title of your page, and should be used in an outline format. In Google’s SEO Starter Guide, they recommend that heading tag sizes should be used in order to create a hierarchical structure for your content. So, you should start at the top of your page with an H1 tag, then if you need a sub-heading use an H2 and so on. The Screaming Frog tool will help you quickly spot any pages that use H2 but not H1, or other issues that are outside of best practices.
- Word Count. How many words should a web page have? The answer depends on what is natural for the top ranking sites for the keyword you want to rank for. You should also get to the point in as few words as possible, but if content is too thin, you may not be offering enough information to be useful. Within this tool, you can sort by number of words to identify thin content and an opportunity where content can be trimmed or split into multiple posts or pages.
- Outlinks. Google has stated that pages with more than 100 links out to other pages are suspect. This rule has roots in bad SEO practices where sites created pages with tons of links for the sole purpose of boosting link value for SEO. Well, that practice doesn’t work, and Google made this recommendation to help put a stop to some of those bad practices. It’s also a better user experience if a page doesn’t have too many links out. If you give a user too many choices, they might get frustrated and just go somewhere else.
- Response time and Size. Page speed is a big factor in how well you rank in search. You can sort by the largest size pages on your site, and by pages with the highest response time to focus in on pages that need some work to help speed up the load time.
You can also export a list of all this data into an Excel spreadsheet and use it to edit and manage all of the tags and updates for all pages of your site. This tool provides a great way to organize the content of your site by page, and setup a plan for optimizing the key SEO elements on each page of your site. In our SEO course, we also have a section on how to use Screaming Frog to export your data into a spreadsheet and use it for on-page optimization. This is a must-have tool for any SEO project.
Read some more of our recent posts
