As you probably know, Google, Yahoo, MSN and other search engines have “crawlers” that scour the Internet for web pages and content to populate their index for the search results.
These crawlers are programs that read in your web site and look at the code behind it. Do you ever wonder what these crawlers see when they come to your web site?
This little tool called the Lynx Viewer allows you to get an idea of what they see. Lynx was one of the original web browsers - before Mosaic or Netscape - and it shows only text, no images or anything. Many webmasters use tools like this to make sure their web pages can be read and understood by the crawlers.
Here’s what comes back when I ask it to view http://www.expand2web.com:
What should you look for in this report?
You want to make sure that what gets returned is humanly readable. Even though the search engine crawlers are just programs, they are very good a determining whether a web site is using language and text that contains real content “for human consumption” and not a bunch of repeated keywords or content. Old search engine optimization consultants used to use these techniques as “tricks” to improve search engine rankings, but the search engines don’t fall for this any more. In fact, they will penalize you if your site uses techniques like that now!
If you're new here, you may want to subscribe to my RSS feed or Email updates. I'm also on Twitter. Thanks for visiting!
{ 0 comments… add one now }