Best SEO Blog|Launch your site.

Bot accessibility is about making sure that search agents known as spiders, robots or “bots” for short, have full accessibility to all the content that you want displayed to the public.

Creating a growing archive that remains accessible to both bots and the public can be instrumental to rapidly increasing the size, strength and overall value of your web property. Like most things in SEO, it is much easier to plan ahead for an effective archive rather than reorganizing one that has been indexed.

Rather than focusing on which database to use or how to build with one of the many excellent content management systems out there, I’m going to list a few key points to watch for.

Bot Accessible Archive Dos:

Each Document Should Have Its Own URL
Whether using pages, posts (or anything else) make sure that each article/press release/product has its own URL. Giving each subject its own URL is probably the most important aspect of proper archiving.

Persnickety Order
Create a logical hierarchy and categorize groupings of similar subject matter. Clean informative structure is as much about human accessibility as it is about bot accessibility.

Proper Labeling and Legible URLs
On the server side, convert all database generated URLs to descriptive URLs. The actual logistics of this have everything to do with your CMS and platform. Labeling the tags and content in your archive in a way that is human intuitive will allow indexing bots to serve your pages to the correct searchers.

Using legible URLs can convey on the SERP level what exactly is on a page and may even lend to your target keywords. Searchers will have to rely less on your descriptions and title tags to surmise what your page is about.

Example:

Bad Archive Structure

Good Archive Structure

Bot Accessible Archive Don’ts:

Flash Archives
Yes I have seen Flash archives that were pretty to look at but overall somewhat cumbersome to navigate and not necessarily bot accessible. Since my guess is that you aren’t trying to impress anyone with your creative prowess as they rummage through your archive, it’s best to apply KISS and focus on functionality.

Ajax Archives and iFrame Archives
Both Ajax and iFrames tend to have a single URL and display content through a window on the page. The problem is when you navigate to the press release or document that you are searching for there is virtually no way to share it without downloading or copy and pasting it.

PDF Archives
While it is possible for a bot to crawl and index a PDF document, people generally hate to see PDFs in search results unless they are searching specifically for a product manual or academic paper. PDFs make good product brochures and lousy product pages.

PDFs appear to pass value such as anchor text, but from a user intuitive standpoint can make awkward link placement. A user clicks on a PDF, it downloads, a link is visible so they click it and the browser opens a new window. Ick!

If your archive happens to be of product manuals or academic papers you might want to consider creating a database driven archive of pages containing an abstract, author and a way to download the PDF.

Happy archiving!

This is part 3 of a 3 post series meant to help you identify some areas to focus on when choosing a web designer who also understands SEO. Keeping these ideas in mind during the building process may help you to get the most out of your designer’s “on-the-clock” hours.

Aesthetic appeal is subjective, but the objectives of your website should be a fully conscious, coordinated effort. If you build with search in mind, it will be much easier to get qualified traffic in effective queries rather than trying to retrofit a built and indexed website with an SEO campaign. Obviously the ladder is still very plausible but in terms of business, may cost you more dollars than sense.

The following are a few things to keep in mind when choosing a designer for a new website. (Note: For the purpose of this discussion we’ll assume that content is not up to the designer.)

Choose a web designer who understands the crawl.

Work with your designer to ensure that all portions of your website will support proper web crawling and indexing.

Spiders, also called bots, are automated scripts in charge of crawling the interweb for information gathering, and other stuff. In regards to search engines, if a spider hasn’t touched your page, it won’t be indexed or served to search users.

Although being crawled by a search engine doesn’t give you an automatic jump to the indexed bin, building a spider accessible, fully indexable website that stimulates frequent crawling to deep rooted pages will go a long way to maximizing your web presence. Like all things search, keeping the crawl in mind during the building process will make your life much easier when you choose to expand.

Redundant Accessibility

Build out new sections with redundant accessibility by linking to each page from multiple pages within the same site. Shoot for at least 2 to 3 ways to access a single page. It’s really not as daunting as it sounds. If you build an HTML/XHTML sitemap for your user, that’s 1. Build parent pages for each individual item or concept, that’s 2. Provide navigational aids, such as breadcrumbs, that provide a link back to each previous section, boom you’re done! Be creative and ONLY ADD LINKS IF IT AIDS THE USER.

JavaScript Navigation

There are always new methods to MacGyver the ability to crawl navigation but to truly put an emphasis on search stick with XHTML/CSS. Hey, you can do a lot with CSS and XHTML!

If you’re not sure if that glitzy navigation allows the spider to pass from section to section then test it with an all text browser. If you can’t navigate your site, neither can the spider.

Stimulate the Deep Crawl

If deploying a larger website, 100+ pages, it may take several visits for spiders to fully index your pages. Anything that you or your designer can do to increase the rate of crawl is a good thing. Placing dynamic, engaging, natural link-worthy content on hub pages will be ideal points of entry for spiders. Channeling spiders like blood flow to deep rooted pages may help keep leaf pages from being isolated or dropping out of the index.

Plug Yourself

When deploying new sections or content remind users, and spiders, with links to existing sections. You’ll want to plug your hub pages because you will have already designed these sections to keep those bots barreling through page after well linked page of your website.

Use the robots.txt

Make sure your designer understands the power of the robots.txt. You can use it to tell the spider to auto-discover your sitemap.xml, ignore certain sections, or strait tell it to go away while you perform maintenance or finish building.

There are really too many options to list. In not so many words it is one of the only opportunities you’ll get to guide the spider with explicit directions of how it should treat your website.

I hope after reading this little 3 post series that it will be easier to choose a web designer that also understands SEO. I tried to approach this topic by identifying what I would look for if I were to outsource a website. Thanks for reading!

___________________________________________________________________

Quick Reference:

How to choose a web designer that also understands SEO

Part 1: Choose a web designer that understands how to build for the search user.
Part 2: Choose a web designer that understands usability.
Part 3: Choose a web designer who understands the crawl.