Best SEO Blog|Launch your site.

Jeremy Bencken wrote an interesting post on his site in which he disclosed an email from 2006 that was accidentally sent to him by a Google engineer. In the message, the engineer mentioned that PageRank determines which pages are assigned to the Supplemental Index.

Let me say something about Google’s Supplemental Results (again). In 2008, I noted on SEO Theory that Danny Sullivan has tried to defuse the SEO industry’s hysteria over Google’s Supplemental Results by pointing out that other search engines have dual indexes, too.

In my SEO Theory post from 2008 I wrote:

Okay, fine. Everyone uses dual indexes. The problem, however, is that pages that rank well in other search engines DUE TO RELEVANCE usually suck in Google’s search results until you point value-passing links at them. That means Google’s Supplemental Index is more than just a “secondary index”.

The Supplemental Index is, so far as we know, still with us today. The earliest confirmation I can find from a Googler that (internal/non-Toolbar) PageRank determined which index a page went into is a post by Matt Cutts from January 2007. Maybe he mentioned it prior to that time. I know Matt brought this up when I asked him a question at SMX Advanced later that year.

Jeremy’s post is trying to support a conjecture Rand Fishkin made on SEOmoz — that Google sets a limit to how many pages it will index for a site. Jeremy argues that your PageRank limits your inclusion in the Main Web Index (really, that is the only index any SEO should want to be in). Jeremy is right, but that doesn’t mean there must be some sort of hard limit. It just means that Matt’s Peanut Butter Principle (”you only get so much PageRank for your site and you have to spread it like peanut butter across a slice of bread”) implies there is a practical limit to how many pages any given amount of PageRank will move into the Main Web Index.

This is, of course, where all the so-called PageRank Sculpting ideas come from. A few people in the SEO industry proposed that it should be possible to direct the flow of your PageRank throughout your site by cutting off “less important” pages so that more PageRank is directed to the “more important” pages.

The concept is theoretically sound. I have said that before and I will say it again. The problem with PageRank Sculpting is that no one outside of Google has been able to do it. Sure, people have changed the flow of PageRank around their sites. In fact, we do THAT constantly by adding and deleting pages, redirecting URLs, building links, etc.

But no one in the SEO industry has the capability to sculpt PageRank flow. They cannot track and measure it — which means they don’t know when Google assesses the PageRank assigned to any given page on a site. If they knew that kind of information, they would be tracking and measuring PageRank. This is the simplest point in the whole debate and yet people continue to miss it.

Do you know how much INTERNAL PageRank Google will assign to any page on your site today, tomorrow, next week, or 2 months from now? Absolutely not. You don’t, you won’t, you can’t, and therefore you cannot sculpt your PageRank.

What you can do is hack your internal navigation structure so that Google and other search engines are less likely to recrawl certain pages as frequently as before. Apparently, enough people did this with such alarming success that Google felt compelled sometime in late 2007 or early 2008 to take the drastic action of changing how it computes PageRank.

Why? Because the brilliant PageRank Sculpting community was screwing up Websites’ search visibility. So let’s zip forward here.

The myth that you can improve your search results by nofollowing or otherwise blocking some of your pages from being indexed persists to this day. We could easily argue that it’s better to leave idiot SEO techniques to the idiots but I feel there is something morally wrong in that position.

Besides, I may one day be asked to help fix a site that has attempted to Sculpt PageRank. I’ve already reviewed quite a few sites in various Webmaster groups and forums where the owners admitted to sculpting PageRank (and they were all complaining about indexing problems — am I the only person to see a correlation here?).

You can’t do it. I admit I cannot do it. And I won’t be stupid enough to try to do it.

But what does all this have to do with the Supplemental Blues? I think I just made that point (twice) but let me put it another way: Sculpting PageRank increases your Supplemental Agony.

The first thing you can do when you’re concerned that your highly relevant content is being overlooked by Google is to see if you’re sculpting PageRank. Someone recently asked me why SEO Theory uses “nofollow” on its link to the root URL. All I can say is that it appears to be built into the Thesis them.

Is it a stupid thing to do? Yes.

Will I fix it? No. Why? Because I don’t care about it.

I can optimize any Website, with or without nofollows embedded on internal pages. It’s not difficult to figure out the solution for when you cannot change the internal structure of a Website that has indexing problems: get more value-passing links to point to the pages that lack PageRank.

I’m pretty sure that all the links people have pointed at SEO-Theory.com through the years will ensure that the root URL has lots of PageRank. Am I sculpting it? No. I’m just not concerned about whether PageRank affects the site’s ability to rank for its own name.

The recent hullabaloo over at SEOmoz concerning their invalid claim to have proven that “PageRank Sculpting still works” shows that people still care about PageRank. The SEO community is extremely hypocritical on this topic. They pay lip service to the principle of “PageRank is not important” but they don’t live it.

Well, I HAVE lived it. I had no idea that the internal links to SEO Theory’s root URL were nofollowed. I never cared about the PageRank. What is the point of caring about PageRank?

Does SEO Theory have pages in the Supplemental Index? I think so. In fact, if I had to bet on whether it does or does not, I would bet that it does. So what?

I had a much bigger cow when I learned that we were using the All-In-One SEO Plug-in for Wordpress. I absolutely hate that plug-in. It uses “rel=’nofollow’” on your category and tag pages by default. Those are some of the most useful pages a blog can have. Who in their right mind would want to block the things?

I find myself talking to Webmasters quite often about which SEO plug-in you need for Wordpress. When I tell them you don’t need any, people get defensive. Why? Because they have read on dozens of blogs and forums that if you’re going to create a blog you need Wordpress but “Wordpress is not SEO-friendly” and therefore you need an SEO plug-in.

Sorry, folks, but that dog won’t hunt. Anyone in the SEO community who is worth their salt knows that the Wordpress development community fixed all those custom field issues a long time ago. They got some very good advice on the matter from a highly reliable source in 2007: Matt Cutts (this is a video).

It doesn’t bother me to know I have Supplemental Pages. It does bother me if I am unable to get pages indexed the way I want them indexed. Sometimes you just need to point a few more links at a deeper page. Sometimes you need to look under the hood and see if maybe someone else’s idea of “good SEO” is pulling you down.

You cannot prove that PageRank Sculpting works because even if you COULD create a set of isolated sites that manage to direct the flow of PageRank as you intend, they would be completely unlike anything in the real world.

Doubling up a document’s PageRank won’t make it more relevant for more queries.

Doubling up a document’s PageRank won’t make it a better quality document.

Doubling up a document’s PageRank MIGHT get it to rank better for its primary keyword if it is stuck in the Supplemental Index.

The real issue facing Webmasters and SEOs alike today, however, is not how much Google’s Supplemental Index is hurting you — it’s how much you’re hurting yourself by implementing truly bad and unreliable SEO methods. You could be doing that right now without realizing it. If you do your job right, though, your site can tolerate a lot of mistakes.

That’s the most important point. It takes a really incompetent SEO to complain that an “About us”, “Privacy Policy”, or other so-called Incidental Page is so powerful it needs to be nofollowed, noindexed, or blocked by robots.txt.

If you are working with a site that has that kind of problem, and you don’t know how to fix it except by hiding the problem, YOU SUCK as an SEO.

And you can quote me on that.

The real-time Web will be grabbing people’s attention for the next few months, but something that many people in the SEO community seem to have done is walk away from their blogs.

I can understand if you’re too busy to write content every day but if you expect to monitor the way search engines handle blogs in a world full of tweets and SMS-based applications, how do you plan to do that?

Blogging is a chore — but it’s also still a great search visibility methodology that should not be lightly abandoned.

To be honest, I don’t have much time for blogging myself right now, but I’m determined to find a way to keep saying something and hopefully it’s something worthwhile, no matter how brief it may be.

Facebook is not yet ready to be your public platform. MySpace was never very good as a consumer-facing medium for pontificators, either. The plain and simple truth is that when you need to remind people that you exist, there is still nothing better than (or even as good as) a traditional good, old-fashioned Web 1.5 blog.

Don’t ever forget that.

Visible Technologies is on holiday for the Christmas vacation. We wish you and your families a happy holiday experience.

You’re probably using anywhere from 2 to 6 keyword research tools in your work. If not, you must be a pretty reasonable person who isn’t drawn in by sales hype and shmarmy promotional pages. I admit I’ve used more keyword research tools than I can count. None of them really help you find good keywords. You have to find the good keywords in the lists of suggestions your tools give you, and those lists are only as good as your intuitive grasp of the vertical you’re trying to optimize for.

But don’t fret, friends. Honest Mike’s Used SEO Tools has just the right resource for you and it will only cost you the time it takes to read this article to get your greedy little fingers on it — I mean, I’m going to show you how to improve your keyword research for free.

Take a keyword research tool — any keyword research tool. Use that tool to develop a list of keywords for which your tried-and-true methods give you strong indications of value. Now go and create content for those keywords. Build the pages! Build the links! Then come back here when your pages are indexed and receiving traffic.

Three Months Later After Doing “Standard” Keyword Research

Now that your newly minted pages are generating search referrals, do you still have the original keyword list you used to build those pages? I hope so, because you’re going to need it.

Now, sit down with your favorite spreadsheet tool and put the following data into these columns:

1. The list of keyword expressions you decided to optimize for
2. The corresponding page URLs
3. The corresponding page titles
4. The normalized count of search referral strings from your raw server logs that match your keyword
5. Your page’s average rank for the targeted keyword in Ask
6. Your page’s average rank for the targeted keyword in Bing
7. Your page’s average rank for the targeted keyword in Google
8. Your page’s average rank for the targeted keyword in Yahoo!

If you do this right you’ll probably notice several things, including (but definitely not limited to):

• You have no search visibility for some of your keywords in Ask, Bing, or Yahoo!
• You’re not getting nearly as much traffic as you have been led to expect from that AOL click data
• You get traffic from a lot of expressions you did not target
• You’re probably not targeting all the right expressions

If you get as much traffic from a number 1 listing as from a number 9 listing, what does that tell you about your choice of keywords?

If you’re not getting traffic from Ask, Bing, and Yahoo! and you have the same rankings on those search engines as on Google you’re justified in saying that there is no point in optimizing for those search engines. If your rankings are different, shut up and start optimizing for those search engines because you’re in no position to complain about not getting enough search traffic.

If you’re getting traffic from fewer untargeted expressions than targeted expressions you probably have too little page copy.

If you’re getting more traffic from targeted expressions than untargeted expressions you probably rely too much on anchor text.

If you’re getting more traffic for your “money” terms than the rest of your site, you probably have a really sucky Website.

If you’re getting more traffic from search than from other Websites, you really, truly, most sincerely suck at this. Find another business.

Or learn how to expand your opportunities for traffic. Search engine optimization should be just the beginning, not the end of your Web marketing. Through search you create visibility and draw in interested people whose interests escape your initial keyword research. If you create good content some of those people will link to your site. No matter how many links you buy, exchange, or steal — if your site isn’t good enough to attract at least SOME natural links you should still be able to improve it so that it gets those natural links.

Website referrals should provide you with more traffic than any search engine. When you don’t care about which search engine sends you the most traffic, you improve your search engine optimization 100-fold.

You should see that each search engine’s users have a slightly (or greatly) different set of terms they use to find your content. If you cannot see that kind of variation in your search referral data you have totally screwed up your search optimization. You need to fix it.

You cannot use one search engine to optimize for another. If you think you can, you have absolutely no clue as to what you are doing. Get out of SEO before you hurt yourself or swallow your foolish pride and learn how to do it right.

Your keywords won’t lie to you. They’ll tell you what you’re doing right and what you’re doing wrong. They’ll tell you what needs to be fixed. They’ll tell you what is working. But you have to know where to look for them. Every keyword has a “before” and “after” picture. Make sure you look at both pictures before deciding you’ve done your keyword research because it ain’t over until you’ve exhausted every possible way of obtaining traffic for a specific keyword from search.

I think one of the largest unsettled questions in the SEO industry is concerned with how to measure link building. It’s not an easy question to answer because, frankly, there is no one right answer. Everyone has their own idea of what is important in the SEO process. Link building is usually deemed to be one of the most important parts of the process but reporting is not always part of the plan.

First of all, there is more than one kind of link building. Some people build links only to get new sites crawled and indexed, allowing nature to take its course. If you consistently publish good content that’s really about all the link building you need to do.

Some people build links to improve search results rankings. This is, of course, one of the least efficient ways to build SEO success but it’s the simplest concept in the business and a lot of people look no further than that (although, to be fair to the industry, most SEOs probably devote far more time and effort to other SEO processes than they realize — thus subconsciously weighting link building more than they should).

Some people build links because they are under pressure to do so. You know the link building is not necessary but someone wants to see some movement (people scurrying about the decks, so to speak). A great deal of the SEO process involves waiting. Clients and executives don’t like to wait. They think you can force the process if you take immediate action now, which is of course absolutely ridiculous in most SEO situations.

The Searchable Web Ecosystem just doesn’t work that way — not even in the era of Real Time Search.

So, still, people want to know more about how to report on link building. It’s an important question because we have to be accountable for the work we do. I’ve had to report on link building in many different ways.

Some clients just don’t care because all they look at are the search results and (hopefully) their conversions. These are easy clients to build links for but they are probably putting themselves at greater risk of being abused by unscrupulous link building practices.

Some clients just want the numbers, with the understanding that if they feel something is amiss they expect immediate disclosure of the details behind the numbers. I like these clients best because they are making me responsible for my work but trusting me to just give them the core information without getting bogged down in the details.

And some clients want to know everything. These are not bad clients, they are simply detail-oriented clients. I was recently asked on the HighRankings Forums how I report links to clients (and that sort of inspired this article). I mentioned that for these types of clients we’ve provided spreadsheets showing link locations.

You know, I’ve often advised people in the industry to NOT share their resources — but there is a difference between wasting your hard-found linking resources on a blog post and building a stronger relationship with a client. Clients can be very touchy if you try to be coy about your links.

Some clients just don’t understand search engine optimization. They can be a challenge to work with if they feel like they need to be asking questions but as long as you build patience into your client relationship you should be able to reach a mutual understanding with them. I’ve never denied being responsible for a link I had placed for a client.

Some clients know enough about search engine optimization that they want to make sure you don’t do anything sneaky or unethical. They don’t necessarily HAVE to know where all the links are, but they want to know they can audit your work for their own accountability.

So I think you have to be consistent in how your manage your link data. Don’t do it just on a client-by-client basis. Do it for all clients, do it consistently as much as possible. Using the same data tracking and reporting methodology for everyone makes it easier to handle sudden changes in client demands for accountability.

Our working philosophy has been that a client might at some point demand a full audit. In fact, this happened (under welcome circumstances, so please don’t read anything into my admission). We were able to collect and report on a lot of link data pretty quickly.

I like to provide detail-oriented clients with spreadsheets — and probably most everyone wants that. Spreadsheets allow for easy annotation and sorting. There was a time when I provided link detail reports in word processing documents and I don’t ever want to go back to that. Spreadsheets or comma/tab-separated files offer the best reporting formats.

Clients do sometimes object to specific links. You as an SEO might feel like they are perfectly good links but for whatever reason the client doesn’t want the links. We remove such links immediately without argument or attempt at justification. Links are much, much easier to replace than client trust. In a very few rare situations I’ve found myself with links that were not so easy to remove. I don’t like dealing with those kinds of linking resources if only because I want to keep the clients happy.

Of course, there are other aspects to measuring link building. After all raw link counts don’t really tell you anything except that someone placed X number of links (maybe with Y anchor texts) on such-and-such sites.

The difference between effective link building and busy work is a significant change in the search environment. But what do you do about highly competitive queries where, say, 1 month’s worth of link building doesn’t necessarily accomplish much?

I had a client call me up once and ask if I was responsible for the sudden surge in links he saw for his corporate site. “How many links?” I asked, thinking he might have seen our full quota for the week by some miraculous coincidence of software.

“About 1 million,” he replied coolly.

One…million…links. Yeah. Right. I can take those down overnight.

Seriously, I would never place 1 million links in a week or a month. That is just insane. I know some people do it but not me. I’m not interested in that kind of SEO.

I know people who have built millions of links, millions of content pages at a time. I respect their skills. I just don’t use those tactics. And I seriously doubt such people really report those links in detail. What Fortune 1000 company would want that kind of information in its email system? Maybe there are some big public companies that don’t care about stuff like that — but I don’t know any of them.

Link value is really not easy to measure, especially when you don’t have the luxury of using unique anchor text to track which links add value to a destination in the search results.

But if you have access to referrer data you can show links that send traffic to specific destinations.

If you have any ability, even though it might only be to subscribe to search engine alerts, to track Website mentions in the blogosphere, that can help you show how effective your blog link placements may be. “I saw this link on such-and-such” happens.

If your client values Toolbar PageRank and you’ve allowed them to box you into the corner of tracking Toolbar PR you’ll be sitting jittery every 3 months or so when Google updates its TBPR data. I don’t tie my reporting to Toolbar PR data, however. It’s not a reliable indicator of value.

Another good measure of link building would be search visibility — how many new pages are being indexed since the link building began, as opposed to how many new pages were being indexed before the link building began. You have to compare trends to trends, not numbers to numbers. Think of a trend as a vector (a series of numbers).

All of these things contribute to showing clients (or management) they are getting value for their investment. The more information you provide the better. Often enough people have too narrow a picture (at first) and they ask for too little or unhelpful reporting. If you can tie the link building to other metrics you’ll provide better value.

Your ultimate goal is twofold: first, to use the link building measurements to improve your own link building; you want to be the first person to know that what you’re doing doesn’t work, not the last. Second, you want to be able to show the return on the investment. That return can change over time. A link placed today may be worth more to you in a year than in a month, and not simply because it’s a year older.

Measuring your link building adds work to your burden as an SEO but in my opinion an ethical SEO technician cannot hide anything from the client. You have to be accountable for the work you do and what you bill for. If you’re ashamed or afraid to disclose your links to your client (or management), you’re probably doing it wrong.

Link building doesn’t have to be completely transparent — in fact, the more transparent you make the process the more you risk being micromanaged by someone who knows less about link building than you do. Reporting and measuring are really two different functions. You SHOULD be measuring your link building regardless of how much you report. And you SHOULD be ready to report everything if you find yourself with the need to do so.

There is intrinsic, tangible value in the measurement itself.