Best SEO Blog|Launch your site.

Jeremy Bencken wrote an interesting post on his site in which he disclosed an email from 2006 that was accidentally sent to him by a Google engineer. In the message, the engineer mentioned that PageRank determines which pages are assigned to the Supplemental Index.

Let me say something about Google’s Supplemental Results (again). In 2008, I noted on SEO Theory that Danny Sullivan has tried to defuse the SEO industry’s hysteria over Google’s Supplemental Results by pointing out that other search engines have dual indexes, too.

In my SEO Theory post from 2008 I wrote:

Okay, fine. Everyone uses dual indexes. The problem, however, is that pages that rank well in other search engines DUE TO RELEVANCE usually suck in Google’s search results until you point value-passing links at them. That means Google’s Supplemental Index is more than just a “secondary index”.

The Supplemental Index is, so far as we know, still with us today. The earliest confirmation I can find from a Googler that (internal/non-Toolbar) PageRank determined which index a page went into is a post by Matt Cutts from January 2007. Maybe he mentioned it prior to that time. I know Matt brought this up when I asked him a question at SMX Advanced later that year.

Jeremy’s post is trying to support a conjecture Rand Fishkin made on SEOmoz — that Google sets a limit to how many pages it will index for a site. Jeremy argues that your PageRank limits your inclusion in the Main Web Index (really, that is the only index any SEO should want to be in). Jeremy is right, but that doesn’t mean there must be some sort of hard limit. It just means that Matt’s Peanut Butter Principle (”you only get so much PageRank for your site and you have to spread it like peanut butter across a slice of bread”) implies there is a practical limit to how many pages any given amount of PageRank will move into the Main Web Index.

This is, of course, where all the so-called PageRank Sculpting ideas come from. A few people in the SEO industry proposed that it should be possible to direct the flow of your PageRank throughout your site by cutting off “less important” pages so that more PageRank is directed to the “more important” pages.

The concept is theoretically sound. I have said that before and I will say it again. The problem with PageRank Sculpting is that no one outside of Google has been able to do it. Sure, people have changed the flow of PageRank around their sites. In fact, we do THAT constantly by adding and deleting pages, redirecting URLs, building links, etc.

But no one in the SEO industry has the capability to sculpt PageRank flow. They cannot track and measure it — which means they don’t know when Google assesses the PageRank assigned to any given page on a site. If they knew that kind of information, they would be tracking and measuring PageRank. This is the simplest point in the whole debate and yet people continue to miss it.

Do you know how much INTERNAL PageRank Google will assign to any page on your site today, tomorrow, next week, or 2 months from now? Absolutely not. You don’t, you won’t, you can’t, and therefore you cannot sculpt your PageRank.

What you can do is hack your internal navigation structure so that Google and other search engines are less likely to recrawl certain pages as frequently as before. Apparently, enough people did this with such alarming success that Google felt compelled sometime in late 2007 or early 2008 to take the drastic action of changing how it computes PageRank.

Why? Because the brilliant PageRank Sculpting community was screwing up Websites’ search visibility. So let’s zip forward here.

The myth that you can improve your search results by nofollowing or otherwise blocking some of your pages from being indexed persists to this day. We could easily argue that it’s better to leave idiot SEO techniques to the idiots but I feel there is something morally wrong in that position.

Besides, I may one day be asked to help fix a site that has attempted to Sculpt PageRank. I’ve already reviewed quite a few sites in various Webmaster groups and forums where the owners admitted to sculpting PageRank (and they were all complaining about indexing problems — am I the only person to see a correlation here?).

You can’t do it. I admit I cannot do it. And I won’t be stupid enough to try to do it.

But what does all this have to do with the Supplemental Blues? I think I just made that point (twice) but let me put it another way: Sculpting PageRank increases your Supplemental Agony.

The first thing you can do when you’re concerned that your highly relevant content is being overlooked by Google is to see if you’re sculpting PageRank. Someone recently asked me why SEO Theory uses “nofollow” on its link to the root URL. All I can say is that it appears to be built into the Thesis them.

Is it a stupid thing to do? Yes.

Will I fix it? No. Why? Because I don’t care about it.

I can optimize any Website, with or without nofollows embedded on internal pages. It’s not difficult to figure out the solution for when you cannot change the internal structure of a Website that has indexing problems: get more value-passing links to point to the pages that lack PageRank.

I’m pretty sure that all the links people have pointed at SEO-Theory.com through the years will ensure that the root URL has lots of PageRank. Am I sculpting it? No. I’m just not concerned about whether PageRank affects the site’s ability to rank for its own name.

The recent hullabaloo over at SEOmoz concerning their invalid claim to have proven that “PageRank Sculpting still works” shows that people still care about PageRank. The SEO community is extremely hypocritical on this topic. They pay lip service to the principle of “PageRank is not important” but they don’t live it.

Well, I HAVE lived it. I had no idea that the internal links to SEO Theory’s root URL were nofollowed. I never cared about the PageRank. What is the point of caring about PageRank?

Does SEO Theory have pages in the Supplemental Index? I think so. In fact, if I had to bet on whether it does or does not, I would bet that it does. So what?

I had a much bigger cow when I learned that we were using the All-In-One SEO Plug-in for Wordpress. I absolutely hate that plug-in. It uses “rel=’nofollow’” on your category and tag pages by default. Those are some of the most useful pages a blog can have. Who in their right mind would want to block the things?

I find myself talking to Webmasters quite often about which SEO plug-in you need for Wordpress. When I tell them you don’t need any, people get defensive. Why? Because they have read on dozens of blogs and forums that if you’re going to create a blog you need Wordpress but “Wordpress is not SEO-friendly” and therefore you need an SEO plug-in.

Sorry, folks, but that dog won’t hunt. Anyone in the SEO community who is worth their salt knows that the Wordpress development community fixed all those custom field issues a long time ago. They got some very good advice on the matter from a highly reliable source in 2007: Matt Cutts (this is a video).

It doesn’t bother me to know I have Supplemental Pages. It does bother me if I am unable to get pages indexed the way I want them indexed. Sometimes you just need to point a few more links at a deeper page. Sometimes you need to look under the hood and see if maybe someone else’s idea of “good SEO” is pulling you down.

You cannot prove that PageRank Sculpting works because even if you COULD create a set of isolated sites that manage to direct the flow of PageRank as you intend, they would be completely unlike anything in the real world.

Doubling up a document’s PageRank won’t make it more relevant for more queries.

Doubling up a document’s PageRank won’t make it a better quality document.

Doubling up a document’s PageRank MIGHT get it to rank better for its primary keyword if it is stuck in the Supplemental Index.

The real issue facing Webmasters and SEOs alike today, however, is not how much Google’s Supplemental Index is hurting you — it’s how much you’re hurting yourself by implementing truly bad and unreliable SEO methods. You could be doing that right now without realizing it. If you do your job right, though, your site can tolerate a lot of mistakes.

That’s the most important point. It takes a really incompetent SEO to complain that an “About us”, “Privacy Policy”, or other so-called Incidental Page is so powerful it needs to be nofollowed, noindexed, or blocked by robots.txt.

If you are working with a site that has that kind of problem, and you don’t know how to fix it except by hiding the problem, YOU SUCK as an SEO.

And you can quote me on that.

The real-time Web will be grabbing people’s attention for the next few months, but something that many people in the SEO community seem to have done is walk away from their blogs.

I can understand if you’re too busy to write content every day but if you expect to monitor the way search engines handle blogs in a world full of tweets and SMS-based applications, how do you plan to do that?

Blogging is a chore — but it’s also still a great search visibility methodology that should not be lightly abandoned.

To be honest, I don’t have much time for blogging myself right now, but I’m determined to find a way to keep saying something and hopefully it’s something worthwhile, no matter how brief it may be.

Facebook is not yet ready to be your public platform. MySpace was never very good as a consumer-facing medium for pontificators, either. The plain and simple truth is that when you need to remind people that you exist, there is still nothing better than (or even as good as) a traditional good, old-fashioned Web 1.5 blog.

Don’t ever forget that.

Visible Technologies is on holiday for the Christmas vacation. We wish you and your families a happy holiday experience.

I’ve been sick for the past couple of weeks and the energy drain has kept me at a barely functioning level. I’ve stopped blogging across the board both personally and professionally. I managed to crank out a couple of posts at the beginning of last week for Best SEO Blog and SEO Theory but only because I felt a sense of urgency. Writing those articles really depleted me.

Today I feel better but not so well that I actually want to think.

And yet, in making my morning SEO blog rounds, I find I continue to be disappointed on a couple of fronts.

Russ Jones has been so busy he’s down to posting about once a month. I’ve asked him privately to resume posting but I understand how offblog life needs to take precedence.

Microsoft’s search blog, however, is committing a cardinal sin of communications. They haven’t posted anything to the Bing Blog since December 3 and their last post is titled “A Note About Today’s Outage”.

Is it obvious to anyone other than me that the message in that title is anything but positive and reassuring to casual visitors? If nothing else, Bing needs to wish everyone happy holidays.

It wouldn’t hurt Russ to do the same thing. But to be fair, there are other blogs where you seldom see anything new: Ask.com, Yahoo! search, Cuil — they all need to blog regularly. These are search engines, and they should be reaching out to the public on a regular basis.

Leaving people to stare at your last blog post for weeks or months on end not only destroys your online audience appeal, it forms a lasting impression. People keep seeing the same post over and over again.

At some point we’re all going to have to sign off and say good-bye. A Grand Exit post would be better, in my opinion, than an opinion or analysis piece that will become obsolete within six months to a year.

Of course, shutting down a Website is a topic unto itself. Maybe I’ll write about that…when I’m feeling better.

Google announced real-time Web search this morning. Because I don’t have time to come up with two blog posts, I’m going to refer you to what I wrote about the real-time spam implications over at SEO Theory: Google launches a new spam industry.

Welcome to the New Age of Search.