Organic Search Results are Getting More Difficult

In The Time Machine (2002 film), a holographic librarian called Vox 114 taps into the sum of all human knowledge. Although Vox insists that changing past events is impossible, he assists Dr. Alexander Hartdegen (Guy Pearce) in his quest to travel back in time and change history.  Found again 800,000 years in the future, Vox is still functional, educating new generations of people.

Now, nearly twenty years after the film’s making, Google is starting to look a lot like Vox, minus the holographic imagery. Its goal is to answer people’s questions and lead them to the best sources of accurate information, just as a librarian would do.

Google’s mission statement says:

Our company mission is to organize the world’s information and make it universally accessible and useful. That’s why Search makes it easy to discover a broad range of information from a wide variety of sources.

In its brief twenty-three-year history, Google has become one of the world’s most valuable companies. In 2020, Google’s advertising revenue totaled 181.69 billion U.S. dollars. With that much money at stake, it stands to reason that Google will do everything possible to reach its goal. The day a competitor does a better job organizing information and making it accessible, Google’s revenue will decline.

With that in mind, it is easy to understand how to partner with Google to participate in answering people’s questions. Simply create the best information possible and put it in a trusted place where people can find it.

But is it really that simple?

How Google Works

Prior to the arrival of the Google search engine in 1997, search was primitive. In 1994, Yahoo! Search was founded, offering man-made descriptions of web pages across the internet. Then came WebCrawler, the first search engine to index entire pages. Too bad it was so slow that it was impossible to use. What followed were a dozen or more attempts at worldwide web search, including Lycos, AskJeeves, Inktomi, Excite, Microsoft MSN, Ask, and many more.

Why has Google been so successful where others have not?

The answer lies in Google’s initial and continued investment in patents on software algorithms, starting with PageRank, an algorithm that analyzes hyperlinks (links) and link relationships.

PageRank’s purpose is to assign a numeric weight to links between pages. As explained in the patent, each element of a hyperlinked set of documents is assigned a weight with the purpose of measuring its relative importance within the set. The numerical weight the algorithm assigned to any given element is called the PageRank, which indicates the importance of a particular page.

With this single algorithm, Google had a mechanism to bring the best information to the top. Links between web pages became a natural voting system, albeit highly exploitable. That PageRank voting system is still in place today with a series of improvements that weed out the cheaters and improve search quality. From the start, PageRank is what made Google better. The other search engines never caught up and never overcame Google’s information superiority.

Through the PageRank algorithm, and subsequent algorithmic improvements, Google can determine both an individual web page’s importance and the authority of an entire site. The more links a website has from other important websites, the more authority it has. The more authority a website has, the more easily Google trusts its content.

That’s not to say that bad content on a high authority website will rank high in a difficult Google search, because it won’t. But given two equal pages of content, Google will generally rank the page on a high authority site over that of a newer website with little or no authority.

This is what makes creating and profiting from a new website so difficult. And it’s becoming increasingly more difficult as older websites gain authority and new websites come into the mix.

Website Authority vs. Page Authority

It is important to understand that Google ranks pages for a particular search, not websites. A website can have stellar authority, but if the content on an individual page isn’t superior to all others, it won’t rank at the top. But how is authority measured?

In the year 2000, Google released the Google Toolbar for Internet Explorer. The toolbar made it easy to use Google search directly from Internet Explorer and it showed a green bar indicating PageRank. The more green showing in the PageRank bar, the more you knew you could trust the content of the web page. A while later, Google started displaying PageRank data in the Google Search Console.

These actions by Google were followed by an onslaught of webspam. Once Search Engine Optimization (SEO) practitioners knew they could measure the benefit of hyperlinks to their pages, it was game on.

Have you ever received emails asking if you want to buy backlinks? Google is to blame.

Have you wondered why webmasters add the “nofollow” attribute to outbound links? Google is to blame.

Have you ever had people drop unrelated comments on your blog post with a link? Google is to blame.

PageRank scores reported through the Google Search Console were given the ax in 2009. Then, in 2010, Google started pulling PageRank from its toolbars on Internet Explorer and Firefox. Seemingly all-at-once, the secret sauce that SEO experts relied on to measure page improvement went back to being a secret.

Making PageRank public was a colossal mistake by Google. Good riddance PageRank. I don’t miss you.

Today, only Google knows the PageRank score for each web page, which it adds in with other factors that make up its ranking algorithm.

The Link Economy

For people doing SEO, the toolbar PageRank meter was the perfect gift. It was a complete disaster for the web and a trap for many SEOs.

With the PageRank algorithm exposed through the toolbar, Google established links as votes cast by the democratic web. Anyone can create web pages and cast their votes, which quickly became a link network Super PAC created to rig the voting system. Cheaters could game the system by buying links. Many organizations, both large and small, got caught in the game.

SEOs who went against Google’s webmaster guidelines, and purchased or manufactured links to game the system, soon found themselves on the wrong end of the search engine’s ever-evolving algorithms and improvements. Google has mastered the art of web spam detection.

In my own industry, eHealth’s Medicare.com website got caught violating the webmaster guidelines and Google slapped them hard. With the flip of a Google switch, a website with over 400,000 monthly visitors, with an organic search value in excess of $2-million per month, was reduced to rubble.

Even with the threat of Google penalties the link economy persists. In fact, it’s thriving.

But why?

From the start, PageRank was just one part of the Google search algorithm.  There are many other ranking factors. A high PageRank score never has meant that a page would rank well for a given topic. Pages with a lower score can rank better than pages with a higher score if they have other on-page factors in their favor. Primarily the quality of the content.

However, even as much as Google now plays it down, links to a web page are a factor and an important one at that. And this continues to drive the link economy.

In its Webmaster Guidelines, Google provides specific guidelines. Among these it says to avoid participating in link schemes:

“Any links intended to manipulate PageRank or a site’s ranking in Google search results may be considered part of a link scheme and a violation of Google’s Webmaster Guidelines. This includes any behavior that manipulates links to your site or outgoing links from your site.”

Although it seems like the statement above covers all methods of acquiring links, it doesn’t. Google doesn’t expect businesses and individuals not to promote themselves. That would be ridiculous. And there are a few valid ways of doing so:

1. Press releases
2. Content syndication
3. Guest blogging
4. Sharable content (e.g., Infographics)

However, what Google is really looking for is content that’s so good, so informative, so thought-provoking that other people share it and recommend it with a link. However, sometimes even the best content needs to be promoted so people can find it and share it. Press releases, content syndication, guest blogging, and shareable infographics are tools you can use to get your content found.

New Methods to Determine PageRank

When Google blasted PageRank off the toolbar, SEOs begged for new tools to help them measure and track the performance of webpages. Many services exploded onto the scene, including MOZ, SEMrush, Majestic, SpyFu, and Ahrefs, to name just a few. Over the years I have tried all the tools mentioned but ultimately settled on Ahrefs.

Why?

For me, it is the easiest tool to understand and use, and it gives what I feel is the clearest picture of web page performance vs. competing pages.

Ahrefs crawls the web, indexing web pages in the same manner as Google and other search engines. Using its own algorithms, Ahrefs creates 0 to 100 numeric scores for domains (domain rating, aka, DR), web pages (URL rating, aka, UR), and search term difficulty (keyword difficulty, aka, KD). For every page in its index, Ahrefs also tracks the keywords the page ranks for, its search position, and position history.

This is highly valuable information. It’s nearly impossible to compete in the world wide web landscape without it.

One of the most beneficial capabilities in Ahrefs is its keyword research tool. It allows you to find Google searches with volume and low keyword difficulty. The combination of which is a goldmine if the search has buyer intent.

Buyer Intent Searches

There are several different types of searches people commonly make on search engines:

1. Informational search queries
2. Navigational search queries
3. Transactional search queries

Wikipedia defines informational search queries as “Queries that cover a broad topic (e.g., arizona or motorcycles) for which there may be thousands of relevant results.” When someone uses Google for informational search query results, they want general information. In most cases, they are not looking for a specific site, or they’d put the site name in. And they are not wanting to make a commercial transaction. They just want an answer to their question, and Google does a very good job giving the user the information they want in a knowledge panel.

Navigational searches are exactly what they sound like. People are lazy and will gladly enter “youtube” instead of “https://youtube.com” to get to the video-sharing site.

Transactional search queries are those that indicate intent to complete a transaction. It could be a brand name product purchase (like “ford bronco”) or a product category (like “mid-size SUVs”). Often these search queries will include words like “buy”, “purchase”, or “reviews,” from which you can infer they are considering a purchase.

Many negative search queries are also transactional. For instance, people will often search for why a product or service is bad. When people do this it’s because they are researching the product before they buy.

Transactional search queries are those that are at the business end of your conversion funnel. These are the searches that will generate business. For this reason, they are also the most difficult to organically rank for in Google search.

This book is only concerned with transactional searches, and how to secure a spot on the first page of Google. Because, until you can do that, you won’t get any business. Period.

It’s Google’s Game. Play It or Die!

With an estimated 90+ percent of the online search market, Google owns the game. If you have a product or service business, you can pay Google to advertise or you can create content Google loves and earn its trust.

In the short term, advertising with Google offers immediate results. In the long term, creating fabulous content and gaining Google’s trust is more cost-effective. There is, however, one more option. You can partner with Amazon and borrow their trust authority.

How?

You write a book and educate people. That’s how.