Why You Have Every Right To Feel Frustrated By SEO

Estimated Read Time: 7 Minutes

Frustrated By SEO

I heard it again last week – “I am so frustrated by SEO!”

The comment came from a marketing manager who was shopping for a new SEO company. He had spent a lot of money over the few past years on two separate companies and saw no meaningful results. He was feeling pressure from upper management and wanted to be sure the advice he was being given moving forward would produce measurable results.

I heard words like “mystery”, “black box”, “black magic” and even a muttered “scam” and “rip-off”.

Thing is, SEO is not a mystery and it is certainly not black magic. But there is some truth to the suggestion that SEO is a black box making it very difficult for SEOs to align expectations with their clients. Let’s start with why we (SEOs) focus on Google, how Google works, and why predicting outcomes can be such a difficult and frustrating task.

Focus on Google

The first thing to know if you are a small business is that Google is the only search engine you need to worry about.

While there are hundreds of search engines around the world – each with their own way of deciding who ranks where in search results – Google has a near monopoly in the United States so it is best to focus your limited time and resources on them.

Then you need to have a high-level understanding of how Google works.

How Google Ranks Content

Google uses an automated ranking algorithm to decide who ranks on top. It doesn’t publish it’s ranking algorithm and it has no stated plans to do so. Google’s CEO (Eric Schmidt) said flat out in September 2010 that “it’s a business secret” and he has no plans to disclose it.

To be fair, Google does provide a helpful high-level explanation of how search works in the form of a Youtube video.

While helpful, the video is not a diagnostic tool and doesn’t unlock the secret to why you might not be ranking. It lacks detail about how any of these things relate or are measured and weighed. There is also nothing in the way of how-to information.

Exactly how Google uses all these factors to make a decision is somewhat of a mystery, even to Google engineers. That’s because machine learning algorithms are opaque as to which signals they use. No one can see inside the black box – even the programmers who originally coded the AI. Stephan Spencer, Search Engine Land

So how exactly, do SEOs figure out how the algorithm works?

How SEOs Decipher The Algorithm

Some SEOs study Google’s guidelines, publications, and patent applications full-time. They monitor trends, perform experiments, conduct correlation studies, and track Google’s algorithm changes over time. They survey the opinions of dozens of the world’s brightest search marketers. All this research gets shared, some consensus and patterns emerge, and we draw conclusions and recommend and publish best practices.

Bottom line, SEOs do not know exactly or verifiably how Google ranks web pages. While not a total black box, the specifics of what goes on behind the scenes is based on continuously updated, collaboratively assembled best guesses based on 3rd-party research, experimentation, and opinion.

By Krauss – Own work, CC BY-SA 4.0, Link

There is not always consensus and opinions vary, but most agree if you follow best practices, you’re likely to produce meaningful, measurable results and avoid Google penalties and bans.

Google’s Ranking Algorithm Changes Constantly

Google’s ranking algorithm also isn’t cast in stone. It changes constantly which means you should also expect fluctuations in your rankings.

Google has said it changes its ranking algorithm anywhere from 350 to 550 times per year. Moz, a well-respected SEO tool maker and thought-leader, estimates Google changes their algorithm 500-600 times a year. It keeps a running log of all the changes (we know about) since the year 2000.

So even with all the research, consensus, and lack of consensus that goes on around the ranking algorithm in the SEO community, some of the conclusions drawn could quickly be outdated.

Competition Is Tough

The growth in competition in recent years has made it increasingly difficult for small business owners to rank on top. It used to be fairly easy. You’d populate some keywords on your website and be done with it.

photo by Mike Fleming on Flickr

Problem was, it was so easy, everyone took advantage it and the quality of search results started to reflect that. Google noticed and began increasing the number and types of criteria it took into consideration when ranking websites and web pages. There are few, if any opportunities to game the system nowadays. If you’re in a small, non-competitive niche, you might be able to rank well with relatively little effort. Otherwise, it’s an expense, a lengthy proposition, and creative struggle.

There’s one other exception. You can pay to be on top.

The fastest way to the top is, and always has been, Google’s Pay-Per-Click (PPC) advertising platform, Adwords. Through a complicated bidding and auctioning process, you can position your website at the top of sponsored search results in less than a day. Problem is, it’s a very complicated process and you can end up spending a lot of money to achieve poor results unless you’re experienced and methodical.

That puts small business owners at a disadvantage. They can ill afford to put the time into becoming highly skilled at PPC. Nor can they typically afford to hire or contract for those services. The dabbling that many small business owners do in order to try to turn a profit often results in poor results and even more frustration.

Google Doesn’t Share Everything

Google has never openly shared its ranking algorithm, but it used to provide business owners with a lot more data – data that helped us better understand how prospects and customers were searching for our products and services. The last few years, Google has been tapering – no cranking – back on the amount of data it shares as well.

Google’s Keyword Research Tool – Google Planner

One of the keys to SEO success is figuring out how people are searching for you on the Web, and then incorporating that same language on your website so you’re eligible to rank.

Given Google’s near monopoly on search, it makes sense you’d want to go to Google to figure out the words (keywords) people use to search for your products and/or services. And that’s where the vast majority of keyword researchers used to go, to the Google Keyword Planner tool.

Problem is, in recent years Google has cut back on the amount and specificity of the data it shares. Google, for example, lies by omission. Rand Fishkin of Moz discovered in February 2012, Google chooses to hide some of its keyword information unless you specifically ask for it. Then in August 2016, Google decided to stop showing specific search volumes for keyword phrases and instead provide ranges. When you used to be able to see, for example, that 720 people a month searched for “a specific keyword phrase”, Google will now report something along the lines of 100-1,000 searches a month for that same phrase and all others that are semantic equivalents.

That means the tool has become much less useful unless you sign up and pay for an Adwords account or subscribe to one of the many 3rd-party keyword research tools that have sprung up on the market, many costing more than a small business owner would like.

Google Analytics

Then there’s Google Analytics. In September 2013, Google announced they were no longer going to share information about unpaid (non-PPC) keywords in their Google Analytics software – the most used analytics tool on the Web.

This information was the key to understanding how people searched for and found your products and services on the Web. The fact that non-PPC keyword data is no longer available puts small business owners at a serious disadvantage. Big business owners don’t have access to this data either, but they do have bigger wallets, and a bigger wallet lets you into the PPC game where you can learn through trial and error, what works and what doesn’t.

If you are a PPC advertiser, you still have access to keyword data. If you are not, you’ll see only a small subset of the words people use to find your website, and the majority of those are going to be branded.

Summary

There is no single, fact-based and verifiable explanation for the inner workings of Google’s ranking algorithm. The information we do have is based on research, experimentation, and opinion. It hasn’t been verified by Google and opinions vary.

Google has a near monopoly on search. The rules it uses for determining who ranks on top are “a business secret” and constantly changing.

Competition is a lot tougher than it was a few years ago and it will continue to grow for the foreseeable future.

Google (and the market) seems to favor big brands and paid advertisers. Small business owners often don’t have access to detailed keyword information – the words people use to search for your products and/or services – because Google chooses to hide it.

So yes, small business owners have every right to feel frustrated by SEO.