If you want to rank your site and get search traffic, you need a list of keywords you can target.
What’s the most effective (and cheapaest) way to do keyword research for your new blog posts?
Many people use the Google Keyword Planner trying to find profitable keywords.
And I have seen important people saying, "if you are unable to get results using free keyword tools, then you will not get any results using paid keyword tools".
This couldn’t be further from the truth.
For free accounts, many times the answer fails. “There was a problem retrieving ideas, please try again.”
This is not a bug - the purpose of the tool is to sell more Google ads, free users are not priority.
If you have been following internet marketing for a longer time, then you know about the misuse of the freely provided keyword data to rank crappy content and scam products to get rich overnight.
This is one reason why such data was restricted.
Also, the whole purpose of the Google Keyword Planner is to sell advertising, and not worry about the long tail keywords without many searches.
Therefore you don’t get an exact estimate of search volume from the Keyword Planner.
You only get ranges, 0-100, 100-1K, 1K-10K, 10K-100K.
And even in those ranges, keywords are grouped, which makes the data even more incorrect.
Also, the competition score you get is for PPC, which only helps you to determine the profitability of the keyword, but not the competition.
Therefore using the Google Keyword Planner is not ideal at all.
(This post contains affiliate links.)
Keyword search volume is the number of times (or volume) searches are conducted for a certain keyword for a specific timeframe (often on a monthly setting).
There are many keyword research tools and there are further reasons why you shouldn't blindly trust keyword traffic estimations.
There's also the point of usefulness. Number of searches may have nothing to do with traffic.
Ahrefs took nearly 100k search queries and plotted them on a graph, where X is the search volume of that query and Y is total search traffic coming to the #1 ranking page:
Obviously, there is a connection between the search volume of a keyword and the total search traffic to the top-ranking page gets.
However, there are many cases where pages that rank for high-volume keywords don’t get a lot of traffic.
There are also many cases where apparent low search volume keywords get lots of traffic.
Depending on the niche you’re in, your marketing goals, and even your budget (among other things), the phrase “good search volume” can mean very different things.
In the the same niche, the search volumes vary greatly because you’re targeting different countries.
There are also scenarios where it makes sense to ignore the search volume and just target a keyword anyway.
For example, if you found keywords with strong buying intent, then it makes sense for you to target them even if they have a low search volume, in fact I am getting hundreds of dollars in commissions from this blog by targeting keywords that have 20 searches/month with this post about WordPress themes.
Previously I didn't see much difference between the main keyword tools.
I thought, whichever you use doesn't really matter, because they're all using the same data and they all have similar features. I was concentrating on price.
Most people think all keyword tools get their data from Google, which means the following sources:
However this is the datum that woke me up, and it is something that kills your profits:
Google only shows 590 possible related keywords for the word "iphone".
SemRush shows 850,000 related keywords.
Ahrefs shows over 3 million!
There are advanced keyword research tools (Ahrefs, Moz, SEMrush) that operate a keyword database of their own and therefore will give you a lot more keyword ideas.
It turns out that Ahrefs has a far superior tool.
I will give you more on this later in this post, but do you see that you (and other bloggers) miss profitable keywords by using the Google keyword tool?
The good thing is, if you know this, you can get an advantage over your competition.
There are keywords that look so good.... However you will never get a sale from them.
Don't be fooled...
I am sure you have experienced that Google's new SERP features make it unnecessary for people to click through...
Because the answer is right there on the search results page.
And there are other factors. People just don't buy certain products online.
An example - would you ever buy a washing machine online? Or a $5,000 diamond ring?
The other type of keywords to avoid is where e-commerce sites are ranking.
You might find a keyword with difficulty 19, which you think is amazing, and you find out you are competing with eCommerce stores - Google gives advantage to these in for certain types of keywords.
I was shocked when I saw the below search distribution diagram by Ahrefs.
In March 2017, they analyzed their entire (1.4 Billion) US keyword database to get some insights about how people search.
As you can see, 93 percent of all search terms are searched for less than 20 times per month.
You can probably guess that big companies only target high-volume profitable keywords.
Around 20% of search queries that get asked every day have never been asked before.
And I am not talking "theoretical". The screenshot you see below is from FunnelXpert, 3 months after I purchased the domain.
It's bottom of first page ranking, long-tail, low-search keyword.
A few things I need to add for completeness:
Many people use number of competing pages or number of times the keyword occurs in the URL to judge competition. Now here’s the question: Is it the number of competitors or the strength of the competition that determines your success?
The number of competitors is completely unimportant and irrelevant. As people rarely click on the second or third page results, the only thing that counts is if you can get onto the first page of Google. Therefore in in order to judge competition, you only need to analyze the top 10 results.
How do you analyze these results?
The most important Google ranking factors are links, content and user behavior.
To complicate things, it is not just the number of the links, but also their quality, the number of referring domains and their anchor texts matter.
As per an Ahrefs analysis, page level ranking factors are stronger than domain level ranking factors. The strongest ranking factor is actually not links, but the number of referring domains to a certain page.
So it is possible to beat a high DA site by optimizing and linking to a single page.
You can beat a page with lots of links if you have more relevant content. Putting your exact-match keyword in the Title/URL/Headline of your page used to give a strong competitive advantage. This now has less importance, as Google is better able to tell what your page is about, based on LSI keywords.
So in order to rank better than the pages with lots of links, you need to make your page more relevant than those in the top 10 results. A deeper, more complete blog post, for example.
The purpose is that when your visitor reads your content, the buck stops there, in other words, he should have no more desire to look for another answer, because all his questions have been answered.
As an example, if you enter “fruit”, you get all kinds of listings relating to the definition of fruit, Wikipedia articles, etc.
However if you enter “silence”, you don’t get a definition. You get listings related to the movie called “Silence”.
Why is this difference? It is searcher intent. Even though the definition of silence would be relevant to the search phrase, experience shows that this isn’t what the users are searching for.
And you need to take this into consideration in your keyword research, because it may make it impossible for your content to rank for certain terms.
This is decided based on user interaction with the search results – time spent on page, going back to the search results and checking another one.
For every keyword you would need to take the top 10 URLs, then get information for all these URLs. Authority of the domains, how many other websites are linking to them with what domain authority, is it an https website, is the keyword in the URL, number of shares on social media, etc.
Then you would need to take all this information and compile it into a number that shows the strength of the competition.
And then, in order to find the keywords you are looking for, you would need to repeat this procedure for tens or hundreds of keywords. Please realize that it is impossible to do it all by hand.
Therefore you need a keyword research tool to do this all for you in seconds.
Hopefully you see by now why the Google Keyword Planner is insufficient for this purpose.
The question now becomes, what tool can you use so that you get the results that you need, but at the same time don’t need to break the bank?
In terms of our purposes, there are two types of keywords:
As an example, if someone is looking for a recipe or the text of a song, they have no buying intention. Therefore this kind of search is difficult to monetize. We are looking for commercial keywords.
The other point is between head keywords and long tail.
The further the potential customer is in the buying process, the longer and more specific his search terms will become. As the searched keywords get longer, buying intention gets stronger.
This is another type of keyword to look into further, and this one is not necessarily easy.
So your company is offering water heater repair... but everyone is already using that keyword.
A related keyword that you need to find would be "no hot water in shower".
In other words, you look for the symptom and not the problem. Look at SEOCockpit:
To demonstrate the power of this strategy, here is another example: sales funnels. Some of the possible related keywords:
Isn't that crazy?
How did I find this? I accidentally ran into the phrase "how to get more customers" on a blog. Then I entered it into the Google Keyword tool, checked the keyword suggestions at the Google search box, and checked the related searches at the bottom of the search results.
KGR is a proven strategy developed by Doug Cunnington.
He builds lucrative niche sites for a living.
The key principle of KGR is to find under 250/month search phrases that are "under-served" by Google - and therefore you can easily rank for these.
Here is a short version of KGR:
Here is a more complete, webinar version of KGR:
As you saw above, SEMRush is a much better tool than Google.
There is an advanced strategy that involves finding forums that are relevant to your niche site and plugging them into SEMRush.
This will provide you with a goldmine of high quality long tail keywords for you to target.
Watch the video, you won't regret it...
SEMRush has a 14-day trial.
You have seen the data how Ahrefs returns more related keywords for your seed keyword.
And I never paid too much attention to its services, because they are "expensive".
What's the difference?
Ahrefs buys clickstream data. Clickstream data shows people entire online journey, such as where they click.
All keyword tools try to predict how much traffic you'd get for a particular keyword, and they're basing this off different types of estimates.
Ahrefs shows the most accurate "traffic" volume of all keyword tools, because it uses a model combining Google Keyword Planner and clickstream.
The reason Ahrefs is better is because their model for creating estimates uses more than one data source.
Aside from Moz, no other tool uses clickstream data to calculate keyword lists and search volumes, which is why Ahrefs ends up with a much more substantial database.
Take note that Ahrefs has its own keyword data base.
It's well known that in any SERP, the website in position 1 would get the majority of the clicks.
Tools will estimate the traffic by looking at a site's search position, and based on that they use a percentage of the overall search volume for a keyword.
Ahrefs also does estimates like this, but thanks to clickstream, they can see the real CTR curve of every keyword, rather than applying general industry averages.
Also, a keyword with lower search volume can actually get higher traffic. People won't necessarily click through...
While other bloggers are going for the keywords that get the most theoretical traffic, if you use Ahrefs, you can optimize your content for the keywords that get the real traffic.
Ahrefs Keyword Explorer gives you the following types of keyword ideas:
When you have your list, you can drill down with search volume and keyword difficulty.
Keyword difficulty is tricky, because what may be “easy” for one person might be “insanely hard” for someone else.
How do your tell what keyword your blog can rank for?
You can identify your “safe” KD range by looking at the keywords you currently rank for.
If most of your keywords fall into the KD 20–30 category, that means that you can safely target any keywords with KD up to 30.
How can you use this to your advantage?
Ahrefs has a 7-day trial for $7.
If you don't want to spend your time on keyword research or you just want the pros to do it for you, Human Proof Designs can create a keyword pack for your niche, using Ahrefs.
I want to explain to you why HPD keyword research is better than how everyone else does it.
They've been doing manual keyword research since they first launched HPD in 2013.
There are other done-for-you services that simply export keywords from any popular tool, and choose niches based on keyword difficulty scores.
This might look great to the beginner who doesn't know any better, and wants to see a bunch of low scores or "easy" ranking metrics, but the reality is, no tool is ever 100% right, and can sometimes be devastatingly wrong.
Manual keyword research is the most effective way of judging competition.
While you should use keyword difficulty scores as a filter to eliminate the more competitive keywords, it's only a filter, not a yes/no decision maker. After you identified possible keyword candidates, you need to look at the SERP results yourself, that's manual keyword research.
HPD is picking niches and keywords based on years of experience, not based on what some tool estimates claim to be a good or bad keyword. If you want to be sure, this is the way to go.
You want to look at the following points:
With SEO Cockpit, you can set the source of data to Google Adwords, Amazon, etc. By setting it to Amazon, you will get more commercial-oriented keywords.
They have a Keyword Competition score, which is 0-100.
This is a cloud-based application where the results for all keywords have already been generated.
You can even enter your own website and see how it would rank compared to the ones ranking right now.
The best thing is to watch the video:
The most basic plan costs $40 for a month, therefore it is worth collecting up all your keyword research needs and do it in one go.
A lot of marketers use Long Tail Pro, and they are happy with it.
Long Tail Pro bases its KC scores on Majestic data instead of Moz. Mostly using the Trust data from Majestic and getting rid of the Page Authority, Domain Authority and others.
Long Tail Pro has a 7-day free trial.
KWFinder can be used for 2 free look-ups per day, which is enough for you to test it.
The above screenshot is an example of how to find a long-tail keyword for which you can write a blog post.
On the left side you see the keywords along with their monthly searches.
On the right side you can see the actual SERP results. The green color means that it is still easy to beat that listing.
It lists out DA, PA, amount of links and Facebook shares for that particular page, along with the estimated traffic for that search position.
I hope the above helped you to understand more about keyword research.
Please leave a comment if you have more data.