How search engines really work and how to rank
So, this article came about when I was reading some posts about how some people feel different search engines are “controlled” by the liberal and/or conservative media. It was a very spirited discussion; each side giving their views about why certain search engines filter and give biased results that hurt their particular “side”.
I was amused, but more by the opinions and how they were pretty much all completely wrong.
We’ve worked for political campaigns on both sides of the fence and can tell you we kind of wish that search engines would filter stuff at times!
The secret way search engines work
Sorry, there is no secret. Search engines use acres of computers, power plants of power and an army of people to make sure things run to do 2 things: “craw” and “index”.
Then, they use all those computers and algorithms to produce answers to questions.
That’s pretty much it. If you have strong opinions on how results are skewed, you would likely be shocked at the lack of actual human intervention in showing your results.
This is not like a 1940’s switchboard where Debbie the operator gets a call and routes it to the store she wants to talk to. Google processes 4 BILLION searches per day. With some of the machine AI for automated requests (like stocks and commodity pricing), they are getting nearly 70,000 queries PER SECOND.
If you are truly a nerd like us, you can see live stats for Internet search processing at http://www.internetlivestats.com/google-search-statistics/
Google basically acts like a big map of different “places” or websites. It electronically goes out to each of these “places” and electronically asks “what do you do?”, “what are your saying?”, and “what are you about?” Then, based on the information on the sites, it puts together an idea of whats in the site and “indexes” it, storing it in the massive data warehouses it has. Sometimes, the search engine “spiders” check out the site once every couple of months, sometimes hourly. This is the part that depends on the people running the site and how they present their data.
Here’s where the factors of how people put their information out there makes a HUGE difference. If Google or any other search engine can’t “understand” what a site is about or what material might be UNIQIE and relevant to the actual theme of the site, then it will often just bypass indexing.
Why Do My Favorite Websites Not Rank Well?
Also, a lot of “information” sites, like political sites and shopping sites simply grab information from other sites. These sites are called “Aggregators”. Back in the early days, these sites were a great way to rank because they had lots of information. Today, the search engines simply go “seen it, yawn”, and don’t give it any credit. This leads many people who are into whatever they are into (shoes, politics, news, finance, music, finance…) to feel their favorite sites are somehow being “ignored”. Well, they are – because they are not giving search engines new and unique content. Instead of an editorial staff writing new content and analysis, they are just re-hashing specialized news. Sorry. We hate this too because it makes our jobs harder and more expensive to do to get clients ranked because we always have to provide “valuable and unique content”, ugh.
Nerd alert: If you want to see how sites “aggregate” data, I’m including two “feeds” from both MSN and FOX News. These xml feeds allow people like me to insert the latest news. While this is super useful if I want to include information for webisite visitors to see things like breaking news or stocks, it doesn’t get you any real “credit” to speak of for the search engines (except time on site, one of the 200ish factors that Google uses for ranking).
Fox News: http://feeds.foxnews.com/foxnews/latest
How Do Search Engines Answer My Customer Questions?
Today, search engines are like answer machines. One of the websites (even before Google) that was around that was the first to take searches in sentence form was Ask Jeeves. It had the image of a butler that would “deliver” your answers. Ask Jeeves died in the Internet wars and dot.com bubble of 2001 and is now the fledgling ask.com.
I bring this up, because since the inception of the “internet” as it is today, people have been asking it questions. Google is now in the question business and not the crawl and index business.
This is the important part for you. Your prospective customers are asking questions about issues they want to solve. Whether it is buying a house, putting together a garden, or hiring an lawyer.
When you or another site presents material, putting it in the form of answering questions has a great impact for search engines ranking your site. However, these answers need to be in a form that is natural. Simply listing questions and answers won’t quite cut it. The search engines want to see “depth” nowadays.
Wait, don’t think they are that smart? Look up Latent Semantic Indexing on Wikipedia. Yes, I wrote a book on this and part of my Master’s Degree was involved in this. LSA is basically figuring out what is natural based on what is around the keywords you are trying to present. Google’s algorithms are getting better and better at filtering out junk content. This is all math and language so this is right up Google’s alley.
This is another reason why we get a lot of questions from clients and answer a lot of questions about why someone’s site is being “oppressed”. If you just talk about one thing, it’s a very narrow road. You’ve got to talk about things that are related to your main thing.
Let’s take kitchen remodeling. Besides making things beautiful, why would people want a new kitchen? Entertaining for work? Adding value to a home? You parents or children moving back in with you? There are tons of things that would help searchers and thus ranking in search engines if you expanded your content.
If you have a great conspiracy site, you might want to mix in some interviews and some things around your views.
Search is a Community
I put the conspiracy theory sites in the last paragraph for a reason.
Your search results are kind of like high school. It’s a little bit of a popularity contest (sorry it really is – an no- it’s not fair). When other important people talk about you, you become more popular.
When other websites “link” to you, that is putting a link to your website on their website, it shows the search engines that people think the information you have on your site is relevant to whatever discussion they are having. Additionally, when people on social media share your link, it has a positive (smaller) effect as well. We call those “social signals”.
If a small blog features your kitchen cabinet rehab program, it’s pretty cool and you might get some local business. If HGTV features you, you become instantly more “important” and will likely also get calls, but now the search engines take notice of you and you may see a bump in rankings.
Essentially, think of Internet links as kids in high school. If a couple of kids say good things about you, that’s great. If the star quarterback makes a speech about how cool you are at prom, you are now popular.
As you become more “popular”, the search engines pick you first for dodgeball and you get put higher in the rankings. This can take a while, and just like high school, you can “game” the system for a little bit by buying the star quarterback beer, but once people find out, you instantly lose your popularity. Google can literally “de-index” you if you play the game too hard. You simply won’t show up in ANY searches for your domain.
Don’t think it really happens? Talk to BMW, you know, they folks who make cars. They got de-indexed for about a day for gaming Google until all the shit could get cleaned of that big fan. This was years ago, but it happens. https://www.searchenginejournal.com/bmw-booted-from-google-for-spamming/2886/
If you provide real value to your “community” of people asking questions, you’ll do fine and you will start improving your ranking in the search engines as you become more “relevant”.
So, we’ve learned that search engines are basically huge math-juiced computer systems that try and figure out what people want to answer as they search the Internet and then give it to them.
It does that by taking a look at websites, using a system of over 200 points to figure out if a site is worthy of helping answer people’s questions, then delivering those results (hopefully with your site included).
There are a number of ways to help the search engines pick you. These include good content, structuring your website to make it easy for search engines to understand it, and then obtaining links from other popular sites to your site or at least getting social people to talk about you and share your content.
Easy right? We’ll maybe not. I’m currently writing a book on SEO and how to get your site on page one within 6 months. Stay tuned, I hope to have it out by summer.
If you really want some ideas and help now, I offer a free strategy session that can help you figure out some strategies and find some low hanging fruit to rank well. And…. Since you made it all the way through this article, I’ll do it for free! Click here for a free strategy session.