Hopefully my mobile friends will allow me to switch to the other part of this blog’s title and talk about natural search again, after spending so much time on mobile-specific SEO. Before I start I want to remind my readers that this is a personal blog with a disclaimer stating that these are my opinions and not necessarily those of Resolution Media or our parent company, Omnicom Media Group.
I don’t usually post about what’s going on in the search industry because I’ve historically been client-focused, but as I start to speak at more conferences, meet with more SEOs of all types at the monthly Chicago SEO Meetup and write more articles on search, I’m noticing how obvious it is that there are people in this industry who seem to do something completely different than I do, and whose opinions on search seem almost entirely irrelevant as a result. Oddly these are some of the most popular SEOs in the industry. On the other hand, I’m honored to have met many in this industry whom I respect a great deal, and whom I have become a better marketer as a result of reading or speaking with.
To illustrate, and to promote the kind of example that I think is worth promoting, I want to highlight two discussions about brands that happened just recently.
The first discussion most people in the industry are familiar with: Aaron Wall’s reported Google update in which Google is preferential to brands on a large scale. I wrote why I disagreed with it on the FindResolution blog on Friday, and participated in a related discussion on Sphinn. The crux of the argument was that Google made a large scale update similar to Florida, but bigger, and that the update was preferential to “branding”. Matt Cutts mentioned on Sphinn that they did change the algorithm for certain queries, but denied a large-scale update. I mentioned some examples, along with Danny Sullivan and Vanessa Fox, that didn’t fit neatly into this theory, and said that I didn’t agree there was a large scale update, and that brands might be appearing in results more often because brands do SEO. The conversation around it was vague and unstructured, the responses were generally irrelevant if they were given, logical fallacy of argument from authority was used to justify a fallacious argument, and many SEOs started at the conclusion that the theory was right before examining the evidence for it. When I mentioned some of these things on Sphinn, the responses were generally childish ad hominem attacks, responses that were ignorant of what had already been written or loose justifications for an algorithm change or some lesser point that the original post did not argue. At the end of the day, it became obvious to me that there wasn’t much of a discussion happening, and that some people were not looking for the scientific method or analysis because they were already believers in the authority of the speaker, and weren’t interested in anything that fell outside of that belief. I must say, that if I thought this was the only face of our industry, I would be busy trying to find another job.
Fortunately, another conversation about brands got me rethinking my career change. Search Insider Gord Hotchkiss, whom I was fortunate enough to meet at SMX West, talked in his column about brands as well, but did a few things that Wall didn’t do:
- He used the scientific method when forming a hypothesis, and defined very clearly what he meant by brand. He listed the Wikipedia definition and he explained how he was speaking of it, specifically. This is important because with the Wikipedia definition of brands, Aaron Wall’s post would have meant that Google was doing a large scale update to be preferential to just about anyone. It seemed in context that he meant big brands versus small, but he never defined how big a big brand has to be to qualify, or how small. By defining his terms off the bat, Gord makes his argument clearer and stronger.
Wall generally didn’t follow the scientific method at all in his post, as he skipped the first step and the last step, and didn’t draw conclusions based on the evidence, but stuck with the hypothesis in spite of data that didn’t fit in.
Here’s the scientific method for those who need a refresher:
- Define the question
- Gather information and resources (observe)
- Form hypothesis
- Perform experiment and collect data
- Analyze data
- Interpret data and draw conclusions that serve as a starting point for new hypothesis
- Publish results
- Retest (frequently done by other scientists)
It’s true, SEO is not rocket science, but I agree with Rand Fishkin that sticking to the scientific method more often when making hypotheses would increase professionalism in this industry, and help us all have more informed discussions and reach better conclusions.
- Another thing that Gord did was ask a question that made a bold statement, rather than make a bold statement that didn’t hold up to questions.
I know that many SEOs like Aaron Wall do what they do because it gets them inbound links, but to me it’s the difference between yellow journalism and Pulitzer Prize winning journalism, or between the Jonas Brothers and Mozart. It may make them popular, but it doesn’t mean that it holds up to scrutiny. And if it doesn’t hold up to scrutiny in the short-term, it’s unlikely to lead to long-term popularity.
Better, in my mind, to take a more Socratic approach, and try to solve problems by asking questions until you get to hard truths.
Gord does this by asking the question “Can brands keep their promise in a digital world?” and then proposes two potential ways in which brands might be able to keep their promise. It doesn’t make what he says any less authoritative, as it’s clear he’s an authority by having the foresight to ask the big questions, but it’s a different approach that tends to appeal to a different audience.There’s a book called Leading with Questions that argues the best businessmen are those who ask pointed questions rather than make bold authoritative statements. I would argue the same is true of the best businessmen who practice SEO.
- Gord’s answer to his question didn’t focus on toolbar PageRank, reverse engineering or gaming the system in order to make fast money because he’s a different type of marketer. In my experience his columns are generally forward-thinking in that they recognize the challenges all brands face in marketing to a brave new digital world. And they focus on long-term brand impact of these challenges rather than short term gains from loopholes that don’t focus on relevance. The answer to the problem facing brands, Gord suggests, is to pay attention to mobile and social media. This method probably isn’t going to get anyone fast disposable traffic and help them work at home, but it is a problem that brands are going to have to solve sooner than later if they want to remain relevant to their consumers.
Aaron Wall’s supporters call him “the Professor”, and this may be the case. But I think he uses different methods than the professors that I follow, and he may be teaching a different class. The professors that I follow use the scientific method to form hypotheses, they focus on long term brand impact and how brands can succeed in a changing digital media landscape, and they ask pointed questions rather than pose grand theories. There’s room for all search engine marketers to follow this example, but right now I think only a few do it exceptionally well. Unfortunately those who do are not necessarily the most popular, but I would argue that they’re the best.
In order to tip the scale a bit, and alert people to the professors that I have consistently learned from since beginning my SEO education in 2001, I’ve listed a few of these teachers below. This list is by no means complete, and should include about everyone at Resolution Media, where I’ve had the pleasure to focus on SEO for brands of all shapes and sizes since August of 2005. It’s also focused on SEO, and doesn’t include my many friends in the mobile industry.
listed alphabetically
Jeremiah Andrick, Microsoft
Jonathan Ashton, Agency.com
John Battelle, Federated Media
David Berkowitz, 360i
Jeff Campbell, Resolution Media
Vint Cerf, Google
Matt Cutts, Google
Rand Fishkin, SEOMoz
Vanessa Fox, Nine By Blue
Aaron Goldman, Resolution Media
Gord Hotchkiss, Enquiro
Avanash Kaushik, Google
Lance Neuhauser, Resolution Media
Paul O’Brien, Zvents
Steve Rubel, Edelman
Chris Sherman, Search Engine Land
Greg Sterling, Search Engine Land, Local Mobile Search
Danny Sullivan, Search Engine Land
Eric Ward, Search Engine Land
Awesome post Bryson – as usual 🙂 Thanks for all your work!
Hi Bryson,
We do know each other personally so you know this is a straight from the heart comment.
Aaron Wall without a doubt is one of the top 5 bloggers of SEO in my book. Having said that, I would not call him a professor per se as a part of what you described as a scientific method. What I do not understand in your logic is that, SEO is not physics. It is an ever stormy sea of logic where the waves are guided by the search engines. There is no scientific logic behind the ever changing algorithm other than the logic tagged by the search engines to fill their own pockets. I believe the man has a great instinct for upcoming change. He has proven that over the years with his “fortune teller” blog posts. Having said that, I do not believe you become a great SEO (even though you might be good) by doing a scientific method of testing of the present. You become a great SEO by having that the natural instinct for the future by being able to decipher the very small movements of today. I strongly believe that cannot be done by the traditional scientific method. If you try to get ready for the future by experimenting on today. You are already behind. That is my humble opinion. See you at the Chicago SEO Meetup.
Hi Mert. I appreciate your honesty and see your comments as an example of how two people who disagree can still treat each other with respect, which has been lacking from many I’ve encountered who share your opinion on this subject. This might be a good topic for discussion at the SEO Meetup, so we can continue this discussion in detail offline if you’d like, but I wanted to clarify something about the scientific method. It can be applied and tested even though the practice of SEO is both an art and a science. In fact, as I mentioned, Aaron Wall presented his branding theory as a kind of scientific argument; he just left out some very important parts. I’m arguing that including these missing parts can help to make claims verifiable and help us all have more informed, more productive discussions.
For example, here is the part of the scientific method that Aaron Wall presented to the SEO community:
1) Form hypothesis: Google has done a large scale update to their ranking algorithm to allow more brands to rank.
2) Perform experiment and collect data: Take seven Rankscape examples.
3) Analyze data.
4) Interpret data and draw conclusions: Seven rankscape examples and quote from Google CEO indicates that Google has, in fact, performed a large scale update.
5) Publish results to SEOBook.com.
And here’s the same process with three new steps, adhering more closely to the scientific method:
1) Define the question:
a. Initial question: why does it seem as though more brands are ranking in the SERPs?
b. Implicit question: What is a brand?
i. Fortune 500 company with annual revenues over x.
ii. a collection of symbols, experiences and associations connected with a product, a service, a person or any other artifact or entity.
iii. Publicly traded company
iv. Publicly traded company with annual revenues in excess of x.
c. Revised question: Has Google made a large scale update in favor of Fortune 500 companies with annual revenues over x?
d. Implicit question: How large is a large scale update?
i. Bigger than Florida
ii. Florida (% change in results)
iii. Smaller than Florida
iv. More than just a change to the ranking algorithm
e. Revised question: Has Google made an update to the algorithm that is bigger than x% change in ranking favoring Fortune 500 companies with annual revenues over x?
2) Gather information and resources (observe):
3) Form hypothesis: Google has made an update to the ranking algorithm that is bigger than x% change in ranking favoring Fortune 500 companies with annual revenues over x.
4) Perform experiment and collect data:
5) Analyze data
6) Interpret data and draw conclusions that serve as a starting point for new hypothesis: The ranking percentage for the collected sample was/was not greater than Florida for listings of brands that are Fortune 500 companies with annual revenues over x. This means that there probably was/was not an update bigger than Florida that favored listings of brands that are Fortune 500 companies with annual revenues over x.
7) Publish results to SEOBook.com
8) Retest (frequently done by other scientists): Others on Sphinn.com find several examples that seem to match Wall’s theory, some find examples that don’t. New hypothesis is formed and new tests performed.
My argument is that Wall’s methodology could be improved to make what he’s arguing stronger and more measurable. Now he relies largely on his SEO celebrity, which works for him among people who believe what he says and thinks he’s generally insightful, like you. If he’s not preaching to the choir, however, his argument doesn’t really hold up. I’m with Rand Fishkin, another authority in this industry, in that I’d like to make us all more rigorous and constructive in our debates so that we can come to more meaningful conclusions. There’s always going to be an art to some aspect of SEO, but we can better understand what’s art and what’s science if we’re not lazy in the way we construct and test a hypothesis. It’s my opinion based on the evidence that in this case Wall was lazy because his authority allowed him to be. I want to avoid that in the future.
Does that make more sense? We can discuss later, if necessary. We may ultimately disagree, but if we use rational argument we’ll have a better chance of communicating our ideas, and coming to more honest conclusions.
For example…
1) Define the question: Is Google’s ranking algorithm based on some sort of ruleset, and do those rules change based on what brings in the most revenue?
2) Gather information and resources (observe):
3) Form hypothesis: Google’s ranking algorithm is not based on logic but whatever brings in the most revenue
4) Perform experiment and collect data
5) Analyze data
6) Interpret data and draw conclusions that serve as a starting point for new hypothesis: Data indicates Google not making as much money as they could be with their ranking algorithm. Does this fit in with the hypothesis? If not, what is our new hypothesis?
7) Publish results
8) Retest (frequently done by other scientists)
There’s another question about whether we have time to be rigorous, given the speed at which things change. This really depends on your resources. I think we would have better instincts if we understood how our industry worked, and if a company had the resources to hire people for research and execution, surely they would be able to compete in the marketplace at least as well as the SEO who relied on the opinions of demagogues to make business decisions. No?
Anyway, thanks for your comments. Always welcome. Best, Bryson
If any company in this world had those kind of research resources, we the SEOs, would probably be unemployed by now. Simply too many dimensions and unknowns for a private organization without super computers to agree. I for one have gained a whole new interest in the mobile algorithm thanks to you. You give the example of Rand. He believes that he can solve Google by simply creating his own index. Where does his logic fail? He starts with certain assumptions and runs with them (trust and quality of links etc). Then he actually “outed” the first SEO company,
saying they spammed Google by “link building” because it clearly does not fit into the assumptions for Linkscape. That day was the day that made me realize that I donot care how much of a venture capital backing you have. Science only goes so far in SEO.
Of course it was a way to draw out Google to do something about the “non quality” links that he claims that dont help you. All SEOs work on assumptions in this world. An assumption can be claimed to be a hypothesis; but honestly 99.9% of the corporations in
the USA do not have the resources to scientifically prove beyond a doubt enough pieces to explain the entire picture of a search engine algorithm. So all we are left is instincts based on assumptions which is based on limited data. That my friend is truly what an SEO really has to deal with. That is far away from the standard we have strived for but those are the cards we are dealt with. Here is a point for you though, Aaron Wall’s last guest blogger made no sense to me either. Hugo Guzman’s last guest blog post at the seobook blog was probably worse “scientifically” but there are really just too many moving pieces that move too fast for my brain to be “scientific”. By the way Hugo, I loved how you put the anchor text for your seo company.
Thanks for a great discussion (debate) Bryson and Mert.
This has been a very informative post and it is nice to see a discussion where your professionalism has stayed in tack (lacking on many of blogs that I read where people disagree with each other).
I also want to personally thank you again Bryson for listing the names above: you’ve just given me more places to read, learn and grow. Thank you.
Regards,
Karl.
Thanks again for your comments, Mert. Since you didn’t argue against the scientific method I’m guessing you now agree that people who make theories should now try to be more scientific about creating and releasing those theories? Sure, there are assumptions, but by identifying what our assumptions are we are at least able to rationally construct hypotheses about the facts while understanding that what we’re left with is still just a scientific theory. As I see it, the difference between the theories that I’m discussing and the theories that are too often bandied about in our industry is that the theories I’m discussing are more democratic in that they can be debated and discussed by rational people, and the others are dependent upon the insight of an authority to validate them. In this specific case, a lot of people agreed with Aaron Wall’s theory simply because of who he is, but when the facts came out it became clear that his theory didn’t hold water because his assumptions were not clearly defined. If he had used the scientific method and defined the questions he was trying to answer early on and the criteria by which those things can be measured, he might not have wasted the SEO community’s time with his speculations. Well, to be fair, it was apparently a theory that appeared on Webmaster World some time before Wall popularized it, so it’s probably not his fault entirely.
It seems from some of these comments that you’re still thinking of this in terms of art versus science, which, again, is really not a point of contention here. Clearly there will always be assumptions. Clearly SEO will never be an exact science in the sense that we can reverse engineer the algorithm and figure out what is going on. I agree with this, and I’m arguing just the opposite in my third point in the post. I don’t think businessmen who help themselves or clients gain visibility in the search results need to reverse engineer the algorithm in order to do SEO. This is a point that sets me and the colleagues that I’ve mentioned apart as SEOs. It’s necessary to understand when big changes have been made so that we can communicate those to clients and to each other, but we don’t need to understand everything about the ever-changing algorithms in order to help clients generate a positive ROI in the search results. We only need to have an expert understanding of our audience including their search behavior, of technical issues that would keep engines from accessing content, of techniques that the engines consider unethical that could have our content removed from the index, and of authority sites/communities/users/consumers in a given industry and how to make our content appealing to those entities. I’m not interested in every ranking algorithm change that the engines make, because sites are helped in the long-term simply by my understanding of these five things. When I speak about these five things, however, I’m going to use the principles of the scientific method, because I want everyone to understand the argument that I’m making and why, so that they can help me move the level of discourse forward.
A couple of points about resources: honestly, would it have taken additional resources for Aaron Wall to have defined the questions he was trying to answer, as I did in five minutes in my last comment? He’s an authority in this industry because smaller companies don’t typically have the money to hire a full-time SEO or the time to do it themselves, so they buy his SEO book or read his blog in order to compensate. Smaller companies who don’t have the resources depend on Wall and others who do SEO full-time and blog about it in order to get good information on how to become visible in search results and reach likely consumers who search. So I think you’re right that people like him wouldn’t exist if companies had the time to do their own research and form their own hypotheses, but I think in this case it’s because they would see that he was irresponsible in not doing the three things I mentioned that experts in this industry should do, and that his hypothesis was a failure as a result. For the companies that have the resources to hire full-time SEOs and for those of us that are fortunate enough to work as a full-time SEO on a team of full-time SEOs for companies who often have full-time SEOs, we shouldn’t be satisfied with lazy arguments. The only way that we are going to learn from each other and help our sites succeed in search results is by taking this conversation to the next level and adhering more to the principles of the scientific method when constructing arguments. There are companies and individuals like the ones I mentioned here that are already doing that. All I’m saying, essentially, is that we would benefit as an industry if there were more supposed authorities in the space who did it more often, instead of relying too heavily on the persuasive benefits of their micro-celebrity.
Again, we can discuss this more at the SEO Meetup this Wednesday. It should be a good one, as Chris King of Agency.com will be doing a presentation on long term ranking for competitive terms. Hope to see you there!
Bryson,
Thanks so much for the vote of confidence in the work my team has been doing at Microsoft. I have been in the shoes of many SEOs and hope we can help educate and add to the conversation. I also feel really humbled to be in that list. Those are some of the most respected names in the industry, many of whom I have learned a lot from. Thanks again and next time I am at a show please track me down and introduce yourself.
Jeremiah