Starting From Search

Yahoo Introduces Quality Score in Ad Ranking Starting February 5

Beginning on February 5, Yahoo will be changing the way they rank pay-per-click ads. The rankings currently us bid price as the ranking method, meaning the higher the bid price, the higher the ranking of a pay-per-click ad. This coming change will include bid price and a quality score to determine relative rank, similar to what is already in place with Google AdWords.

What is a Quality Score?
A quality score applied to pay-per-click ads means that bid price is not the sole ranking method. This means that an ad with a lower bid price could rank above a higher bidding advertiser. While the exact factors involved in the quality score have not been divulged by the engines themselves, one influence is historical performance and performance relative to other ads displayed at the same time. Simply, if more people click on an ad, it can be assumed by democratic process that the ad is more helpful to searchers and could get preferred rating.

The quality score was devised to try to keep the playing field even. In a pay-per-click process based solely on bid price, nothing can stop a large advertiser with big pockets from taking top spots. In theory, with a quality score feature that effect is minimized as a small advertiser could compete with Wal-Mart by strategically picking keywords and writing well-focused copy for customers. In other words, better quality ads help with better placement.

This process has been used by Google AdWords for some time now, and it does make pay-per-click campaigns a bit more of an art form than a simple function of calculating a bid price. If your SEM vendor can't tell you why your ad is not increasing in position despite bid increases, the quality score is the reason. (Interestingly enough, the quality score feature can also reveal which PPC vendors are truly savvy marketers versus those that practice "simple" SEM.)

Between now and February 5th, it would be a good idea to scrutinize your PPC ad copy. Also be prepared for some wild position fluctuations that week as well.

Another Google Killer…(Yawn)

While I was away on holiday vacation, I read two articles about so-called "Google killers" – an overused catch-phrase that should affect more people as partially ridiculous.

(Consider earlier recipients of the title, Teoma (here and here), Quaero (here), Baidu (here), and even Microsoft's Live.com (here), and... you get the point.)

And again this month, still none of these new search engine ideas are novel, or novel enough to be called a "Google killer". To me this idea sounds like someone inventing a new elixir and  calling it the "Coke killer”, or an operating system and calling it the "Windows killer." After all, even Pepsi hasn't put Coke out of business, as thorough as their competition and advertising may be.

This week’s entrant to the long line of "killers" is Search Wikia, created by Wikipedia founder Jimmy Wales. It is intended to be a "people-powered search engine" that taps into the essence of Wikipedia, meaning user controlled content.

A volunteer force like this has been utilized before and became an albatross for . Besides, a search engine with user controlled content sounds like it could quickly become a spam magnet.

Next to throw down the gauntlet is Powerset. Their innovative technology claim is that a user can type a search query in "natural language." Sorry, Powerset, but Ask beat you there by about six years.

The other problem Powerset faces is that people do not search in natural language sentences anymore, which is part of the reason Ask re-branded away from their "natural language" search feature. It's still there; they just don't tout it anymore. Why? Because users have long been trained to search with keywords. Powerset would first have to un/re-train this now natural behavior, and then demonstrate it's more relevant than Google. But that's not stopping one of its founders, Steve Newcomb, from saying that Powerset could "become the next Google."

Lest I be a stick in the mud, I'm not decrying the ambitious attempts of these upstart engines; I'm questioning the rave surrounding them. Such claims tell me that the product is focused on the competition. The best products, the ones that truly succeed, focus on the customer.

Don't tell me who you want your new shiny search engine to topple in a few years, tell me why it's useful to me (and my clients) right now.

Worried about Search Engine Privacy? TrackMeNot

With the accidental release of the search queries of 658,000 anonymous AOL users onto the web, people have begun to ask questions about their own privacy. While the search records were considered anonymous because they were identified only by number, several sources, including the NYTimes, were able to track down some of the actual people behind the search data.

A person does not have to be very paranoid to wonder if searches could come back to haunt them. The US government has requested search data in the past. How long before search histories are subpoenaed for a divorce proceeding?

It's actually the harmless searches that are more revealing. If I search on fear of flying therapy, it should be my choice on who I share that with, not AOLs (or Google’s, or Yahoo’s, etc.) There shouldn’t be a database of my fears out there somewhere.

That's where TrackMeNot comes in. It's a Firefox extension that pulls random word combinations from a list of words and submits them as searches, essentially, diluting whatever cookies and log files may be stored on your searches.

While I think this technology is interesting (and actually kind of funny), there is a downside. I'm concerned how a search randomization extension could ruin keyword research.

Keyword research is an extremely effective tool to uncovering what people are actually typing into engines when they are searching for products in your industry. If I have to start wondering if keyword combinations and traffic estimations are truly accurate, then the advice I give my clients could be compromised.

I think it's an interesting idea to solve the problem — dilute the log files so much that they are worthless. But couldn’t this problem be solved in a much more effective way? Oh, like, the engines realizing there is a privacy concern and not keeping log files for longer than a few days?

I would hope they draw that conclusion before someone starts a class action lawsuit to make it happen.

It's Not Always What You Typed...

...but why you typed it.

A snip from Scoble's rant:

When I search on “Office Furniture” why is the first thing I see stores? I don’t wanna see freaking corporate info. I wanna know what HUMANS like to use in their offices.

What Scoble just learned is that the difficulty from the search engine side is not matching keywords, but in trying to figure out the intention of the searcher. "Office furniture," while a logical term, is also quite general, thus making it harder to assume what the intent of the search might be. Each engine has an element of mind reading to do.

For example, a search on "red wine" could give a range of results. The engine has to try to determine if you are looking to buy wine, want to learn about wine making, or matching types of wine to food choice. It is the intent behind the search that leads searchers to be satisfied or unsatisfied with the results returned.

This is the reason why 53% of all searchers use up to three separate engines. We keep searching until we get the "right" results.

Yahoo Product Focused or Just Looking for Speed?

I was intrigued by a patent that was awarded to Yahoo this week. The patent mentions auction sites in regards to search engine results and relevance.

From paragraph eight: “Matching outdated, e.g, eBay listings pages to search queries can erode search user trust. The relevance problem that occurs when a query matches the title of an expired listing contributes to this.”

Patents are often written a vague as possible to hide the true intention of the idea/technology they are trying to patent. This patent seems to revolve around auction listings, referencing them no less than 10 times and specifically referencing eBay twice. It would make one wonder why Yahoo is focused on auctions. Are they concerned with delivering more product related returns?

Nope.

Obviously search result that goes to an outdated eBay auction would erode confidence. Not in that a user would care so much that the auction is over, but in that the search results were not fresh.

And that is what this patent is about – freshness. Auction sites provide a good example of the problem they are trying to solve yet is oblique enough to the true objective – indexing more sites much faster to provide very fresh results.

Why mask this? Online forums have seen their share of webmasters complaining about the amount of bandwidth search engines spiders consume – which is a problem since most hosting companies charge on bandwidth consumption. Jakob Nielsen covered this topic in the past.

Yahoo doesn’t want to bite the hands that feed them (webmasters) while trying to serve the freshest food (sites) to their consumers (searchers). I applaud this on many levels. As a search marketer, I want my client’s sites to be indexed as often as possible, and as an active searcher I want results that are up-to-the-minute fresh.

Yahoo is trying to find ways to push the gas pedal down while trying to respect bandwidth.

From paragraph 22: “This enables the search system to use update cycles which are best suited for particular types of content or listings, as well as reduce the number of queries required to maintain fresh and relevant content.”

In other words, update your site often and Yahoo is more likely to index more often. Yet they are clearly working on a system to reduce potential server load.

Technorati Tags: yahoo, patents, search results

Everything You Never Thought to Ask About Google

Baseline has an excellent primer on Google. It's lengthy, but a great read if you want to see how the giant works.

Link: Baseline: How Google Works

Technorati Tags: search, google, technology

Google Awarded Patent on "Voice Interface for a Search Engine"

Yesterday, Google was awarded a patent for a method to get "search results from a voice search query."

Overall, the application is described as:

A system and method consistent with the present invention address this and other needs by providing a voice interface for search engines that is capable of returning highly relevant results.

I can think of a few interesting ways this technology could be used -- some more realitic than others.

1. Making search easier to use for sight-impaired individuals. As search becomes more and more prevalent, I'd love to see engines embrace all aspects of 508 compliance. But the patent says this is meant "to satisfy the average user." That implies that it is intended for a larger, general audience.

2. Search via cell phone. Not everyone has a web enabled handheld device, but it seems like everyone has a cell phone. 1-800-Google? The results could be returned via a text message. The Google Voice Search page has been up for several months, but I don't believe this is the ultimate aim for this feature. It is to clunky and merely nothing more than a public test page.

3. Search via Google Talk. Probably the most far fetched, but Google Talk does have a VoIP feature. Imagine clicking an icon on the application, voicing a search query, and a browser opening to show the results.

I do not expect to see Google using this technology soon, as even the patent application mentions the difficulty of voice recognition.

Read the patent application here. I'd enjoy hearing your ideas on how Google could use this.

What Was Alexa Thinking?

Today, Alexa opened up their full data warehouse of information to anyone who cares to pay for it. For a fairly small fee, you can now utilize their spiders and processing power in whatever creative ways you can think to use it.

Did Alexa want to "democratize data search" as its CEO claimed? Will this cause other engines to commercialize their spiders? Does this mean that anyone can now make their own search engines?

No, no, and no. Let me explain.

Did Alexa want to "democratize data search" as its CEO claimed?
Open Source democratizes processes. Charging for a service is, well, a way to make money. Alexa hasn't been part of the search conversation for quite some time. Now they will be, at least for the remainder of this week. This is a way for Alexa to try to make themselves a relevant part of the search world again.

They are not going to make money in creating their own pay-per-click model. That's too time consuming for an engine with little market share to really make money at. When it comes down to it, this way an innovative way to offer a service they could charge for.

Will this cause other engines to commercialize their spiders?
No, unless this Alexa program is extremely successful. I would just be happy if the other engines beef up their APIs as a result.

Does this mean that anyone can now make their own search engines?
I'm seeing this speculation the most. Over the next few months, we'll see some novel tools come out of Alexa's Web Search Platform. But ultimately it will be too expensive to use for novel purposes. Yes, Musipedia may allow you to whistle a search query, but ultimately that is only useful as a curiosity.

Unless you can find a way to derive actionable data for your company/product, it won't be worth the cost. When the buzz settles down, it will be used by people looking for specific things. It will allow business to dig into web data in new ways to find new niches. Or as Seth Godin said, find needles in data haystacks.

Think along the lines of Lexis-Nexus. A storehouse of great information, but because it is fee-based, it becomes useful only to those that will balance ROI against the cost-of-use.

What Search Engines can Learn from Supermarkets

In a poll released in early May by Harris Interactive, consumers rated which industries they felt were doing a good (or bad) job of serving their consumers.

Supermarkets ranked number one in terms of consumer satisfaction with 92% of respondents saying supermarkets were doing a good job, as opposed to 8% who didn’t. Computer hardware companies were second with an 84% favorable to 10% unfavorable ratio.

Search engines held a respectable 7th place with 79% favorable and 11% unfavorable. (Not surprisingly, Tobacco and Oil companies came in last with two out of every three people saying they did a poor job.)

What can a search engine learn from a supermarket? Categorization.

When you walk into a grocery store you are presented many aisles – items divided in categories. Like foodstuffs are put together with signs in each aisle telling customers what is there. The end of each aisle has other items that could be related or just sale items. In search terms, I think of those as the pay-per-click listings. Easy to skip if you want, but sometimes very helpful.

Search engines have failed to grasp this concept. Most engines present information in a single, long list that spans many pages. That would be the equivalent of a grocery store having a single, long aisle. Shoppers would find that tedious and unhelpful, yet search engines have stuck to that very motif for years.

Search professionals fight for the first 10 spots for any keyword search. Statistics show that most searchers don’t view results on the second page. That makes sense, if you think about it. People are looking at what is up front, even if it is not particular relevant.

If I was a producer, I would want my products at the very front of the aisle because I know that the things up front would tend to be noticed more, even if they weren’t relevant to what the customer really wanted.

Look at this research by Thorsten Joachims, et al., at Cornell University [Link to PDF] and perfectly summed up by Jakob Nielson. After typing in a search query, a majority of searchers clicked the very first link returned even if it was not relevant.

This is why categorization becomes important.

Teoma beat everyone with this concept with their Results, Refine, and Resources method of listing results. While I would argue the results of the refine and resources are not always helpful, they have the right idea. They are trying to find ways to present relevant information to their users in a way that is also quick to browse – just like the signs hanging in the supermarket aisle.

Zoom with Jeeves
In late May, Ask Jeeves introduced Zoom, which they describe as the "next-generation related-search tool that gives users suggestions to narrow or expand their searches." They also claim it is the only search technology that clusters the web into topical communities in real-time.

It displays three sets of related results on the right side of the page next to the search listings. The first set lets you "Zoom in" or narrow your search, the next set lets you "Zoom out" by offering listings that are wider but still conceptually related, while the third and least useful of the sets offers names of people that are related to the original search.

I have also had mixed results with Ask Jeeves' Zoom tool. But I applaud the effort and hope it blossoms and becomes more wide spread. Hopefully over time, they will spend as much time refining their categorization tools as they spend on their ranking algorithms. Then search engines will be more useful than supermarkets.

Search Toolbars for your Browser

Last week Google has released the beta version of their essential toolbar for Firefox. I know many users that were holding out on switching browsers because they felt they couldn’t do without their trusty toolbar. Overlooked in this news however was Yahoo’s toolbar for Firefox.

Since many search engine companies play follow the leader, I will be interested to see if other toolbar applications, such as Alexa’s, follows suit. Most telling would be if Microsoft releases a Firefox build of their MSN Search bar.

Toolbars for your downloading pleasure (in no particular order):

Alexa Toolbar

Google Toolbar

MSN Search Toolbar

Yahoo Toobar for IE
Yahoo Toolbar for Firefox

AltaVista Toolbar

AskJeeves Toolbar

Dogpile Toolbar

Teoma Toolbar

Yahoo! Update

Tonight Yahoo! will be making an index update. Tim Mayer of Yahoo! Search indicated that this change would allow more (deeper) pages to be indexed and that pages currently in the index may experience some ranking fluctuations.

What’s this mean?

The major engines battle it out in a two-front war. The first is indexing “the deep web” with is essentially a fancy name for pages buried deep in a site. The second is providing those pages in search results in a meaningful way. The spiders work on the first part, the algorithms on the second. Data mining and data warehousing.

If you have implemented a sound, ethical search strategy then updates such as this are nothing to worry about. Page rankings will fluctuate as the algorithm needs to rank the newly indexed pages. But overall, if your site has a healthy optimization program, it will continue to pay dividends. Some adjustments may be needed to some pages, such as revisiting headlines or reworking some body copy, but engine adjustments such as this natural and should be welcomed.

Yahoo! Acquires Blo.gs

Blo.gs, a blog tracking service, has been acquired by Yahoo!. Blog.gs tracks about nine million online blogs, mostly via pinging, and has about 12,000 members.

In the past, engines have acquired other properties in order to quickly increase their database size or to nab interesting technology – such as when Yahoo! acquired Overture (for their database and pay-per-click model) and Inktomi (for their spider). I would suspect this is Yahoo’s way to quickly increasing the size of their blog indexing.

But before you rush off to create a corporate blog thinking it will help with your Yahoo rankings, check out my earlier post: Blogs and Search Rankings.

Finding Gifts Online with Search

Any time I start looking for specific gifts or gift ideas, I often turn to the web. In doing so, I found myself at the mercy of various facets of online searching.

For Mother’s Day I went to specific brand sites I know my wife likes and used their in-site search tools. In most cases, I found them to be woefully inadequate.

Many brands are focusing much time and effort (and rightfully so) on a search strategy for the major search engines while ignoring, or just not considering, how their in-site search works.

If you are going to focus attention on getting customers to your site from online searches, don’t over look providing them that same help once they get to your site.

In another case a few weeks ago I was searching for some specific cactus products. Many of the search results did not exactly match my search, so I clicked on a link to a cactus nursery thinking that once I was there I could find the specific products I was looking for. But the search capability was so poor I left the site in frustration.

Many in-site search functions suffer from the same flaws – they either return far too many results, making it difficult to find the one product you are looking for, or they return way too few results that do not relate.

Too Many Results
While on a site of a major craft retailer, I did a simple search for a specific product. Even after modifying my search terms to make them as specific as I could, I would get upwards of 800 products, spanning 14 or more pages. After browsing the first few pages, I lost interest in going any further.

Research supports that most online searches end this way – with a vast majority only scanning the results of the first few pages.

What was worse was there was no way, or guidance from the site, to narrow down my results. Offering the results in a format by category would have been more helpful, rather than a giant list of 800+ products.

Too Few Results
On other sites I would find the opposite problem – a return of 3 or 4 products that left me scratching my head as to how they related to my original search query with no manner to naturally widen my searches.

This type of problem is the dead end of in-site searches. If the results are very thin with no lead outs, similar or suggested items, then you are inviting a customer to leave in frustration.

The Balancing Act
There is no magic number of results that should be returned for any given search. Despite what I mentioned above, the functionality of the return is the important aspect.

If a search provides no results (or way to many), it helps to offer suggestions. A links to the most popular products is often helpful. Other helpful break-downs could be a link to gift ideas for the upcoming holiday, category or price break-downs, or a list of the 10 most recent searches.

First and foremost, use in-site search as a tool to help customers find the product(s) they are looking for. But also use it to show customers the breadth of your site and to provide ideas for products they may not have originally thought about.

Google Update Looming?

There is a rumor spreading across the land (or net, rather) that Google is making plans for another algorithm update. My prediction is that it is in response to "comment spam" in blogs. I generally have to delete several comments a day on this blog, all comments laced with links to gambling sites. (Though, I haven't had this problem since moving this blog to Typepad.)

Google does pay attention to which sites link to yours as an indication of popularity. It is assumed the more people that link to you, the more useful your site must be. In the last few years, link spamming has been a source of major abuse mostly at the expense of blogs as they have become more popular. Engines are being forced to make adjustments to this portion of their ranking equations before it becomes irrelevant.

Many engines have endorsed and put into practice a new instruction for the rel attribute for anchor tags. When building a link with the standard A tag, a webmaster can include "rel=no follow" as a signal to a search spider to not follow that link, and thusly not counting it in their equations. But that has not stopped comment spammers and their predecessors, link farms.

But getting back to the point, should we fear a Google update? No, we should welcome it. Should we fear our sites dropping in rankings? If you followed sound SEO techniques, you have nothing to worry about. If you are a marketing manager or webmaster that may have unknowingly fallen for and added your link to link farm, then you might see a drop in rankings as incoming links (at least from blogs and farms) may be devalued in ranking equations. But learn from it. (Sometimes a little SEO knowledge can lead you to make such honest mistakes, a topic I will be writing about in the next week. Consulting with an expert in this complicated and rapidly changing field is always a sound marketing investment.)

Engines change their algorithms all the time. I predict they do it much more often that we notice. I would not be surprised if it’s done on a monthly (or more) basis. In fact, I bet MSN is making daily changes since the release of their new engine two weeks ago. Changes in site rankings are part of this “wild-west” of marketing. Sites can shift position by the hour, and such changes are natural in this industry.

MSN Launches New Search Engine

Microsoft has finally delivered on its promise by developing a search engine. It came out of beta Tuesday morning.

I decided to take it for a test drive and the results were somewhat curious. As I typed in keywords associated with some of my clients, I noticed that some of the top ten rankings were actually listings from Google’s directory.

I found this curious since Microsoft’s venture is to now complete with Google. But if many of the results that are returned are going to be from Google’s directory, then Google might as well be providing the results. It also gives the appearance that Google, not MSN, has a better handle on the web.

The truth is that engines pretty much all steal from each other. If you are building a search engine, the fastest way to build a cache of sites is to point your spider Google and Yahoo!’s directory of sites.

I give the MSN engine a C+. The + is for the ambition, not so much the actual search results.

Microsoft ambitiously wants to compete against Google. Microsoft has done it before – coming from behind in the browser wars and driving Netscape out of business. But in all honesty, that had more to do with giving their browser away for free, thusly taking away Netscape’s profit model.

But I’m digressing here… Microsoft is still a distance from unseating Google. I expect MSN’s search engine to go through a few overhauls before it’s any real competition.

Smart Tags that aren't so Smart

Microsoft is at it again. This time by adding in a feature on their browsers called SmartTags. After doing some research on this issue I find them to be less than smart.

SmartTagging works as follows: a user pulls up a web site about vacations getaways. On that page the world “cruise” is underlined with a little squiggly line that, when clicked, takes you to another site with cruise information.

That is how smart tagging works. It is a feature of IE 6 that allows Microsoft to scan the contents of a web page and link out from nearly any word they choose. This affects a company in many ways.

First, it affects site sticky-ness. That’s a phrase site developers use to describe the ability to keep people on the site by offering interesting information and features. Generally I’m only working against the other web clutter to try to keep eyeballs on my site. Now I’m literally fighting against the browser itself to keep people on my site.

Second, let’s look at the impact from a brand perspective. When a user comes to your site, any link you put on your site to another site does reflect to a small degree how people look at you. If you have links on your site that are not helpful to your user, they will thusly come to the conclusion that your site was not instructive to them.

(This is also why I voice caution to clients when they get to aggressive in link building campaigns in order to help their search rankings.)

If you no longer have control over what links flow out from your site, how could you insure those links are actually helpful to your users? You can’t, end of story.

Desktop Search Software

It’s been a busy year for desktop search with Google, MSN and now AskJeeves releasing their own tools.

Desktop search tools can generally access Microsoft Office files, basic text files, email, Instant message archives, MP3 data, video files and metadata appended to pdf files.

This is a big advance in desktop management. Who hasn’t searched and searched for that file (or piece of information) that we knew we had, but just can’t quite find? Not all of us are content managers that can perfectly catalog the myriad of files we save on our hard drives.

The makers of desktop search indicate that your files are safe; they are not accessing them for any reason other than helping you make your way around your hard drive. But what they don’t mention is the potential security risk desktop search possesses.

Follow this scenario: Let’s say I’m a paranoid person, or have some reason to think a co-worker does not like me. Hey, we’ve all worked with people that seem to hold petty grudges. Or better yet, let’s suppose someone told me they heard another person say something unflattering to my boss.

In the time it takes a co-worker to go to the restroom (or out to lunch) I could sit down at their computer, pull up their desktop search. After typing in my own name, I could quickly see every time that person mentioned me in an email, instant message, in a memo or report. I’d know very quickly whether that person talked about me. Suddenly a confidential email is no longer confidential.

If fact, if I was a savvy user of a search tool, I could quickly find out things about you by using a desktop search function on your PC. The only way around this at present is to password protect your screensaver.

Google Drops the Ball with Desktop Search
Google was the first out of the gate with desktop search. Google maintains the contents on your hard drive are safe and its desktop search feature is not a security risk (outside of the scenario I outlined above).

While I believe that to be true, part of the way they can reinforce that is the method in which they display the search results. Google is now working with Verity to provide text ads in the desktop search results window. Even if my data is safe, this implementation makes me feel like Google is getting a tad to intimate with the documents on my hard drive.

Have they forgotten the lesson of Gmail and all the controversy raised over presenting ads inside emails? Security risk or no, this level of push-technology will make people feel uneasy. I’ll be interested to see if other desktop search tools follow this path. I think Google dropped the ball here.