3.3 Search
“Google’s mission
is to organize the world’s information and make
it universally accessible and useful.”
—Google1
In Web 2.0, the saying “content is king” remains
a prevailing theme. With seemingly endless content available online,
the findability of content
becomes key. Search
engines are the primary tools people use to find information
on the web. Today, you perform searches with keywords, but the future
of web search will use natural language (see, for example, Powerset.com).
Currently, when you enter a keyword or phrase, the search engine finds
matching web pages and shows you a search engine
results page (SERP) with recommended web pages listed and sorted
by relevance. People-assisted search engines have also emerged, such
as Mahalo, which pays people to
develop search results.2 The popularity of vertical
search engines—ones that focus on a specific
topic or industry—is on the rise, though traffic to these search
engines is still far behind the major (more generalized) search engines.
Traffic to the major search engines is growing rapidly—according
to a recent comScore (a web analytics company)
report, Americans conducted 8 billion search queries in June 2007, up
26% from the previous year. In the same report, the comScore analysis
of U.S. market share across the most popular search engines reported
Google at the top with 49.5% of the U.S. search market, followed byYahoo! with 25.1%, Microsoft
with 13.2%, Ask with 5.0% and Time Warner Network with 4.2%.3
John Battelle’s book aaa.he
Search: How Google and Its Rivals Rewrote the Rules of Business and
Transformed Our Culture provides an extensive history of search
engines and presents strong arguments for the importance of search in
almost every aspect of our personal and business lives. John Battelle’sSearchblog discusses search and technology
issues (http://battellemedia.com).
Attention Economy
“Telecommunications bandwidth
is not a problem, but human bandwidth is.”
—Thomas Davenport and John Beck, The
Attention Economy4
The abundant amounts of information being produced and
people’s limited free time has led to an attention
economy. More content is available than users can sort through
on their own, especially given the demands on their time, such as responsibilities
to children, parents, friends, employers, etc. The
Attention Economy, by Thomas Davenport and John Beck, begins with
the familiar story of a man whose attention is constantly demanded by
work and family. The authors explain that the constant flow of information
in today’s world causes attention to continually be diverted.
Though it used to be difficult to obtain diverse content,
there are now seemingly endless options competing for an audience’s
attention. As a result, search engines have gained popularity
by helping users quickly find and filter the information they want.5
Google Search
Google
is the leading search and online advertising company, founded by Larry Page and Sergey Brin while they were Ph.D.
students at Stanford University. Google
is so popular that its name has been added to the Oxford English Dictionary—the
verb “Google” means to find something on the Internet using
the Google search engine. (“google” with a lowercase “g”
is a cricket term, whereas “googol” or 10100
is the mathematical term Google was named after.)6
Google’s success in search is largely based on itsPageRank™
algorithm (patented by Stanford University and Larry Page) and
its unique infrastructure of servers that uses linked PCs to achieve
faster responses and increased scalability at lower costs.7 Estimates on the number of Google servers range from hundreds
of thousands to over one million.8 The
PageRank algorithm considers the number of links into a web
page and the quality of the linking sites (among other factors) to determine
the importance of the page. Each inbound link is a vote saying that
site is valuable to someone else; however, votes are given different
weights depending on the “voter” site’s own value.
So, two pages could have the same PageRank even if one has numerous
links in from other pages and the other has fewer links in but from
pages with higher PageRank. Google search also considers all of the
content on the page, its fonts, its headers and the content of neighboring
pages.9 Sites with the highest PageRank will appear at the top of the
search results.
In addition to its regular search engine, Google offers
specialty search engines for images, news, videos, blogs and more. UsingGoogle
web services, you can build Google
Maps and other Google services into your applications (seeSection 3.13, Web Services, Mashups, Widgets
and Gadgets).
AdWords, Google’s
pay-per-click (PPC) contextual advertising program (launched in 2000),
is the company’s main source of revenue. AdWords ads appear next
to search results on the Google site (and are related to the search
query). Advertisers write their own ads, which are unobtrusive and uniform
in appearance—each ad consists of a headline, limited text and
a URL. Advertisers bid on search keywords related to their ads and pay
based on the number of users who click on the ads.
AdSense is Google’s advertising
program for publishers (sites like http://www.deitel.com
that offer content), inspired by Susan Wojcicki, the vice president
of product management. (In 1998, Wojcicki rented a spare room in her
house to Larry Page and Sergey Brin where they founded Google.)10 AdSense is a fundamental and popular form of website monetization,
particularly for Web 2.0 startup companies. Google text ads (as well
as banner and rich-media ads) are placed on participating sites with
related content. Click-through rates on contextual ads are often higher
than on non-contextual ads because the ads reach people expressing interest
in a related topic. As a result, contextual pay-per-click ads generally
pay a higher eCPM (effective cost per thousand impressions).
Yahoo!
Yahoo!
was started in 1994 by Jerry Yang and David Filo (also Stanford Ph.D. students) as a
web directory rather than a search engine. The original site, “Jerry
and David’s Guide to the World Wide Web,” consisted of their
favorite websites manually added to a categorized directory.11 As the web grew, maintaining the directory structure became increasingly
difficult, and a search capability was created for better access to
the data. Focusing more on search, Yahoo! also expanded into other areas,
becoming a popular provider of e-mail, user groups and more. In 2003, Yahoo!
acquired Overture (now Yahoo! Search Marketing),
which was the first search engine to offer sponsored search results
successfully.12
MSN
MSN search was created in 1998,
a year after Google was launched.13 Over the past few years, Microsoft has made search engine technology
development a top priority.14 Microsoft search query volume and its search market share grew
rapidly in June 2007; analysis companies comScore and Compete attribute
this boost largely to MSN’s Live Search club, a program introduced
in May 2007 to reward users of Live Search.15, 16 MSN’s
Live Search includes a new search engine, index and crawler.17 It allows you to search the web, performing specialized searches
(news, images, or local listings) or MSN content searches.18 Another approach that Microsoft is taking to increase its search
market share is buying vertical search sites such as MedStory, a health
search engine. 19 Microsoft is also looking to gain market share in the contextual
advertising market through Microsoft adCenter (similar to
Google AdWords and Yahoo! Search Marketing).
Ask
Ask
(formally known as AskJeeves.com) is owned by InterActiveCorp (IAC), which
also owns Ticketmaster®, Match.com®, LendingTree.com®, RealEstate.com®
and many other Internet properties. In June 2007, Ask launched a new
search site, which includes a new design with a simple homepage default,
customizable backgrounds, new video search (powered by Blinkx) and the ability to view video
previews and listen to music clips. The search results are based on
the searcher’s location—Ask will report relevant local businesses
and events. Searching for movies, for example, will show local show
times.
Vertical Search
Vertical
search engines are specialists (focusing on
specific topics) in comparison to generalists (e.g., Google and Yahoo!).20 Vertical search engines enable you to search for resources in
a specific area, with the goal of providing you with a smaller number
of more relevant results. Popular vertical search engines include travel
sites (such as Kayak or Expedia), real-estate sites (such asZillow or Trulia), job search sites (such asIndeed or Monster)
and shopping search engines (such as Shopzilla
and MySimon).
Location-Based Search
Location-based
search (offered by most major search engines as well as some
smaller specialized ones) uses geographic information about the searcher
to provide more relevant search results. For example, search engines
can ask the user for a ZIP code or estimate the user’s general
location based on IP address. The engine can then use this information
to give higher priority to search results physically located near the
user. This is particularly useful when searching for businesses such
as restaurants or car services. (See Section 3.14
for more information on location-based services.)
Creating Customized
Search Engines
Rollyo—a build-your-own customized
search engine website—allows you to explore, create and personalize
search engines (“searchrolls”) created by others. This helps
you narrow your search to sites you already trust.21 Other custom search sites include
Gigablast and Google
Custom Search Engine.
Search Engine Optimization (SEO)
Search
Engine Optimization (SEO) is the process of designing and
tuning your website to maximize your findability and improve your rankings
in organic (non-paid) search engine results. To maximize traffic, you
need to take into consideration how search engines work when you design
your website. There are two ways of employing SEO. The first, white
hat SEO, refers to methods that are approved by search engines,
do not attempt to deceive the search engines, and produce quality, long-term
results. Top white hat techniques for SEO include: offering quality
content, using proper metadata and effective keywords, and having inbound
links from relevant high-quality pages.22 Black hat methods are
used to deceive search engines. Although they may result in temporary
improvement in search engine results, these tactics could get your site
banned by the search engines. A “Googlebomb” (or link bomb)
is an example of a black hat method—it attempts to trick the Google
algorithm into promoting a certain page (generally for humorous reasons).23
Link Building
Link
building is the process of increasing search engine rankings
and traffic by generating inbound links to a particular website. Search
engine algorithms regard each link as a vote for the destination website’s
content, so sites with the greatest link
popularity (or number of high-quality inbound links) appear
highest on search engine result pages (SERPs). The three most practiced
methods of building links include reciprocal linking, link baiting and
natural linking. Reciprocal
linking is an exchange in which two related websites link
to each other, increasing the link popularity of both sites and adding
value for site users.Link baiting involves creating
attention-grabbing web content specifically for viral (exponentially
increasing) exposure through social media and social bookmarking websites.Natural linking is the process
of building one-way inbound links by optimizing website content and
user experience without the explicit solicitation of a backlink. Search
algorithms are continuously updated to prevent black hat SEOs from deceiving
search engines with automated linking software and links from directories
or other low-quality websites. One-way links from websites with strong,
related pages are given greater weight than reciprocal links, links
from sites with unrelated content or links from sites with low PageRank.
Search Engine Marketing (SEM)
Search Engine Marketing (SEM)
is the method of promoting your website to increase traffic and search
results by raising the site’s visibility on search engine results
pages. Danny Sullivan (founder of Search
Engine Watch and, more recently, Search Engine
Land) introduced the term “Search Engine Marketing”
in 2001 to include SEO, managing paid listings, developing online marketing
strategies and submitting sites to directories.24 SEO
is the most popular form of search engine marketing, which continues
to take away business from other marketing channels (especially offline
sources). According to the Search Engine Marketing Professional
Organization’s annual State of Search Engine Marketing survey,
North American advertisers spent $9.4 billion on search engine marketing
in 2006, a 62% increase over 2005 spending.25
Search Engine Watch and Search Engine Land
Search
Engine Watch is a search engine
marketing resource site. It includes articles, tutorials, conferences
and more. The site, launched in 1997 by Danny Sullivan, was inspired
by his 1996 release of “A Webmaster’s Guide To Search Engines.”
Search Engine Watch incorporates Web 2.0 features (blogging and forums
in addition to expert columnist articles). Other Search Engine Watch
departments include search engine submission tips, web searching tips,
popular search engines and search engine resources (numerous topics
related to search engines). Danny Sullivan served as Search Engine Watch’s
editor-in-chief until November 2006, when he left the site and became
the editor-in-chief for Search Engine
Land. The site provides news and information
on the major search engines—Google, Yahoo!, and Microsoft—as
well as search engine marketing and searching issues. The site also
informs users of upcoming related conferences and webcasts.
Search Engine Strategies Conferences
Search
Engine Strategies is a global conference series
focused on search engine advertising (including current SEO and SEM
issues). Search Engine Strategies (hosted by Search
Engine Watch) offers event information given by the top experts
in the field as well as representatives from search engine companies.26 Because traffic and advertising are so important to most Web 2.0
businesses, understanding the search process and making sure your site
is easily found is vital.
Discovery
Rather than the traditional use of search engines (searching
with a topic in mind), discovery refers
to finding new content you would not have otherwise sought out. For
example, Yahoo!’s original directory
design allowed users to browse categories, and discover new interesting
sites. StumbleUpon, a social bookmarking
site, addresses discovery with its recommendation system that helps
you discover and share websites based on your interests. Content networks also direct
users to web content they would not necessarily have looked for otherwise.