pagerank

It's a pity the National Security Agency can't talk about its computational challenges, because it's leaving a lot of the boasting rights to Google.

In a blog posting on Friday the company shared some detail about the challenges of one aspect of its search operation, the Web indexing and processing that must take place before the results are delivered to users. The short version: Google has no choice but to think big.

First comes surfing. "We start at a set of well-connected initial pages and follow each of their links to new pages. Then we follow … Read more

Though a distant third place to Google, Microsoft thinks it can teach its rival a thing or two about searching the Internet.

A big part of Google's rise to search engine leadership was an algorithm called PageRank that assesses a specific page's importance by how many other Web pages link to it and by the importance of those linking pages. Microsoft researchers and academic collaborators, though, detailed an idea this week it calls BrowseRank that seeks to bring more of a human touch to that assessment.

Essentially, the researchers tested out a system that replaces PageRanks' link graph--a … Read more

Q&A Search has become central to the functioning of the Internet, but Udi Manber isn't the kind of person who takes that for granted.

"I don't have to tell anybody around here that search is important. That's a very nice luxury to have," said Manber, the Google vice president in charge of search quality.

Search quality may seem like an unassuming element of Google's operations, but in fact it's at the core. Manber oversees the company's search algorithm--all the different inputs Google weighs to judge which Web sites to rank highest in search results.

Manber's work has been highly secret, partly because search is central to Google's competitive advantage and partly because Google doesn't want people gaming the system to get artificially prominent results. But the company has begun sharing a smidgen, including an opening blog post by Manber in May. I talked to him at Google headquarters recently.

How mature is search today on the Internet? Are we 5 percent of the way done with the problem? Ninety percent? My best analogy is that a 15-year-old thinks he's very mature. A 19-year-old thinks he's extremely mature. Every few years you learn that you were not mature before. Search on the Web is about 15 years old, and obviously we were much more mature than we were 5 years ago and 10 years ago and 15 years ago. One way to put it is that it's science fiction every 5 years. What's possible today to me was science fiction 5, or definitely 10 years ago. What was (ordinary) 10 years ago was science fiction 15 years ago. The development is really pretty amazing. It surprised even me. I expect a certain level of progress, and we're actually surpassing it.

You were at the University of Arizona, then Yahoo and Amazon, then A9, then you moved to Google in 2006. Is there anything you've learned from looking at it from different perspectives, or have you been just tackling the same thing with different phone numbers on your business card? It's the same problem, and I've looked at it from many different angles. It's bigger here, and it's better here. We have a team that's beyond any other team I've ever been with. We put more resources into it. I don't have to tell anybody around here that search is important, and that's a very nice luxury to have.

I remember the old days of AltaVista and HotBot and WebCrawler some of these other search engines--days when search was really very primitive. I remember starting those things. They looked very sophisticated and mature at the time, which is my point about the 15-year-old.

It's clearly become a lot more usable. But even 10 years ago, everybody hadn't been trained to think the way we get information is we go to a search box and type something in. Now that seems abundantly obvious. What 10 years from now is going to look stunningly obvious as having a search box is today? It was clear to some people. I don't want to brag too much, but it was clear to me. That's why I moved to search in the early 1990s, because everybody was talking about the information revolution. It was very clear that to have an information revolution, it's not enough to store the information and move it around, you have to find it. I know a lot of people at the time who were talking in those terms--that's going to be the revolution. The ability to find things among huge amounts of information is the key factor. So while nowadays it's completely obvious, even 6 or 7 years ago it was not obvious. I think the reason Google is so successful now is because it was obvious to (co-founders) Larry (Page) and Sergey (Brin) 10 years ago, they put in all the effort, and they're still doing it now.

Don't take that for granted. It was not that well understood, but it was understood by some people. When I started working on search when I was in academia and I said I'm working on search, they looked at me and said, "What do you mean you're working on search? Did you lose something?" In the early 1990s, even, very few people worked on search, because search was done by professionals in various limited domains. There was legal search, there was medical search, there was chemical search, and some limited news search. And it was done by a searcher--professional people. You tell them, "This is what I want to find," and they find it for you. I went to trade conferences with searchers. The idea that people will do the search themselves--that it'll democratize the whole thing and you don't have to go to a professional--that's the revolution.

I think that'll advance much more because you'll do more searches. There are a lot of things you don't search for now, because you don't expect Google will know or that the search engine will find out. We are finding that user expectations grow. The kind of searches people do now are more complicated than the kinds they were doing five years ago. People expect a lot more from us.

Ten years ago, if you actually found an answer to some specific question, it was, "Hey, look at this, it's so cool!" It was an event. Nowadays if you don't find exactly what you want in the first or second result, something is wrong. That's nice. The expectation is that we'll do it.

… Read more

Google has concluded it's been a little too secretive about the inner workings of its search engine.

The company has deliberately stayed mum about the algorithm that decides what to put at the top of the search results list, in part because the company doesn't want competitors copying it and in part because it doesn't want Web sites gaming the system, said Udi Manber, the vice president of engineering in charge of search quality, in a blog post Wednesday. Now, though, it plans to share a little more.

"Being completely secretive isn't ideal, and this … Read more

The blogosphere and Twitter have been abuzz with talk about this article by Shari Thurow, published Thursday on Search Engine Land. The article warns of supposed dangers against the SEO tactic of "PageRank sculpting." Readers are coming away feeling reticent to employ the tactic, fearing retribution from the engines in the form of penalties. The article paints PageRank sculpting as poor usability and black hat. I can't be any more adamant about this: neither is the case.

No disrespect intended to the article's author, but this article is classic FUD (Fear, Uncertainty, and Doubt). "Nobody … Read more

In the process of reviewing a client's Flickr account with my colleague and fellow Searchlight blogger Brian Brown, we noticed that Flickr has recently added nofollow tags to links placed within its Web site. Flickr has been one of the few social-media entities to continue to offer "link juice" from links placed with user-generated content (in this case photo descriptions), making it a viable entity for improving inbound links to a given site.

While it's understandable that Flickr implemented nofollow tags for the exact same reason other social-media sites have--misuse and spamming--it nonetheless marks another step … Read more

Link building is one of those challenging subjects that carries a lot of technical undertones. It's a subject that often requires some explanation, depending on the audience, as to why it is so important to begin with.

The importance of building links to a site is something that SEO (search engine optimization) practitioners understand all too well. It's not just a quantity thing, but a qualitative measure. Links aren't just a conduit for traffic; they serve as an important signal to search engines. Of course, it is the significance and understanding of this signaling that often steers … Read more

With Google's recent crackdown on Web sites the sell PageRank--which really means selling links--a new era has begun for backlink building. In Google's eyes, links coming into a given Web site from external, quality sites increases that site's PageRank, and therefore its standing in the search engine's eyes. Until recently, there were many sites that had quality in Google's eyes (in other words, they had great PageRank) and also sold links. Anyone could get a piece of that good PR for a price.

Google is now actively lowering the PageRank of sites that deal in … Read more

After what felt like a long hibernation period, Google has recently made at least a partial PageRank update of the visible or public PageRank, often referred to as Toolbar PageRank. And boy was it noticed--not so much with the update but with the PageRank drops targeted at sites that were buying and/or selling text links.

A lot has been said already by many who have a much deeper understanding and forecast this event. What I find particularly interesting is how PageRank has become a form of currency, yet who can say what precisely that value is? Of course, I … Read more