Thu Aug 8

A US judge just called Google the ‘highest quality search engine’. But how do we determine ‘quality’?

Written by: Mark Sanderson, Dean of Research and Professor of Information Retrieval, RMIT University

In his landmark ruling ^[1] against Google earlier this week, United States district judge Amit Mehta said the tech giant has built “the industry’s highest quality search engine”.

Judge Mehta made clear this was partly because Google had an illegal monopoly over the market. Nonetheless, Google was keen to promote the praise it received for its flagship product. Its president of global affairs, Kent Walker, said ^[2]:

This decision recognizes that Google offers the best search engine, but concludes that we shouldn’t be allowed to make it easily available.

But is the Google search engine as good as the company (and Judge Mehta) says it is? And by what metric do we measure whether Google has the “best” search engine in the world?

To answer these thorny questions, it’s important to think about the broader context of the internet – and, in particular, the powerful place of advertising.

Search engines are an expensive business

On September 4 1998, computer scientists Larry Page and Sergey Brin launched Google ^[3]. In the 26 years since, the company has radically transformed our ability to find information.

Its search engine currently processes ^[4] 8.5 billion queries per day – 15% of which ^[5] have never been made before.

People expect the search engine to rapidly deliver accurate answers to every one of those queries. To fulfil this expectation, Google must keep the index up to date by regularly scanning and re-scanning the internet.

This huge task requires thousands of staff – and is therefore very expensive.

One edge Google has over its competitors when it comes to delivering relevant results is its large customer base. They can tune their algorithms based on customer clicks to be more accurate and cover a broader range of queries.

Crucially, however, they wouldn’t have as large a customer base were it not for them having an illegal monopoly over the market ^[6].

Advertising is key

A good way to measure the quality of Google’s search engine is by tracking the presence of advertisements to see how much they affect peoples’ ability to find the information they are looking for.

Advertising has long been a key part of Google.

The company doesn’t appear to keep copies of its search result pages. However with some sleuthing, examples of how ads in search have changed over the years can be found on the Internet Archive’s Wayback Machine ^[7]. The picture that emerges indicates the line between high-quality search results and sponsored content is increasingly blurred.

The first page captured in the year 2000 shows only two adverts at the very top of the page. These are clearly identified by different coloured boxes and the prominently displayed message “sponsored link”.

Screen capture from 2000 of Google search results for the query ‘hotel’. Supplied

The next example, taken from 2013, shows many more ads. But they are clearly labelled in a coloured box and in a separate column on the right.

Screen capture from 2013 of Google search results for the query ‘hotel’. Supplied

In 2016, the column has disappeared and the ads at the top lose their distinctiveness from Google’s main result list, for which Google receives no money.

Screen capture from 2016 of Google search results for the query ‘hotel’. Supplied

Finally, the capture of the Google result today shows sponsored links occupying much screen space before the main results can be seen at the bottom of the page.

Screen capture from 2024 of Google search results for the query ‘hotel’. Supplied

There are other problems impacting the quality of Google’s search engine – as well as its competitors’. In a study published earlier this year, German researchers found that spam and other low quality content is very prevalent ^[8] among the top results for product review searches on Google, Bing and DuckDuckGo.

They concluded:

We find that search engines do intervene and that ranking updates, especially from Google, have a temporary positive effect, though search engines seem to lose the cat-and-mouse game that is [search engine optimisation] spam.

So, what’s the fix?

The impact of forcing Google to give up some of its market share might increase competition, which could push Google to improve the experience search engine users have by reducing the volume and display of advertising.

However, reducing the search engine’s customer base too much might impact on the search engine’s ability to deliver high quality results, because the number of customer clicks that help tune the search engine algorithm would drop.

Apart from breaking up a monopoly, are there other ways to improve search quality?

The most promising approach at the moment is to incorporate artificial intelligence (AI) behind the scenes.

A recent leak ^[9] of how the Google algorithm works found that a generative AI system was being used to judge the quality of web pages.

Microsoft has also applied ^[10] an “AI model to our core Bing search ranking engine, which led to the largest jump in relevance in two decades.”

Hopefully this works. Because with multiple disruptions from the courts and AI innovations including Chatbots, the sedate changes in the quality of search results are about to accelerate.