﷽
..


Last Tended:

❔ will Google's self-importance be its downfall?

In my recent explorations of the IndieWeb, I came across a search engine called BlogSurf. Independent search engines are everywhere, from Marginalia to the 28-year-old WebCrawler, each with their own algorithms. BlogSurf uses an algorithm called MarketRank, which they describe as follows:

PageRank [6] was built on the analogy of the research paper. A research paper with more citations is probably better.

MarketRank is built on the analogy of the market. An object with a higher market value is probably better.

In our case, the markets we refer to are online communities such as Reddit, Hacker News, and Twitter. Each of these communities has an upvote mechanism, which can be interpreted as an indication that the user believes the website is undervalued. They are willing to bid up the price by spending their upvote on it.

And so, the oversimplified description of MarketRank is “just add up all the upvotes”. We have some more work to do to make the values accurate, but this is the general idea.

MarketRank’s approach hits on the trust issue many of us have with Google these days. As Tracy Durnell puts it:

Google Search sucks – it’s been conquered by SEO sp*m sites so it’s untrustworthy and unuseful for anything but essentially directory lookups. Social media and the parasocial relationships we have there are a more trusted source of information – it’s coming from an actual human who is providing a testimonial/ advice about what has or hasn’t worked for them.

We trust humans and personal recommendations. That’s what made PageRank so good to begin with. In the web 1.0 era, all links were generated by humans. Which meant PageRank, which looked at links across the web, was able to best qualify sites. But today’s automation and SEO optimization has rendered that useless. By using forums and upvotes/likes as a proxy for human votes, MarketRank is better poised to surface what PageRank would once prefer.

The downside, as DKB calls out in the MarketRank post itself, is that humans only crawl a small percentage of the web. Which means that 99% of websites will have a MarketRank score of 0.

But it’s still notable that there is competition for Google. And that competition relies on admitting that Google doesn’t always know best. I’m trying to imagine Google prioritizing Reddit and Twitter shares so heavily. Yes, I could see them saying that “forum links correlate with high quality”. But the drastic change required and the emotional impact of saying “Reddit knows best”? I can’t see it. They instead seem to be focused on surfacing their subpar results in more “fun” places1. Will their self-importance give competitors room to innovate on web search algorithms for the first time in 20 years?

What innovations does web search need? Let me know what you think.

(Update: In Sept 2022, Google did admit that sometimes Reddit knows best with a new discussions module that prompted headlines like “Google will help you find better results without tagging ‘Reddit’ onto every search”)

Info:

commonplace book

A place to collect relevant smatterings from around the web:

how too many results can be a bad thing

From Mastodon:

I feel that the measure of how useful a search engine is whether giving it more information gives you fewer or more results.

Generally, you want fewer results the more information you give it, as you are narrowing your search by giving more information.

But a bad search engine will just try to find things that matches SOME of the information you give, and will thus return MORE results the more information you give it.

And this causes an incredibly frustrating situation where you have to try and game the search engine by figuring out what the smallest set of information you can give it is that is still likely to return you what you want.

The original Google won over all the existing search engines at the time it was introduced because it stuck VERY strongly to the principle of giving you less results as you give it more input.

But these days, it very much feels like Google has completely given up on this principle, and will just try to drown you in results that vaguely match some of your input query, maybe, if you squint your eyes and tilt your head.

It’s really getting to the point where it is nearly unusable for locating very specific information. I feel like information I used to be able to rely on being able to find with Google is now lost forever, as it is no longer locatable in the sea of noise Google returns.

From a Trapital podcast episode:

[00:15:43] Dan Runcie: Yeah. The next one is TikTok Search. A lot of us had seen the viral tweet that someone had. I don’t Google I TikTok. And a lot of that spoke to how a lot of folks in Gen Z are looking for information and I get it, I’ve even done it myself, my wife and I were recently searching to buy a new mattress. And you know what? I didn’t want to go through a Google Search and just read some sponsored content about a mattress. I wanted to see a video of someone unboxing this thing to see what it looks like. 

[00:16:10] Denisha Kuhlor: Exactly. 

[00:16:10] Dan Runcie: And TikTok was the quickest place to do that, even quicker than YouTube. YouTube’s going to show me a mix of explainer videos and then also concept from the company. I just wanted to see some random person be like, oh, hey, here’s what I think about this bed. And here’s what I think about that bed. It was quick, it was easy. So I do think that that works, but I think there’s a few caution flags with it. A, I still think that even though TikTok was able to offer that, there’s still deeper search functionality that went into how Google got to be as good as it is, even going back thinking about 20 years ago about like why Google succeeded where Lycos and AltaVista and all those other go.com, .com era search engines didn’t work. So I don’t know if TikTok has all of that baked in to really go beyond just, you know, people like me looking for random purchases that they want to look through here or there or just want to look up a certain topic. And I also think the other bigger, more important pieces, the misinformation, and just being able to correct for that because that’s already been an issue on TikTok. And I think that could potentially continue if there isn’t some way to relegate what’s happening in search. So, high likelihood success, but still some trepidation.

(Related: a twitter thread showing how tiktok is used for search)

in passing, in a piece about LLMs

This prediction ability seems like a simple party trick but it’s what makes models so good at answering our random questions in the way Google used to be.

Potential Updates

  • 2023: Do LLMs change anything about this?
    • It’s interesting to consider the idea that whoever can sift through the noise best is the winner. Do LLMs give Google an advantage or disadvantage in that?
    • The idea of what you’re selling being information vs a good/service. If you’re selling a good or service, it’s good for your ideas to get rolled into an LLM: in order to be useful, Google must link the user back to you. If you’re selling information, Google doesn’t need to link back to you to be useful.
    • The idea that LLMs will settle with publishers and pay to train their models on publisher paywalled data. All of a sudden, it’s a private web. The only way to access the NYT, WSJ and FT is by paying $$$/month (subscribe to each) or subscribe to 1 LLM trained on them all. You can’t fact-check anything the LLM tells you.
  • on Google getting worse

  1. Thanks to Tracy Durnell for surfacing this story in Trusted Information Sources ↩

Don't want to do that? @ me on twitter or mastodon

Every post on this blog is a work in progress. Phrasing may be less than ideal, ideas may not yet be fully thought through. Thank you for watching me grow.

Updates

  • : Mentioned Google's new discussions feature in an update after the conclusion. Added additional section to collect relevant quotes from around the web.
  • : Added LLM and Potential Updates sections to commonplace book