What the Open Web Looked Like Before the Algorithms

A field guide to the open web in the era before the algorithm, with what the platforms were doing, what the specific things were that the platforms were doing, and the part about the things the algorithms are not going…

A single brass hourglass with sand on dark wood, dim warm amber side light, deep navy shadows, no people, no logos.

The open web before the algorithms was a memory of a different kind of internet, with the platforms behaving like platforms, the search engines behaving like search engines, and the personal sites behaving like personal sites. The web where the user was the customer, the web where the publisher was the producer, and the web where the platform was the intermediary. The web was small, the web was messy, the web was full of amateur work, and the web was, in retrospect, the best era of the web for the user who was willing to do the work to find the good stuff.

What the platforms were

The platforms from 1998 to 2010 were mostly tools. The platforms did not pick the content the user saw, the platforms did not decide which post was the most important, and the platforms did not train a model on the user behaviour to predict what the user wanted to see next. The platforms were the place the user went to find the content, the platforms were the place the user went to publish the content, and the platforms were the place the user went to talk to the other users.

The platforms had a chronological feed (or no feed at all), the platforms had a search function, and the platforms had a list of links. The platforms were not the curators, the platforms were not the editors, and the platforms were not the gatekeepers.

The tools the platforms gave the publisher were also simpler. The blogger template, the Movable Type install, the WordPress.com signup. The publisher wrote the post, the publisher hit publish, and the publisher was done. The publisher did not have to think about the algorithm, the publisher did not have to think about the engagement metrics, and the publisher did not have to think about the optimal post time. The publisher wrote, the publisher published, and the publisher moved on.

What the search engines were

Google in 2005 was a remarkably good product. The PageRank algorithm was a real signal. The search results were the most relevant pages for the query, not the most relevant pages for the query that the algorithm had been trained to optimise for. The user searched for the thing the user wanted, and the user got the thing the user wanted.

The search engines did not have the personalisation layer the modern search engines have. The search engines did not track the user across the web, the search engines did not build the user profile for the advertising auction, and the search engines did not adjust the results based on the user history. The search engines were the same for every user, and the search engines being the same for every user is what made the search engines useful.

The SEO industry existed, but the SEO industry was a different shape. The keyword stuffing, the link buying, the cloaking. The practices that worked in 2005 are the practices that got the site penalised in 2010. The search engines were winning the arms race, the search engines were giving the user what the user wanted, and the search engines were not yet corrupted by the advertising business model that was going to corrupt them in the 2010s.

What the personal sites were

The personal sites were everywhere. The blogger with the food blog, the photographer with the portfolio, the programmer with the technical writing, the hobbyist with the links page. The personal sites were the part of the web the corporate web never managed to displace, and the personal sites were the part of the web that gave the user the actual signal the user was looking for.

The personal sites were not optimised for engagement. The personal sites were not designed to keep the user on the page. The personal sites were written by the person the user wanted to read, and the personal sites were updated when the person had something to say. The personal sites had the RSS feed, the personal sites had the comment section, and the personal sites had the link list that pointed to the other personal sites the user was going to want to read.

The personal sites were small. The personal sites were not part of the platform. The personal sites were not at the mercy of the algorithm change. The personal sites were on the domain the personal site owner controlled, and the personal sites being on the domain the personal site owner controlled is what made the personal sites durable.

What changed

The platforms became the publishers. The platforms went from the place the user went to find the content to the place the user went to consume the content. The platform feeds replaced the blog subscriptions, the platform recommendations replaced the friend links, and the platform metrics replaced the personal satisfaction. The publisher who used to write for the audience the publisher had built became the publisher who writes for the algorithm the platform has built.

The personal sites did not disappear. The personal sites are still there, and the personal sites are still being read by the people who care enough to look. The personal sites are just harder to find, and the personal sites being harder to find counts as the cost the user pays for the convenience the platforms offer.

The user can go back. The user can subscribe to the RSS feeds, the user can follow the personal sites, and the user can curate the personal reading list the platforms used to curate for the user. The user who is willing to do the work can find the good stuff. The user who is not willing to do the work is going to be served the algorithmic feed, and the algorithmic feed is going to be the feed the platform has decided the user is going to engage with.

The web is not dead. The web is more open than it has ever been, in the sense that the open source tools let anyone publish to the web without the platform’s permission. The web is more closed than it has ever been, in the sense that the open source publishing tools do not help the user find the things the user wants to read. The user can still publish, the user can still read, and the user can still find the good stuff. The user just has to do the work the platforms used to do for the user, and the user has to do the work the user is no longer used to doing.

What the Open Web Looked Like Before the Algorithm - inline
Key points from What the Open Web Looked Like Before the Algorithm

The bottom line

The patterns the post covers have been showing up in production for long enough that the patterns have names, the failures, the mitigations, the gaps. The work the security team and the engineering team and the operations team are quietly doing today sits as the work that decides whether the practice the post names sits as a tool the team uses or a liability the team is paying for.

Sources & Further Reading

All claims in this article are sourced from primary documentation, vendor advisories, and reputable security researchers.

Spotted an error? Email the editor. Corrections are issued with a visible correction note.

Editorial standards. Every article on humanrequired.org is reviewed by a human editor before publication. AI may assist with drafting or research; final editorial control is human. Read the full standards.

Continue reading