For the first few decades of the web, the tacit agreement was that web crawlers could take data from sites in exchange for traffic back. But now, are AI crawlers (incl. Google) taking too much and offering too little in return?
This site is made possible by member support. ๐
Big thanks to Arcustech for hosting the site and offering amazing tech support.
When you buy through links on kottke.org, I may earn an affiliate commission. Thanks for supporting the site!
kottke.org. home of fine hypertext products since 1998.
Beloved by 86.47% of the web.
For the first few decades of the web, the tacit agreement was that web crawlers could take data from sites in exchange for traffic back. But now, are AI crawlers (incl. Google) taking too much and offering too little in return?
Socials & More