Erik Jonker<p>For those who want to "farm" the open internet for LLM content, all kind of tools are available, Firecrawl is a good example, partly opensource. Most people are negative about this probably but i think if a website is openly accessible/available for a human we almost can't prevent it to be crawled/scraped and used for AI training.<br><a href="https://docs.firecrawl.dev/introduction" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">docs.firecrawl.dev/introductio</span><span class="invisible">n</span></a><br><a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>AI</span></a> <a href="https://mastodon.social/tags/crawling" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>crawling</span></a> <a href="https://mastodon.social/tags/scraping" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>scraping</span></a> <a href="https://mastodon.social/tags/firecrawl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>firecrawl</span></a> <a href="https://mastodon.social/tags/llm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>llm</span></a></p>