Recent searches

No recent searches

Search options

Only available when logged in.
shakedown.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A community for live music fans with roots in the jam scene. Shakedown Social is run by a team of volunteers (led by @clifff and @sethadam1) and funded by donations.

Administered by:

Server stats:

269
active users

shakedown.social: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.2

#scraper

0 posts0 participants0 posts today
@francks<p>Our small team vs millions of bots</p><p><a href="https://www.fsf.org/blogs/sysadmin/our-small-team-vs-millions-of-bots" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">fsf.org/blogs/sysadmin/our-sma</span><span class="invisible">ll-team-vs-millions-of-bots</span></a></p><p><a href="https://mstdn.fr/tags/fsf" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>fsf</span></a> <a href="https://mstdn.fr/tags/freesoftware" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>freesoftware</span></a> <a href="https://mstdn.fr/tags/ddos" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ddos</span></a> <a href="https://mstdn.fr/tags/javascrip" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>javascrip</span></a> <a href="https://mstdn.fr/tags/anubis" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>anubis</span></a> <a href="https://mstdn.fr/tags/botnet" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>botnet</span></a> <a href="https://mstdn.fr/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llm</span></a> <a href="https://mstdn.fr/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://mstdn.fr/tags/crawler" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>crawler</span></a> <a href="https://mstdn.fr/tags/proprietarysoftware" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>proprietarysoftware</span></a> <a href="https://mstdn.fr/tags/malware" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>malware</span></a></p>
Kevin Karhan :verified:<p><span class="h-card" translate="no"><a href="https://mastodon.pnpde.social/@spielleitung" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>spielleitung</span></a></span> also ich mach's mir professionell einfacher und blockiere einfach alle bekannten <a href="https://infosec.space/tags/Scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Scraper</span></a>...</p><ul><li>Wenn ihr weitere Ranges an <a href="https://infosec.space/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://infosec.space/tags/Crawler" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Crawler</span></a>|n kennt: Immer her damit! </li></ul><p><a href="https://github.com/greyhat-academy/lists.d/blob/main/scrapers.ipv4.block.list.tsv" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/greyhat-academy/lis</span><span class="invisible">ts.d/blob/main/scrapers.ipv4.block.list.tsv</span></a></p>
Paolo Amoroso<p>A Mastodon bot account at mastodon.cloud scans the fediverse, scrapes selected web pages shared there, rewrites them with AI, posts them to its own site, and shares on Mastodon as tech news the rewritten AI slop. The bot scraped a post of mine (including the attached image) within minutes of my federated blog publishing it.</p><p>Is it worth flagging the bot and reporting it to its instance? Are the mods likely to take action?</p><p><a href="https://fosstodon.org/tags/mastodon" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>mastodon</span></a> <a href="https://fosstodon.org/tags/moderation" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>moderation</span></a> <a href="https://fosstodon.org/tags/ai" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ai</span></a> <a href="https://fosstodon.org/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a></p>
Tom :damnified:<p>Anubis: self hostable scraper defense software | Anubis</p><p><a href="https://anubis.techaro.lol/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">anubis.techaro.lol/</span><span class="invisible"></span></a></p><p><a href="https://metalhead.club/tags/ai" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ai</span></a> <a href="https://metalhead.club/tags/bots" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bots</span></a> <a href="https://metalhead.club/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a></p>
zeyus :gopnik_implant_b:‎<p>Hey does anyone know if there's still a working zip bomb style exploit that can be deployed on a static site/JS (or as a asset/resource)? Specifically to target web scrapers and AI bullshit? The second any server goes online now it's immediately bombarded by stupid numbers of requests.</p><p><a href="https://corteximplant.com/tags/hacking" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>hacking</span></a> <a href="https://corteximplant.com/tags/aislop" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>aislop</span></a> <a href="https://corteximplant.com/tags/crawlers" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>crawlers</span></a> <a href="https://corteximplant.com/tags/webscraping" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>webscraping</span></a> <a href="https://corteximplant.com/tags/webcrawler" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>webcrawler</span></a> <a href="https://corteximplant.com/tags/robots" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>robots</span></a> <a href="https://corteximplant.com/tags/zipbomb" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>zipbomb</span></a> <a href="https://corteximplant.com/tags/zipbombing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>zipbombing</span></a> <a href="https://corteximplant.com/tags/exploit" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>exploit</span></a> <a href="https://corteximplant.com/tags/robotstxt" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>robotstxt</span></a> <a href="https://corteximplant.com/tags/server" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>server</span></a> <a href="https://corteximplant.com/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a></p>
Kevin Karhan :verified:<p><span class="h-card" translate="no"><a href="https://climatejustice.social/@stefanmuelller" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>stefanmuelller</span></a></span> die <a href="https://www.robotstxt.org/" rel="nofollow noopener" target="_blank"><code>robots.txt</code></a> ist eine <em>Bitte</em>, KEIN <em>gesetzlich verpflichtender Opt-Out</em>!</p><ul><li>Wenn dir <a href="https://infosec.space/tags/Scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Scraper</span></a> und anderer shice bekannt ist sag' bescheid und ich pack die <a href="https://github.com/greyhat-academy/lists.d/blob/main/scrapers.ipv4.block.list.tsv" rel="nofollow noopener" target="_blank">öffentliche Blocklist</a> die ich maintaine...</li></ul>
Kevin Karhan :verified:<p>The whole <a href="https://infosec.space/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://infosec.space/tags/Enshittification" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Enshittification</span></a> <a href="https://infosec.space/tags/shitshow" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>shitshow</span></a> is so bad, at work entire address blocks as big as /12 have to be blocklisted because they basically <a href="https://infosec.space/tags/DDoS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DDoS</span></a> clients unless we'd want to bankrupt customers for <a href="https://infosec.space/tags/bezos" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bezos</span></a>' <a href="https://infosec.space/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://infosec.space/tags/bots" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bots</span></a>! </p><ul><li>If it was my decision the entire <a href="https://infosec.space/tags/aws" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>aws</span></a> <a href="https://infosec.space/tags/ASN16509" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ASN16509</span></a> would've been blocked!</li></ul><p><a href="https://infosec.space/tags/Sysadmin" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Sysadmin</span></a> <a href="https://infosec.space/tags/SysadminProblems" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SysadminProblems</span></a> <a href="https://infosec.space/tags/Amazon" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Amazon</span></a> <a href="https://infosec.space/tags/ITsec" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ITsec</span></a> <a href="https://infosec.space/tags/InfoSec" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>InfoSec</span></a> <a href="https://infosec.space/tags/ComSec" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ComSec</span></a> <a href="https://infosec.space/tags/OpSec" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpSec</span></a> <a href="https://infosec.space/tags/Tech" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Tech</span></a> <a href="https://infosec.space/tags/GAFAMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GAFAMs</span></a> <a href="https://infosec.space/tags/AllGafamsAreEvil" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AllGafamsAreEvil</span></a> <a href="https://infosec.space/tags/AllGafamsAreBad" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AllGafamsAreBad</span></a></p>
PaulaToThePeople 😷FediBlock newsmast
Kevin Karhan :verified:<p><span class="h-card" translate="no"><a href="https://mastodon.social/@khobochka" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>khobochka</span></a></span> guess why I <a href="https://github.com/greyhat-academy/lists.d/blob/main/scrapers.ipv4.block.list.tsv" rel="nofollow noopener" target="_blank">maintain</a> a <a href="https://infosec.space/tags/Scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Scraper</span></a> <a href="https://infosec.space/tags/blocklist" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>blocklist</span></a>?</p><ul><li>In fact I know <em>multiple</em> people and organizations that decide to basically redirect <a href="https://infosec.space/tags/ValueRemoving" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ValueRemoving</span></a> <a href="https://infosec.space/tags/Scrapers" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Scrapers</span></a> like <a href="https://infosec.space/tags/GPTbot" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GPTbot</span></a>, <a href="https://infosec.space/tags/ByteSpider" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ByteSpider</span></a> (which <a href="https://www.youtube.com/watch?v=Hi5sd3WEh0c" rel="nofollow noopener" target="_blank">literally</a> <a href="https://infosec.space/tags/DDoS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DDoS</span></a>'d <a href="https://infosec.space/tags/MattKC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MattKC</span></a> because <a href="https://infosec.space/tags/ClownFlare" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ClownFlare</span></a> are a <em>criminally incompetent</em> <a href="https://infosec.space/tags/RogueISP" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>RogueISP</span></a>!) to <a href="https://infosec.space/tags/Hetzner" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Hetzner</span></a>'s <a href="http://hil-speed.hetzner.com/" rel="nofollow noopener" target="_blank">10GB Speedtest file</a> which can be found at <code>http://hil-speed.hetzner.com/10GB.bin</code> as an extra middlefinger!</li></ul><p><a href="https://infosec.space/tags/Cloudflare" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Cloudflare</span></a> <a href="https://infosec.space/tags/hetznered" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>hetznered</span></a> <a href="https://infosec.space/tags/ByteDance" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ByteDance</span></a> <a href="https://infosec.space/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ChatGPT</span></a></p>
🌈 vanta rainbow black 🌈<p>found another scraper indexer thingy</p><p><a href="https://mastogizmos.com" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">mastogizmos.com</span><span class="invisible"></span></a></p><p><a href="https://cyberpunk.lol/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://cyberpunk.lol/tags/indexer" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>indexer</span></a> <a href="https://cyberpunk.lol/tags/fediblock" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>fediblock</span></a> <a href="https://cyberpunk.lol/tags/FediAdmin" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>FediAdmin</span></a></p>
🌈 vanta rainbow black 🌈<p>think i might've maybe found another scraper indexer thingy but i'm not sure</p><p>if someone else who knows their shit better could take a look and lmk that'd be great</p><p><a href="https://mastogizmos.com" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">mastogizmos.com</span><span class="invisible"></span></a></p><p><a href="https://cyberpunk.lol/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://cyberpunk.lol/tags/indexer" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>indexer</span></a></p>
Kevin Karhan :verified:<p><span class="h-card" translate="no"><a href="https://tech.lgbt/@xogium" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>xogium</span></a></span> this issue of <em>excessive crawlers</em> is sadly nothing new. <span class="h-card" translate="no"><a href="https://hasthebig.gay/users/MattKC" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>MattKC</span></a></span> / <a href="https://infosec.space/tags/MattKC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MattKC</span></a> experienced the same with <a href="https://infosec.space/tags/ByteSpider" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ByteSpider</span></a>, the <a href="https://infosec.space/tags/Scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Scraper</span></a> used by <a href="https://infosec.space/tags/TikTok" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>TikTok</span></a> which results basically in his site getting <a href="https://infosec.space/tags/DDoS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DDoS</span></a>'d despite <a href="https://infosec.space/tags/ClownFlare" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ClownFlare</span></a> being tasked to prevent it!</p><p><a href="https://youtu.be/Hi5sd3WEh0c" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">youtu.be/Hi5sd3WEh0c</span><span class="invisible"></span></a></p><p>Personally, I've run out of patience and tolerance for such actions by <a href="https://infosec.space/tags/GAFAMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GAFAMs</span></a> and <a href="https://infosec.space/tags/TechBros" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>TechBros</span></a> and I'm so close to just <a href="https://github.com/greyhat-academy/lists.d/blob/1bb1f185d8e4451e541d820dde96b19f40c44eae/blocklists.list.tsv#L23" rel="nofollow noopener" target="_blank">blocklist their entire ASN</a> as a matter of principle!</p><ul><li>It's just that I'd likely have to make an entire dedicaded blocklist and toolup some script to pull a BGP feed or rather IP assignments data <a href="https://ipinfo.io/AS32934" rel="nofollow noopener" target="_blank">for their entire ASN</a> and submit these in <a href="https://infosec.space/tags/git" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>git</span></a> as branch updates, merge that and block said network in it's entirely <a href="https://github.com/greyhat-academy/lists.d/blob/main/milintel.ipv4.block.list.tsv" rel="nofollow noopener" target="_blank">as I did with the DoD networks</a>.</li></ul>
🧿🪬🍄🌈🎮💻🚲🥓🎃💀🏴🛻🇺🇸<p>Y'all really putting a file on your webserver that says "don't look *here* if you're a bot!" and expecting people not to look there first 🤡 </p><p><a href="https://mastodon.social/tags/webDev" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>webDev</span></a> <a href="https://mastodon.social/tags/robotsTxt" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>robotsTxt</span></a> <a href="https://mastodon.social/tags/bots" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bots</span></a> <a href="https://mastodon.social/tags/search" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>search</span></a> <a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ai</span></a> <a href="https://mastodon.social/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llm</span></a> <a href="https://mastodon.social/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://mastodon.social/tags/privacy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>privacy</span></a> <a href="https://mastodon.social/tags/web" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>web</span></a></p>
🌈 vanta rainbow black 🌈<p>holy shit WHAT</p><p>that is certainly A Way to go about apologizing lol</p><p><a href="https://cyberpunk.lol/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://cyberpunk.lol/tags/indexer" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>indexer</span></a></p>
Austin Huang ❤️<p>With regards to the utoots.com <a href="https://mstdn.party/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a>:<br>1. It currently depends on a Mastodon instance flashist[.]video; it is recommended to block the instance. flashist.(me|health) and previously flashist.(org|vip|live) is also operated by the same person. Ban evasion is to be expected.<br>2. I wrote a GitHub issue about it, archived at <a href="https://archive.ph/8ynKh" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">archive.ph/8ynKh</span><span class="invisible"></span></a>. However he has chosen to cover up his GitHub profile instead.</p><p>Update: <a href="https://cyberpunk.lol/@vantablack/112849043193285926" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">cyberpunk.lol/@vantablack/1128</span><span class="invisible">49043193285926</span></a> (tldr: it's gone)</p><p><a href="https://mstdn.party/tags/FediBlock" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>FediBlock</span></a> <a href="https://mstdn.party/tags/MastoAdmin" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MastoAdmin</span></a> <a href="https://mstdn.party/tags/FediAdmin" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>FediAdmin</span></a></p>
🌈 vanta rainbow black 🌈<p>okay yeah <a href="https://utoots.com" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">utoots.com</span><span class="invisible"></span></a> is DEFINITELY a scraper</p><p>i've updated the original post, making a reply too since edits don't always federate cleanly</p><p><a href="https://cyberpunk.lol/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://cyberpunk.lol/tags/indexer" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>indexer</span></a> <a href="https://cyberpunk.lol/tags/fediblock" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>fediblock</span></a> <a href="https://cyberpunk.lol/tags/FediAdmin" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>FediAdmin</span></a></p>
🌈 vanta rainbow black 🌈<p>just found another scraper indexer thingy</p><p><a href="https://utoots.com" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">utoots.com</span><span class="invisible"></span></a></p><p><a href="https://cyberpunk.lol/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://cyberpunk.lol/tags/indexer" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>indexer</span></a> <a href="https://cyberpunk.lol/tags/fediblock" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>fediblock</span></a> <a href="https://cyberpunk.lol/tags/FediAdmin" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>FediAdmin</span></a></p>
Roni Laukkarinen<p>Ping <span class="h-card" translate="no"><a href="https://cathode.church/@s0" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>s0</span></a></span>, I guess it’s time to reset the counter. <a href="https://www.reddit.com/r/Mastodon/s/TKQ5lMdIoY" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">reddit.com/r/Mastodon/s/TKQ5lM</span><span class="invisible">dIoY</span></a></p><p><a href="https://cathode.church/fedi-scraper-counter.html" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">cathode.church/fedi-scraper-co</span><span class="invisible">unter.html</span></a></p><p><a href="https://mementomori.social/tags/Fediverse" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Fediverse</span></a> <a href="https://mementomori.social/tags/Search" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Search</span></a> <a href="https://mementomori.social/tags/Scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Scraper</span></a></p>
🧿🪬🍄🌈🎮💻🚲🥓🎃💀🏴🛻🇺🇸<p>Blocking robots.txt is not very cyberpunk</p><p><a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ai</span></a> <a href="https://mastodon.social/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llm</span></a> <a href="https://mastodon.social/tags/cyberpunk" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cyberpunk</span></a> <a href="https://mastodon.social/tags/openAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>openAI</span></a> <a href="https://mastodon.social/tags/perplexity" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>perplexity</span></a> <a href="https://mastodon.social/tags/anthropic" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>anthropic</span></a> <a href="https://mastodon.social/tags/gemini" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>gemini</span></a> <a href="https://mastodon.social/tags/claude" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>claude</span></a> <a href="https://mastodon.social/tags/stableDiffusion" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>stableDiffusion</span></a> <a href="https://mastodon.social/tags/aiArt" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>aiArt</span></a> <a href="https://mastodon.social/tags/llms" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llms</span></a> <a href="https://mastodon.social/tags/chatGPT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>chatGPT</span></a> <a href="https://mastodon.social/tags/copilot" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>copilot</span></a> <a href="https://mastodon.social/tags/robots" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>robots</span></a> <a href="https://mastodon.social/tags/robotstxt" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>robotstxt</span></a> <a href="https://mastodon.social/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://mastodon.social/tags/search" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>search</span></a> <a href="https://mastodon.social/tags/google" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>google</span></a> <a href="https://mastodon.social/tags/crawler" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>crawler</span></a></p>
🌈 vanta rainbow black 🌈<p><strong>⚠️ FEDI SCRAPER AND INDEXER ⚠️</strong></p><p>okay according to multiple peeps in the replies of the original post, this is indeed in fact a fedi scraper and indexer i found</p><p><a href="https://fediscanner.info" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">fediscanner.info</span><span class="invisible"></span></a></p><p><a href="https://cyberpunk.lol/tags/fediblock" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>fediblock</span></a> <a href="https://cyberpunk.lol/tags/scraper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scraper</span></a> <a href="https://cyberpunk.lol/tags/indexer" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>indexer</span></a></p>
TrendingLive feeds
About