@AMS @cadey well, #Anubis allegedly does account for #Browsers like #LynxBrowser and handles them gracefully, unless they violate thresholds to stop scrapers that fraudulently use a fake #UserAgent, and then it'll soft-ban those.
@AMS @cadey well, #Anubis allegedly does account for #Browsers like #LynxBrowser and handles them gracefully, unless they violate thresholds to stop scrapers that fraudulently use a fake #UserAgent, and then it'll soft-ban those.
@lukeshu So I guess #Anubis has an explicit exception to handle #Lynx and will instead rely on rate-limits and other static means to detect #scrapers and handle with #UserAgent #abuse cases, like #fail2ban-style autobanning of violating IPs...
I wounder if anyone has tried using Anubis on @torproject / #Tor to protect #OnionService|s since that would be a reasonable application for it as well.
@briankrebs why am I not surprised at this?
Needless to say, it was only a matter of time till we see auto-copying #JavaScript to be weaponized for that...
@S_Paternotte @GrapheneOS meanwhile I see #OpenAI literally using falsified #UserAgent|s and #DDoS'ing clients at work so hard I have to ban entire ASNs and /10 networks just because they ca't be assed to respect the robots.txt
and refuse to accept beibg given 403 errors.
-Needless to say banning #GrapheneOS which are by far the most security-focussed and most diligent in terms of #Aftermarket-#Android-#ROM|s whilst not banning #outdated Android versions is like banning a "#SecureBoot|ed" #UbuntuLTS or #OpenBSD installation and going out of one's way to brick #Wine whilst still supporting #WindowsXP in 2025!
#TLDR my brain-drippings generated by your toot, feel free to skip
this is very interesting…
my instinctive response to this (as a long-ago web dev but casual keeper-upper with stuff) was, “don’t you get what you need from #UserAgent strings any more?” so I went (a short way) down a rabbit hole…
despite recent changes by Google to restrict info leakage by Chrome user agents ( yay #privacy^), it seems like they still give plenty of info about devices that can be used to put them into at least buckets of “phone”, “tablet”, and “desktop”^^ (happy to be corrected, of course
)
do we really need websites tailored to the pixel rather than a few buckets based on approximate sizes, with layouts to match those sizes using percentage-based definitions for elements / containers? is that not sufficient for decent responsive / adaptive design any more?
is the #UX *actually* significantly better using exact-pixel tailoring?
do sites see significantly higher bounce rates or lower sales conversion or whatever metric they care about if we approximate with well-thought-out layouts?
[rant]what I mostly see these days (that I loathe) is web devs doing “mobile only” rather than “mobile first” layouts that are ridiculously large & shouty on desktops and provide almost no detail on anything ‘coz they sell to assume that no one will ever look at the site on a non-mobile device… [/rant]
if a site offers me something that fits well enough into “mobile” or “desktop”^^^ (and *actually* allows me to switch between them if I explicitly request it to do so), AND gives me an on-page way to increase / decrease text size without altering other elements, I’m usually very happy (rant about WhyTF every mobile browser doesn’t bake per-site text size controls into their #UI left for another day )
^ well, more like “privacy” since you can still pull all the details from headers or via JS so ¯\_(ツ)_/¯
^^ e.g. data from https://www.useragents.me/
^^^ bonus points for offering “tablet” as well
The #UserAgent of the #browser feature of a Precor #treadmill at my place. It somehow claims to be an iPhone 6.1.3, yet the browser UI looks Android (it uses Android 4 cursors and a keyboard in that style). Anyways, it does lack support of modern web features (its homepage is the low-spec version of Google, and the styling is completely broken on many sites), so the only thing I've been reading on it is NPR Text...
(And yes, I've been running a bit recently...)
#HeadsUp, my entire website was scraped by a new #AI #LLM scraper #Ai2Bot . #UserAgent is:
Mozilla/5.0 (compatible) Ai2Bot-Dolma (+https://www.allenai.org/crawler)
Some of the IPs:
100.27.119.55
100.28.58.124
100.28.84.26
174.174.51.252
18.205.170.247
3.214.92.12
34.198.246.31
44.194.249.41
44.196.112.226
44.196.49.185
50.19.235.114
54.144.167.95
54.162.34.15
54.236.199.27
54.237.131.196
54.80.81.52
Would have blocked earlier if I had noticed. Reads robots.txt, no idea if respected.
Privacy matters! But what if the tools meant to protect us are being misused? Our latest study (to appear ARES '24) reveals surprising facts about HTTP Client Hints (HTTP CHs) on the Web. [THREAD]
Paper + Website: https://rbainfo.org/clienthints
#Youtube intentionally slowdown your loading time if you use #Firefox.
Video demostration: https://www.reddit.com/r/youtube/comments/17z8hsz/youtube_has_started_to_artificially_slow_down/
(Possible) fix:
1. Click on uBlock icon
2. Dashboard button
3. Add "www.youtube.com##+js(nano-stb, resolve(1), *, 0.001)"
4. Apply changes
Alternatively you can change your #UserAgent to #chrome.
Nice behaviour, #Google!
EDIT: **UPDATE IN REPLIES**
Did you know we've been reducing the data in the #Chrome #UserAgent string for a while? This phase replaces Android version and device model with fixed values:
Android 13; Pixel 7 Android 10; K
If you use those values, I've got a few links for you
https://goo.gle/ua-reduction