I mean, I enjoy linux sysadmining, but fighting bots takes time, experimentation, and research, and there’s other stuff I should be doing. For example, accessibility updates to our websites. But, accessibility doesn’t matter a lick if you can’t access the website anyway due to timeouts.
I’m a systems librarian in an academic library. I moved over the Lemmy after Rexxit 2023. I’ve had an account on sdf.org since 2009 (under a different username), and so I chose this instance out of a sense of nostalgia. I do all sorts of fiber arts (knitting, cross stitch, sewing) and love dogs.
- 4 Posts
- 128 Comments
Yep, they’ll just burn taxpayer resources (me and my poor servers) because it’s not like they pay taxes anyway (assuming they are either a corporation or not based in the same locality as I am).
There’s only one of me and if I’m working on keeping the servers bare minimum functional today I’m not working on making something more awesome for tomorrow. “Linux sysadmin” is only supposed to be up to 30% of my job.
Like I said, [edit: at one point] Facebook requested my robots.txt multiple times a second. You’ve not convinced me that bot writers care about efficiency.
[edit: they’ve since stopped, possibly because now I give a 404 to anything claiming to be from facebook]
I just looked at my log for this morning. 23% of my total requests were from the useragent GoogleOther. Other visitors include GPTBot, SemanticScholarBot, and Turnitin. That’s the crawlers that are still trying after I’ve had Anubis on the site for over a month. It was much, much worse before, when they could crawl the site, instead of being blocked.
That doesn’t include the bots that lie about being bots. Looking back at an older screenshot of a monitors—I don’t have the logs themselves anymore—I seriously doubt I had 43,000 unique visitors using Windows per day in March.
Timing and request patterns. The increase in traffic coincided with the increase in AI in the marketplace. Before, we’d get hit by bots in waves and we’d just suck it up for a day. Now it’s constant. The request patterns are deep deep solr requests, with far more filters than any human would ever use. These are expensive requests and the results aren’t any more informative that just scooping up the nicely formatted EAD/XML finding aids we provide.
And, TBH, I don’t care if it’s AI. I care that it’s rude. If the bots respected robots.txt then I’d be fine with them. They don’t and they break stuff for actual researchers.
You’re right. AI didn’t just triple the traffic to my tiny archive’s site. It way more than tripled it. After implementing Anubis, we went from 3000 ‘unique’ visitors down to 20 in a half-day. Twenty is a much more expected number for a small college archive in the summer. That’s before I did any fine-tuning to Anubis, just the default settings.
I was getting constant outage reports. Now I’m not.
For us, it’s not about protecting our IP. We want folks to get to find out information. That’s why we write finding aids, scan it, accession it. But, allowing bots to siphon it all up inefficiently was denying everyone access to it.
And if you think bots aren’t inefficient, explain why Facebook requests my robots.txt 10 times a second.
Yes, it’s a þorn.
Oh, shiny. That could do.
I’d love to replace it but I’m not sure I could deal with CRT whine anymore.
I was on my Gentoo linux laptop trying to find an open WiFi network on a sidewalk when someone came out of the nearby bank to shoo me away because she was told I was “trying to hack the bank”.
I was just trying SSH into my school account to send an email using PINE to let my parents know my train was 5 hours delayed. This would have been 2005 or so.
My parents got rid of mine without asking when I went to college in the 2000s :(
Y’all have some good points. What I’m hearing is “install it on a fresh hard drive, play around, then move on to something more stable.”
Well, I was thinking of moving to Linux full-time anyway now that my Windows install is obsolete. Any reason to avoid this distro? Past experience is with Ubuntu, Gentoo, and SuSE. I mostly game.
You’d think that, but I’ve had the command “get a tan for God’s sake you’re transparent” used as an insult against me. You can be too white for white supremacists.
grysbok@lemmy.sdf.orgto
Technology@lemmy.world•“You can't be expected to have a successful AI program when every article, book or anything else that you've read or studied, you're supposed to pay for”, President Trump saysEnglish
3·4 months agoMy library, you have to check out books on reserve from the circulation desk. They’re for in-library use only, 3 or 6 hours at a time, and if you take it into a study room and scan the whole thing with your phone we saw nothing.
We don’t like the constant churn of textbooks, either. They eat into our budget. We really appreciate when a professor lends us their personal copies of a textbook for us to keep on reserve. We also try and steer instructions to Open Educational Resources (OER), which are available for free.
Wealth disparity sucks and shouldn’t result in different access to education.
grysbok@lemmy.sdf.orgOPto
Programmer Humor@programming.dev•fix(blog/v1.21.1): we avoid breaking changes · TecharoHQ/anubis@1cb1352English
1·5 months agodeleted by creator
grysbok@lemmy.sdf.orgto
Leopards Ate My Face@lemmy.world•A clinic blames its closing on Trump’s Medicaid cuts. Patients don’t buy it.English
2·5 months agoThank you for clarifying. TBH I got who was saying what a tad mixed up in the thread.
I can get behind not outright mocking folks that seem interested in leaving the cult. I just also think the sadistic and casually cruel folk are the norm rather than the outlier, based on experiences growing up in a Red State and visiting as an adult.
grysbok@lemmy.sdf.orgto
Leopards Ate My Face@lemmy.world•A clinic blames its closing on Trump’s Medicaid cuts. Patients don’t buy it.English
3·5 months agoI’m not convinced that “showing empathy” is the way to court Republicans. It’s just seen as weakness, and weakness is a joke to them.
I’m thinking back to my grandmother’s funeral in Appalachia. She had a small terrier as her companion in her final years. They were inseparable. Multiple of her male relatives made the joke “I thought we were just going to toss the dog in the grave and bury him with her”. Hardy-har har. The dog was small and therefore weak and therefore a joke.
There were similar scenes between my mom and her coworkers when my dad was in hospital dieing of COVID-19. I’m not going into them because I already done triggered myself.
Anyways, that’s how I ended up with a surprise dog to take back home to a blue state. He’s still a good boy, even though he’s now a grumpy teenager.
grysbok@lemmy.sdf.orgto
Not The Onion@lemmy.world•Mattel introduces its first Barbie with Type 1 diabetesEnglish
3·5 months agoTBH, ADHD is a part of my identity and my ADHD coworkers clocked me, as did my boss whose daughter has ADHD. (Librarian-hood is not the realm of the neurotypical.)
So, not something folks at the grocery store would notice and mileage varies as to what is part of someone’s identity.




It’s nice to be able to call your parents when you’re bleeding out in the school atrium.