I hate Claude’s voice so much. So much ceremony spent on a few simple questions and caveats. I feel like AI defaults to assuming you have no idea what you’re doing. ChatGPT is the same but with different verbal tics.

8 USB ports and gigabit Ethernet are wild given the rest of the specs, though. Given the rest of the specs I’d assume 2 usb and 10/100 lan. Whatever their source is for 30 year old hardware must be out of NICs.

I’m an old cis het white guy. Those fuckers have nothing on me.

We can live in fear or we can live in solidarity, seeking to watch over one another and holding them to account. If no one stands up they win. There are worse causes to spend the balance of one’s days on if that’s the cost.

I voted against these fuckers and I can’t wait to do it again. And if they want, I will tell them to their face how absolutely fucked they can get.

Oh for sure. Agents often use under 30k tokens, but the orchestrator needs to have enough information to instruct the agents so it typically is a fair bit bigger in context. Agents tend to save you money on frontier models, but I’m skeptical about local LLMs. I suppose if you aren’t pressed for time it’s probably just fine. I haven’t played that much with it because anything big enough to bother with agents I typically feel is too big for local anyway. But I’m sure others have experimented more on that front than I have.

Qwen is really good. For coding, it’s the best local model I’ve found.

Yeah so I’ve been working with Claude on that. Typically it greps to find what it’s looking for and that can be a lot of tokens. So I created something halfway between rag and that (semantic search), and overall it lowers token usage a bit, but even if it optimistically reduces tokens 20%, Claude is hungry for docs.

There is automatic compaction, but I typically want to control that myself when I change focus (if I don’t just /clear it). Still I’d say almost all of the stuff I do runs north of 100k tokens.

64k context is nowhere near enough. I try to remember to compact Claude when I hit 200k but sometimes when I’m not paying attention I can hit 600k. Yeah I’ve written little Python scripts on my machine using qwen, but some of my work projects have 80k tokens of just documentation.

128 GB of unified memory

I thought I was asking a lot when work asked me to pick a computer (with no guidelines) and I asked for a 36GB MacBook and justified it by saying I needed to run local models to save money. I didn’t ask for nearly enough. To its credit it does run local models fast, but it’s very limited in context window. It starts slowing down long before hitting context sizes I hit in frontier models.

I use it lots of ways. My posts are long enough without getting into asides like that, don’t you think?

Great reply. I agree with a lot what you said, but I want to make two points:

First, based on a human breakdown and pointing of stories as if AI weren’t being used, using the methodology we have used for years, we are seeing real delivery velocity increases. Now measurement is tricky; story points are arbitrary for a team. Some of those gains are in stuff like “wrote AI tool” or “created AI documentation” which means we’re being more productive at things we never used to need to do. Still, when you factor all of that in as best you can, we are seeing a measurable 20% increase. (The analysis does point at 30%, but I see holes in how those numbers are built.)

Second:

we are in the dark about what pieces of information would be needed to shift the probability towards the thing we actually want

I don’t think we are completely in the dark. I think we are working on figuring out how to improve the context. The problem I see with the current approach is everyone is doing it independently, their methodology is poor because we can’t afford the tokens for exhaustive tests that are invalidated with the next model, and the results are entirely subjective (and frequently written by AI to sound like absolute success).

So I come up with something that is amazing for my teams, and you try it, but because you’re in a different domain my technique isn’t quite right, and because our lives have provided us with different context, you don’t implement it quite the way I would have, and so your results differ and is it the domain, or the implementation, or did I just get some lucky rolls?

One of the things I do in my spare time because I’ve been fascinated by AI for years, is I use AI to write stories. I come up with a premise, maybe do some world building, write an opening, and just let AI go and see what happens, steering it when necessary.

Now there are two ways to steer: you can “reroll” or you can just edit the text directly. And what I’ve seen over years of doing this is the more human text you inject in, the better the results are. Even if you reroll to steer in the direction you want, the AI will eventually start producing utter gibberish. The quality starts to go down and then falls off a cliff.

Human input slows that or even stops it altogether. Human input is an essential element in getting good results out of AI. Because bullshit built upon bullshit is multiplicative. An AI that is 99% good falls to 50% good very quickly.

My point is that humans need to interact with the AI to provide that context you correctly point out the AI can never have, and it has to be frequent because you get to a point of saturation where a human is overwhelmed by the volume of text.

I use Claude all the time at work. It is good. But it makes massive mistakes, it misses tests, it confidently says something it screwed up will be fixed by something that certainly isn’t the right way to fix the problem.

I recently explained to a colleague: if you can use 1 AIU (arbitrary quantity of ai usage) and get 10% productivity bump, that doesn’t mean 5 AIU gets you 50% and 10 doubles your speed. The AI will do and say promising things, make you believe it’s on the verge of solving the problems, but it never quite arrives. There’s always one more problem and if you’re very lucky the AI will find it itself, but most likely it will be found when you pass it on to another person and it’s completely useless.

Let me put it this way: in addition to development, I use Claude to help with production support issues. It wrote some scripts I didn’t have time to and it pulls logs and data from multiple systems — honestly it works great and has saved me so much time. But I’m constantly in meetings and so I set Claude to investigate an incident so I can focus on my meeting and return when I have time, and it gets RCA wrong well over 50% of the time.

If it is so bad at RCA, how do you imagine it is fixing the bugs in the code it finds? Badly. It misunderstands the cause of problems, and so it fixes the wrong things until it has cobbled together the creakiest of code that passes the test. In fact I think AI is far worse at fixing code than it is at writing it in the first place.

I’m not anti AI. I’m trying to find ways to make it effective. And my teams are seeing 20-30% productivity gains - I think because they are skeptical about AI rather than trusting. But it has to be used appropriately, and everywhere I look, even within my own company, people are trying to do too much with it and creating huge problems I have to sort through.

“Don’t tell me what isn’t so,” has to be something I say on an hourly basis when I use AI. If you ever correct anything, it just writes itself little notes like doing something dumb would be the obvious behavior otherwise.

“This service does not use JPA, it uses JDBC, so don’t add Hibernate to the dependencies.”

Motherfucker, what? Four words: “this service uses JDBC.” I swear it’s a gimmick to make AI use more tokens.

Connect to spray bottle aimed at… well nope.

Emdash is useful punctuation — I use it all the fucking time.

On the other hand, “it’s not foo; it’s bar,” once a quirky rhetorical flourish, now scans lazy and sloppy as shit.

Also, swearing like a motherfucking sailor helps establish one’s humanity. So fuck off. No offense. Have a lovely day!

Do you have any scientific evidence to support this claim?

Yes, thank you — you provided it. According to the paper cited, people with more familiarity with AI are much better at detecting its output than an average person. But I already knew that empirically.

What model are you using?

Mostly a fine tuned GLM 5.2 purpose-built for writing. Not that I haven’t also used frontier models but they suck at uncensored roleplay with violence and conflict and evil bad guys and such. GLM still fails with errors of attribution and state.

Bold claim from someone who spends hours writing AI stories for themself. Then again, I guess I am assuming you consider yourself a person who enjoys reading.

shrug It’s pretty bad but it’s better than no roleplaying at all. Plus the interface lets me edit whatever I want so I can fix any outright errors.

I am completely certain I could get a decent AI model to output a full length book in that genre and she wouldn’t notice.

That is interesting. The only time I really tried to use AI to write was a noir mystery. Dear god did it suck. It couldn’t follow instructions at all. Show don’t tell is probably more on display in noir than other genres and it couldn’t do it without constant correction. It kept falling back to its standard voice of third person omniscient instead of unreliable narrator.

There were parts that were written well, but a lot that was just grating and gratuitous. I had to edit like 85% of the output. I had to keep arguing with ChatGPT to write differently. And every once in a while it would swallow its own writing because it tripped its own content warnings. I suppose that’s neither here nor there on the writing quality, but we are far from having an AI write a novel without being full of tells.

AI isn’t there yet in terms of a quality or undetectability. And it doesn’t look to me like it’s going to get there any time soon.

That isn’t really a rebuttal of my statement. It’s not even particularly salient. Sure your average person might like AI writing and not be able to tell it from human works — just like they can’t tell the authenticity of a photograph of Aunt Margaret from a photograph of Trump meeting an alien from Rigel IV. But there are plenty of people who can, and so anyone trying to pass slop off as their original work will be discovered. It’s just a matter of time.

That being said, I use AI quite a bit in writing stories — though it’s more like solo roleplaying. I don’t think it’s particularly good at all, but it does while away the hours sometimes, as long as your expectations are low. I can’t see anyone who enjoys reading being enamored with AI. My wife reads over 300 books per year, and she spots it pretty easily.

If they aren’t heavily editing, the text will be full of tells anyway. I don’t think anyone is passing slop off as their own for long. They either use it a little bit and edit it enough to obliterate any markers and tells (which is fine, imo), or they are going to be discovered.

My server sits in Frankfurt. So everyone overseas waits forever for what is basically a text file. Unacceptable.

Mate my first internet connection was a 2400 baud modem. Waiting a whole second for a text file is entirely acceptable.

No, I get it, and good job. It’s about figuring out how to do it, and good job there. I’d have done the same back in the day. I have a raspberry pi with an ssd just begging for something to do. I have more ideas than time, though. Work is consuming all of my IT energy. I’m living vicariously through you.