You can absolutely train a nontrivial task model on a gaming gpu (image classifier, sound classifier, text model with a very structured input and output, etc). You can also post-train addon layers on top of existing open-weight models (LORA).
Ubuntu and bad updates
This is the only drama in your list I’m not familiar with already. What/when did they last break?
I would argue that Linux kernel scale basically doesn’t work. It is a huge barrier to would-be developers to get into. You might argue that’s a feature, but it’s pretty hard to argue that it doesn’t add a lot of friction to the process
Genuinely, pull requests being off platform without good integrated diff and merge conflict resolution tooling just doesn’t work at a certain scale.
I’m not aware of any public frontier LLM provider that uses a static seed for inference. Meaning, even with an identical prompt and identical model you will not get the same output. Seeds should absolutely come back with the streaming metadata on requests imho, but they don’t in any api/harness I’m aware of.
A model could hypothetically be trained to insert zero-width characters (I doubt any have though). But any other layer could also very trivially insert these codes. The inference engine could be designed to delay output streaming by however many tokens is required to embed their coding and ninja-insert them during the decode stream. A proxy between the inference engine could insert them. A harness could insert them. Hell, even the rendering javascript frontend in your browser could insert them.
Either the inference engine or proxy would be the prime target if they want to enable this on api responses as well as copy/paste from a chat interface. They could also do a combination of the above depending on final output mode.
You are definitely correct though that it’d be trivial to detect and strip by someone aware of it.
Not necessarily, there are valid “characters” that are not rendered in most text parsers, or are rendered as whitespace. A great example of this is the byte-order-mark (BOM). You can embed a BOM code in the whitespaces of a text string and it looks exactly the same to a human as one without.
I assume that’s what they’re talking about here
Here’s an example of a steganography encoding technique using this method. https://330k.github.io/misc_tools/unicode_steganography.html
I don’t think his argument is that we should restrict anything, more so that circumvention requires learning, and is a great motivator. I definitely can +1 this from my own youth. Learning how to bypass well-intentioned internet protections on school computers so I could play flash games sparked an interest in technology that I may never have discovered otherwise.
It’s a weird internal conflict to know that my core is shaped by an experience I otherwise believe shouldn’t exist in a reasonable society.
I don’t understand, is suggesting someone write a 7 line bash script in a Linux forum post about using a cli downloader, unreasonable?
It would be more accurate to say they try and block the vpn providers and Tor exit nodes. In practice, they do have ~95% of Tor exit nodes blacklisted but major vpn providers it’s closer to 40%. Vpn providers go through a lot of effort to hide their exit traffic and it’s pretty simple to use a provider that has a reasonably robust cli to just rotate nodes until you find one that works
Connect
Test known-good video using yt-dlp
If fail, use vpn cli to connect to a new node
Reeat until success
If your provider let’s you connect to specific nodes you can even store the last-known-good and use it on first connect until it gets blacklisted, extending uptime considerably
Ubuntu server (headless)
Install pulse, x11, awesomewm.
Building the gui stack yourself on top of Ubuntu server gives you all of the debian/Ubuntu stability and issue searchability with none of the gnome/canonical cruft.
You can’t just write off capital expenditure though. The hardware, even for “effecient” MOE inference is still very expensive to buy, house, run, and cool. Even assuming open-weight model serving at $0 r&d for the models themselves, mixing high-prefill workloads doesn’t batch well with decode heavy concurrency (or other prefill-heavy jobs). The moment you do anything nontrivial you start running into very complicated architectural problems to efficiently solve at scale.
Hardware that is useful for 5-10 years at most, plus development and support for the inference workflows, doesn’t leave a lot of margin on the table.
My gut, along with basically everything I read, suggests that not most (even pure inference) shops are not profitable and are still floating on loans or vc money.
Can you cite your source on the claim that “inference is currently insanely profitable”? Everything I read suggests that openai and anthropic lose money on their plans.
For music enthusiasts plexamp is also basically unbeatable. I welcome the day open source catches up.
People shit on it but there’s a lot of good open-source tooling that supports it.
There are nist l1 profiles
Tutorials and guides for everything
etc
Part of being a good sysadmin is knowing when not to reinvent the wheel. Ubuntu has a lot of options for vetted, hardened, “other people’s wheels.”
Also, for posterity, the competent ones are running the headless, server version of Ubuntu. (As opposed to the bloated mess that is Ubuntu Desktop). The server version catches a lot of flack it doesn’t deserve.
Code rewrites are always going to have growing pains. Rewriting gnu-corrutils in rust is a noble effort.
Microvms or containers could give you external control of the networking. Then you would put whatever you want behind warp inside the warp container/vm.
+1 to this observation. I run zfs arrays at both home and work and it’s way more likely that your controller is flaking than you have that many simultaneous drive failures.
The unfortunate reality though is that you can’t trust the current copy of this data, even the snapshots, unless the restore passes a scrub post-restore.
Copyparty is what I’ve been using. Seems very similar in philosophy.
+1 though for anti-discouragement. Competition breeds competence!