@jmoiron.bsky.social replied: I should clarify, I think the OP is a good rundown of the parameters of this decision which is why I shared the link. They explicitly highlight Gemini free tier as something that makes the economics of local inference questionable, though I personally think all of the $20 plans d...
jmoiron plays the blues
@jmoiron.bsky.social replied: This is provided that demand on other components don't continue to raise prices on consumer computing equipment. I bought 2x 32GB CORSAIR modules in 2024 for $210 and they are now being sold for $899.
@jmoiron.bsky.social replied: The biggest change to the economics for local inference likely to happen in the next 12-24 months will be an increase in the number of PC "Ryzen AI Max" style unified architecture chips to compete with the Mac Pro, the current local inference machine of choice.
@jmoiron.bsky.social replied: Google's search index in 2006 was ~850TB of data. This was a very large cluster at the time, but with nearly 20 years of progress, Seagate's latest and greatest is a $800 36TB drive. Replicating the 2006 index "locally" would still be around $20,000 just for storage.
@jmoiron.bsky.social replied: For all of the stories of the Apollo computers having fewer transistors than the circuitry in a novelty keychain, it takes forever for consumer hardware to catch up to the capabilities of a distributed system like a modern AI product.
@jmoiron.bsky.social replied: This change in capabilities didn't happen because smaller models ran better, it happened because we've created bigger and more focused models, and they can orchestrate themselves to use more silicon to do more things.
It's scaling out, not optimization.
@jmoiron.bsky.social replied: Small model capability is improving, but not nearly fast enough, and it hasn't kept pace with user expectations on what AI coding agents can do.
In just a year we've gone from "An AI agent can write small scripts badly" to "A swarm of AI agents can collaborate to plan and create...
@jmoiron.bsky.social replied: But if you get a few of them, or you buy access from some of the third party resellers, they're a lot cheaper and more capable than building something yourself.
Part of that is that hardware to run inference is still costly. I don't see this changing in 2026.
@jmoiron.bsky.social replied: There are non-cost reasons you might want to run local, but for a "hobbyist coder" use case, I just can't see any of them being better than the current offerings.
Yes, the $20 plans have been slowly undermined by stricter metering, because they don't actually come with any guara...
@jmoiron.bsky.social I hope local inference becomes cost effective someday, but that day isn't today. Most hobbyists who want to use AI can easily run with a $20/mo plan and there's simply no competing against that until the bubble bursts and that rug gets pulled.
www.aiforswes.com/p/you-dont-n...
@jmoiron.bsky.social I went digging regarding what we currently know about AI's environmental impact, and while the picture is not entirely unproblematic, it's not nearly as dire as often depicted: jmoiron.net/blog/on-ai-e...
@jmoiron.bsky.social I was very vocal in my opposition to the Iraq war before it started, and if this is where things are going, I'm strongly against the extrajudicial killings in the Caribbean and any escalation into war with Venezuela.
@jmoiron.bsky.social These claims were always transparently hypocritical projection and an excuse for thought policing, and if you reported them as anything else then you're a mark and should not be in journalism.
www.cnn.com/2025/12/15/p...
@jmoiron.bsky.social There's definitely a class of people who recognized their own dark frustrations and prejudices in MAGA's xenophobic rhetoric, relishing the thrill of validation, but seeing the horrific consequences of the followthrough laid bare they are nonetheless disgusted.
@jmoiron.bsky.social I wrote about housing preferences, suburban sprawl, and the way that centrist liberalism on the topic of housing serves to reinforce a conservative worldview tied to all sorts of problematic historical baggage.
jmoiron.net/blog/sublurb/
@jmoiron.bsky.social I've searched far and wide for a "Valencia Latte" outside of Japan and have been disappointed, so it's great to be back in Ebisu at the café where I had my first one.
@jmoiron.bsky.social My year-end shing mun river walk from Tai Wai out to the harbor is going to feel bittersweet this year.
The fire in Tai Po has now claimed 159 lives, more than twice as many live as the Grenfell Tower Fire, and it's made thousands homeless. Devastating. 💔
@jmoiron.bsky.social I have a theory that if we had cars that would not function if your abv was over the legal limit then the desirability of car centric suburbs would plummet overnight.
@jmoiron.bsky.social I went to Portugal a lot in the 90s to visit family, and it's still shocking to me when I go that it's basically a Nordic country now in terms of English proficiency.
@jmoiron.bsky.social replied: Surely fear of the airborn pandemic was also a factor. How do you untangle them?
I live in the most dense urban environment in the country but since many there have second homes it was practically empty mar~aug 2020. They didn't all decide city life wasn't worth it.
@jmoiron.bsky.social replied: > In downtown Minneapolis, rent concessions are still available, but they’re not as common — or generous — as they were a year ago because demand has been strong and vacancies are falling.
The article is mostly about how the demand is continuing to increase but there's no new su...