Qwen 3.6 35b A3B Q5 本地運行，令開發者憂慮軟件業前景

Honestly, tonight I had a realization moment. My token pan for qwen cloud expired, so I thought 'you know what, I wanna see what local AI can do'. For context, I'm working on my IDE, so I took my buildbox (ryzen 5500, 32gb DDR4, GTX 1070 8gb, Ubuntu Server), using Drone, let my AI set up an Ollama instance, running Qwen 3.6 35b A3B Q5. 22gb total, sits comfortably in ram, A3B sits comfortably in 8gb vram with room to spare... 35 TPS... Sounds like nothing, but considering a word average is probably around 1.2 tokens... Suddenly that becomes a sentence a second. I imagine practically everyone reading this, given we're developers and tend to put our money where it's earned, into hardware, likely has a comparable system? 35 TPS, is about 1/5th what you can expect from a cloud model, except this is running on the hardware you already own and it doesnt stop you from using cloud too... The reason it made me question life as I know it, is because 35b params used to be the models of legend, the things that only ran in cloud, because nobody could afford a GPU large enough to run it... Now 8gb of vram is enough to get quite reasonable performance out of it... It changed how I saw open weight models and the future of the entire industry. Go buy a 2nd hand RX 580 for $50, a scrap PC for $100, if ram wasnt so insanely expensive, $100 for 32gb of ram, now you added a new permanent node to your AI pool for $250 and it runs a damn good model, you can get away with half that if you go for a lighter model, or decide to offload execution to 1 system, instead of dispersing it to the nodes. Think about that, what you pay per year on AI subscriptions (or what some pay in 2 months), you can have a permanent AI added, no subscriptions and it doesnt stop at 1 node, add 10, add 100, it doesnt matter, the models are smart enough, that you'll barely miss cloud models.

I just read an article too on how Halo got ported with AI acceleration and improved, Adobe and Microsoft's suites are being redone in rust ALL AT ONCE! We're reaching the point where code isnt cheap anymore, it's worthlessly inexpensive, because for every name-brand, there's a vibe-coded knockoff that likely performs better... And that's scary as a developer, where are we all gunna be in 5 years? 'But enterprise companies pay for software', not if the knockoff is open source and they have AI too... To say we're screwed is an understatement. And maybe it's not a bad thing. The Halo port meant that the creative arts of making games is nearing it's final call, software companies are pushing open source, because trying to protect your code is impossible now, we're hitting the point where I honestly cant see what value any software can hold that a knockoff produced over a weekend doesnt. 'Infrastructure', it takes AI 10 minutes to get the same setup on Cloudflare, it's as secure, if not more than what you built. 'Quality', they set the agent up to patch it continuously. 'reliability', the rewrote it in rust, it takes less memory, runs faster and is memory safe by default. 'but mine looks better', they've used screenshots and replicated GUIs perfectly for years... What can we honestly provide today, that AI cant do better, cheaper, faster?

Even IT support is becoming a thing of the past, hell I built Drone exactly to do that, now I dont even troubleshoot myself anymore, I just let my AI do it for me, cuz it saves me the hassle. The part that should really scare you, is what comes after we're replaced. If it can rewrite adobe's suite, microsoft's suite, build an OS in 12 hours and write a complete ERP system in less than that, how much effort do you really think it takes to automate it's use? 'But surely entertainment is safe?', CGI replaced practical effects 20 years ago and AI generated content trends longer and harder than human created content today. Take Gangnam Style, it broke records... But that Ai generated trumpet sound used on TikTok... How many times do you think THAT's been heard? 1 billion times? Or closer to 10 billion times? The global economy has already shifted and we're all still playing catchup. The irony is, as anyone who works a desk job, where's our future?

Thank goodness it wont happen overnight and the water is barely coming to a boil, but if there's any advice I can give anyone, it's learn a physical skill for the day using a computer wont earn you a penny, because it might come sooner rather than later.

Sorry if all of this sounds super pessimistic, I imagine in a way, it's actually optimistic, if not for the fact that it means we're all on the chopping block... What are your thoughts?