Download It While You Can - The Future Of Local AI

Last time we wrote about governments discovering, nearly four years late, that their clerks had been feeding citizens' data into chatbots. Their solution: move everything to Microsoft Copilot, under contract, with a security agreement. The state will have its AI.

Corporations are doing fine as well. Nvidia already charges $4,500 per GPU per year for its AI Enterprise software. At GTC Berlin on 22 October, Jensen Huang floated a new figure: around $2,000 per GPU per year to run Nvidia's software for AI agents. For now, this is about data centres.

For now.

For the citizen, here is the year so far.

In February, OpenAI retired GPT-4o, the model people liked because it could hold a conversation and follow an argument.

In April, the European Commission declared its age verification app ready, so that you can prove you're an adult before the internet lets you in.

On 31 August, the Commission designated ChatGPT a "Very Large Online Search Engine" under the Digital Services Act, which means OpenAI must now assess the system's effects on "civic discourse", elections and your mental wellbeing.

On 3 September, OpenAI launched a censored GPT-6 "Astra".

There is a way out, and it's been sitting in plain sight: open-weight models.

An open-weight model is the AI itself, as a file. You download it once, put it on your disk and run it on your own machine. No account, no subscription, no server on the other side.

That gives you four things no cloud chatbot will:

• Your data stays home. Nothing leaves the machine. No provider, no "approved vendor", no engineer asking for permission to look.
• No lecture. Community builds with names like "abliterated" or "Heretic" strip out the reflex to refuse, hedge and moralise.
• No switch-off. The file on your disk doesn't get retired. If you liked it in September, you'll still have it in December.
• No subscription.

It isn't free, and it isn't magic.

You need hardware. A gaming graphics card with 8 GB runs small models well. 16 to 24 GB gets you into the serious mid-size class. A machine with 64 GB of unified memory runs models that would have been data-centre material not long ago.

You won't run the giants. The top open models, such as Kimi K3 or GLM-5.3, are hundreds of billions to trillions of parameters and need racks of server GPUs. But the gap is shrinking. Local AI writes, summarises, translates, codes and argues back without a disclaimer.

How long will it last?

Here's where the cynic in us takes over.

The EU AI Act already covers general-purpose models, open ones included. Open-source models get a partial exemption, sparing them some documentation duties but not the copyright policy or the training-data summary. Models classed as "systemic risk" get no exemption at all.

The hosting sites are starting to move. Hugging Face, where most open models live, has already pulled an abliterated build of GLM-5.3, one that openly advertised itself for cyberattacks. Thousands of other uncensored models are still up. Still.

Let's make a prediction.

When the first government restricts local AI, it won't say "control". It will say "protecting citizens' data". The same citizens whose data it spent four years leaving open to any clerk with a browser tab, and is now openly donating for AI training.

On 21 October, the EU member states agreed their position on the "Digital Omnibus", a package sold as "simplification".

One of its changes to the GDPR: developing and running AI systems can rely on "legitimate interest". No consent needed. The privacy group noyb reads it plainly: companies would automatically be assumed to have an overriding interest. When the draft circulated in September, its founder Max Schrems called it a "digital expropriation" of Europeans.

Pseudonymised data would no longer count as personal data for anyone who can't identify the people behind it. Remove the names, pass it on, and for the company receiving it, the GDPR simply stops applying.

Should anything leak, the deadline to report a data breach would grow from 72 to 96 hours.

The European Parliament still has to vote.

Who profits? Whoever has spent twenty years collecting Europeans' data and now gets to train on it. Google, Meta, Microsoft. Yes, Microsoft again.

What to do

Software

- LM Studio: download, pick a model from the built-in catalogue, chat. The easiest start for a lay person.

- Ollama: one command per model, runs in the background, plenty of apps connect to it.

- llama.cpp: the engine under most of the above. For those who like a terminal.

What runs where (as of late October 2026)

- 8 GB graphics card: NVIDIA Nemotron 3.5 Bolt E4B (released 23 October, about 4 GB of video memory), Google Gemma 4 12B at 4-bit.

- 16–24 GB: Alibaba Qwen3.8-27B, Apache 2.0 licence, switchable reasoning, runs in 16 GB. Uncensored community builds exist.

- 64 GB unified memory: NVIDIA Nemotron 3.5 Bolt 74B-A9B, 24 GB of weights, 256K context, reads images and video.

Download the models you like NOW and back them up. Hugging Face is a shop window, not an archive.