Explore AI
Should you run this locally at all?
Local AI is a hobby with a real hardware bill and a real learning curve. Here is when it repays that, and when it does not.
The local-AI hobby has a strong bias: everyone in it has already decided. These appendices are more useful if they start with the argument against.
Run locally when
The data cannot leave your machine. This is the case that justifies everything else. Client files, medical notes, legal documents, unpublished writing, anything with someone else's name in it. A hosted model is a third party, and "we do not train on your data" is a policy, not a physics.
The volume is high and the task is dull. Transcription, classification, extraction, translation of a large archive. Hosted pricing is comfortable at ten documents and alarming at ten thousand.
You need it to keep working. A downloaded model does not change its terms, get deprecated, raise its price, or go down on a Tuesday.
You want to learn how this works. Running a small model and watching it fail teaches more than reading about large ones.
You are offline by circumstance. A plane, a site visit, a ship, a country with unreliable connectivity.
Do not run locally when
You need the best possible answer. The best hosted models are better than anything you can run on consumer hardware, and the gap is wider on hard reasoning and agentic tasks than on chat.
You need it today, and do not want to troubleshoot. Local inference involves drivers, quantisations, context limits and memory arithmetic. It works, but it is work.
You have 8 GB of memory and a deadline. You will spend more time fighting the machine than using it.
You only need it a few times a month. The cheapest hardware that does this well costs more than years of a hosted subscription.
The awkward comparison nobody makes
The usual claim is that local is free. It is not:
- A machine that runs 30B-class models well costs several hundred to a few thousand euro.
- It draws real electricity under sustained load.
- Your time costs something, especially in the first month.
Against that: a hosted subscription is a recurring cost, and usage-based pricing scales with work rather than being fixed. If you would spend €20 a month on hosted AI, local hardware pays for itself in a few years — and only if you actually needed the privacy or the volume. If you did not, you bought a hobby.
The one-line test
Would you be comfortable pasting this text into a stranger's website? > > If no — local, or do not do it at all. > If yes — use the hosted model, and spend the saved effort on the actual work.
That test resolves most cases in about two seconds.