LLM@home - privacy-first local inference in practice
Speaker: Evgenii, Xecut Hackerspace Belgrade
Large Language Models are increasingly being integrated into technical workflows, but their practical usefulness strongly depends on the use case, model choice, and how they are deployed. In some scenarios they can be helpful tooling; in others they add complexity without much benefit.
In practice there is a trade-off between privacy, performance, and cost. Cloud-hosted models offer strong performance and low operational overhead, but operate through external services. Small local models keep everything on your own hardware, but often fall short in quality or speed. A third option is running larger models locally - not a cheap setup, but a realistic path for a dedicated hobbyist who wants to understand and control the full stack.
This talk is a practical walkthrough of building and operating a security-first home LLM setup: a 64-core / 512 GB RAM machine used for local inference, capable of running 200B+ parameter models at kinda usable speeds. I’ll cover hardware choices, firmware and OS considerations, inference software, and real bottlenecks - from an engineering and operations perspective rather than product marketing.
The focus is on experience and knowledge sharing: what works, what doesn’t, and what it actually takes to run capable models locally with a privacy-first mindset.
Hoćete li da napravimo događaj da idemo po gradu i popunjavamo OpenStreetMaps koristeći “street complete” ili “scee”? Vidim u street complete ima i da se podesi “rad u timu” gde idemo zajedno, ali dobijamo različite zadatke/pitanja. A možemo i da krenemo svako na svoju stranu, pa onda da se sastanemo na kraju.
Pošto ljudi slabo pišu na forumu, računaću da ste za ako ovaj post ima više od pet like-ova. Ali bilo bi lepo da onaj ko stavi like i dođe na događaj.
Možemo neki ponedeljak u martu pcb radionicu koju smo odložili. Ja bih napravio neku brzu prezentaciju, a posle prezentacije možemo da napravimo par komada.
Ako neko ima ideju za PCB, spremite layout. Maksimalne dimenzije su negde oko 130x80 mm, a minimum trace width oko 0.3 mm. @coja je pomenuo da želi PCB za neki svoj esp kod kuće, to zvuči kao super ideja.