# LLMs, marketinski receno vestacka inteligencija (nije)

**URL:** <https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938>\
**Category:** General\
**Tags:** tutorial\
**Created:** [January 31, 2025, 9:50pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938 "2025-01-31T21:50:56Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [January 31, 2025, 9:50pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/1 "2025-01-31T21:50:56Z")

</div>

Trebalo je ranije da otvorim ovaj thread, al eo ga sad, uz hype-a oko Deepseeka.

> ChatGPT, Perplexity, Coopilot i ostali toolovi su pokrenuti na tudjem serveru, ako koristite neki od tih toolva imajte na umu da time pomazete [proprietary software-u](https://en.wikipedia.org/wiki/Proprietary_software) i pritom ugrozavate svoju privatnost, moguce je da se loguje i svaki vas prompt, tako da pazite i sta pisete. Poslednji put kad sam pogledao za registraciju na ChatGPT-u je potrebno ostaviti broj telefona.

Alternativa je da se lokalno pokrene [LLM](https://en.wikipedia.org/wiki/Large_language_model), pa bih probao jednostavno da objasnim kako poceti i koje su opcije.

- [Ollama](https://ollama.com/) - [meta](https://en.wikipedia.org/wiki/Meta_Platforms)-in open source program koji pokrece LLM-ove ([github](https://github.com/ollama/ollama)) , jednostavno za instalaciju
- [Llama.cpp](https://github.com/ggerganov/llama.cpp) - malo kompikovanije, za mnoge OS je potrebno buildati ga [llm gen guide](https://deepwiki.com/ggml-org/llama.cpp/2.1-installation)
- [PocketPal](https://github.com/a-ghorbani/pocketpal-ai) - za mobilni, iz nekog razloga nije na F-Droidu, voditi racuna da ne sprzite fon, moj se pregreva

## Clients

- [Spisak UI klienata](https://github.com/ollama/ollama?tab=readme-ov-file#web--desktop)
- [Open web ui](https://github.com/open-webui/open-webui) - msm da ima najvise feature-a
- [Oatmeal](https://github.com/dustinblackman/oatmeal) - terminal user interface (TUI)

## Instalacija

- [ollama](https://ollama.com/download/linux) download ili install preko skripte, neki package manageri na linuxu ga vec imaju (znm za pacman)
- llama.cpp [build guide](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md), [pre-built install](https://github.com/ggerganov/llama.cpp/blob/master/docs/install.md)

> Sigurnija opcija je pokrenuti ih unutar virtualne masine ili containera (lxc, docker,..).

## Modeli

- Ollama modeli se instaliraju preko komande koja se moze naci kod njih na [sajtu](https://ollama.com/search), odabere se velicina modela i copy paste u terminal
- Llama.cpp je malo kompilovaniji, potrebno je skinuti model i onda ga gadjati komandom [readme iz repoa](https://github.com/ggerganov/llama.cpp/tree/master?tab=readme-ov-file#obtaining-and-quantizing-models)

Na pitanje kako odabrati model, nisam siguran sta pravi odgovor, zavisi od jacine masine na kome se pokrece, kolko sam skapirao, radi valjda na 2 nacina,

- obradu vrsi CPU (procesor), a model se smesta u RAM (memoriju)
- obradu vrsi GPU (graficka karta), a model se smesta u VRAM (memoriju) od GPU

Nzm da li se memorije kombinuju i nisam siguran sta se desava ako model zauzima vise nego model, ali u tom slicaju modeli odgovraju drasticno sporije. Vodeci se ovime, jasno je da modeli mogu da se pokrenu na vecini masina, pogledati da li je vas [GPU podrzan](https://github.com/ollama/ollama/blob/main/docs/gpu.md), instalirati nesto dodatno za podrsku ako treba, ako ne onda na CPU.  
Dobra stvar je spremiti jedan na laptopu i imati “search engine”, za slucajeve ako se nadjete negde bez internet konekcije.

Preporuku za modele, birati tako da zauzima manje memorije nego sto masina ima _slobodnog_ **rama** / **vrama** , da bi islo brzo, ako vam to nije bitno, nego preciznost odgovora, onda neki veliki i cekanje.

Modeli koje sam pokretao i valjaju (ovo se menja drasticno, posto izlaze novi non stop)

#### General (chat modeli)

- [Deepseek](https://ollama.com/library/deepseek-r1) 14b (9GB) i 32b (20GB)
- [Mistral](https://ollama.com/library/mistral)
- [Mistral-small](https://ollama.com/library/mistral-small) - 24b (14GB)
- [Dolphin Mistral](https://ollama.com/library/dolphin-mistral) - 7b (4.1GB)
- [Llama3.1](https://ollama.com/library/llama3.1) - 7b (4.9GB)
- [LLama3.2](https://ollama.com/library/llama3.2) - 3b (2GB)

#### Instruct modeli

Ovi modeli su vise direktni sa odgovorima, “rade po instrukciji”

- [Mistral Small Instruct](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501)

#### Code modeli

- probao sam par code modela, ali nisam nasao neki koristan, za python ih ima najvise tu se vrv moze naci neki koji je bolji od ovih generalnih
- codellama
- deepseekcoder

#### IDE assistenti (kao coopilot)

- [Continue](https://github.com/continuedev/continue) - VSCodium client
- [ollama.nvim](https://github.com/nomnivore/ollama.nvim) - NeoVim plugin
- [oatmeal.nvim](https://github.com/dustinblackman/oatmeal.nvim) - NeoVim chat bot

#### Image reasoning modeli

- [Llama 3.2 vision](https://ollama.com/library/llama3.2-vision) - prihvata sliku u promptu, moze procitati tekst sa slike [OCR](https://en.wikipedia.org/wiki/Optical_character_recognition), meni je radilo kad prompt glasi “Can you do OCR on this image”

##### Disclaimer:

> - Modeli cesto haluciniraju, izmisle odgovor, dosta informacija moze biti netacno itd
> - Modeli odgovaraju u zavisnosti od podataka na kojima su trenirani. Obratite paznju kad je model izasao. Proveru mozete odraditi sa nekim desavanjem u svetu, tipa pitati ko je trenutni predsednik odredjene drzave, koja je poslednja verzija nekog software-a, itd.
> - Neki modeli razumeju i odgovaraju i na srpskom

Subreddit `r/LocalLLaMA`

Ispravite me ako negde gresim, dopunite.. pa bi mozda mogli ovo da pretvorimo u [wiki](https://wiki.dmz.rs/) stranu. GL&HF

---

<div class="post-metadata">

**Author:** ![venmo](https://forum.dmz.rs/user_avatar/forum.dmz.rs/venmo/32/445_2.png) [@venmo](https://forum.dmz.rs/u/venmo)\
**Post date:** [February 1, 2025, 2:01pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/2 "2025-02-01T14:01:18Z")

</div>

Super post! Ja koristim llama.cpp i najbolje mi se pokazala, ne znam ni sam zasto. Jeste malo komplikovaniji proces instalacije jer mora kompilacija, ali nije to toliko strasno. Ono sto mi se svidja kod nje je to kada se kompajlira u `bin/` folderu ima i `llama-server` i onda odmah moze da se koristi veb interfejs.

`llama-server -m model.gguf`, i onda `localhost:8080` i to je to, moze i sistemski prompt da se namesti isto, lepo izgleda takodje.

Isto napomena, za llama.cpp treba da se koriste modeli u `.gguf` formatu. Ovo je [query](https://huggingface.co/models?library=gguf) za gguf modele sa hugginface, a tamo se i ostali modeli mogu naci, hugginface mu dodje kao neki github za modele. Nisam upoznat sa nekom alternativom odakle skinuti modele, pretpostavljam da ima po torrentima isto.

I ovo sve radi zacudjujuce brzo i na slabijem hardveru, meni na laptopou sa 8gb rama i ne bas jakim procesorom daje oko 5 tokena/sec i kvalitetne odgovore. Od modela sam koristio llama-3b-instruct i qwen-2.5b-instruct.

Sad ne znam koji model lepo radi na srpskom da moze da prevodi recimo, ovo do sada sam sve na engleskom probao.

---

<div class="post-metadata">

**Author:** ![venmo](https://forum.dmz.rs/user_avatar/forum.dmz.rs/venmo/32/445_2.png) [@venmo](https://forum.dmz.rs/u/venmo)\
**Post date:** [February 16, 2025, 9:03pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/3 "2025-02-16T21:03:42Z")

</div>

Jel probao neko RAG, rag omogucava da ubacimo svoje dokumenta “unutar” llm-a i da llm odatle pruza odgovore. Lokalno mislim naravno

---

<div class="post-metadata">

**Author:** ![seraphi](https://forum.dmz.rs/user_avatar/forum.dmz.rs/seraphi/32/738_2.png) [@seraphi](https://forum.dmz.rs/u/seraphi)\
**Post date:** [February 17, 2025, 10:50am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/4 "2025-02-17T10:50:19Z")

</div>

Probao sam rag sa llama3 langchain i chromadb. Prilicno dobro radi sa pdf i txt koji cak i nisu bili lepo sredjeni.

---

<div class="post-metadata">

**Author:** ![venmo](https://forum.dmz.rs/user_avatar/forum.dmz.rs/venmo/32/445_2.png) [@venmo](https://forum.dmz.rs/u/venmo)\
**Post date:** [February 17, 2025, 3:41pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/5 "2025-02-17T15:41:48Z")

</div>

a jesi koristio nesto gotovo, ili su sam pisao kod i postavljao sve ?

---

<div class="post-metadata">

**Author:** ![venmo](https://forum.dmz.rs/user_avatar/forum.dmz.rs/venmo/32/445_2.png) [@venmo](https://forum.dmz.rs/u/venmo)\
**Post date:** [February 25, 2025, 6:50am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/6 "2025-02-25T06:50:47Z")

</div>

> **[12 Days of EXO](https://blog.exolabs.net/day-4/)**
>
> 12 Days of Truly Open Innovation

Pokrenuli su llm na wind98, pentium 2 350mhz, 128mb rama, opisan je proces kako su to odradili.

Ovi exlobas se inace bave “demokratizacijom” ai pa ima zanimljivih stvari. Pominju [BitNet](https://arxiv.org/abs/2410.16144), ovo su neke dobre stvari koje bitnet donosi(iz teksta gore):

The advantages are striking:

```auto
   - A 7B parameter BitNet model needs only 1.38GB of storage - small enough to fit on most hardware, even decades-old PCs (the PC we used had a 1.6GB Hard Drive)
   - It's CPU-first: Microsoft's BitCPP can generate 52 tokens/second on an M2 Ultra CPU and 18 tokens/second on an Intel i7
   - Even more impressive: a 100B parameter BitNet can run on a single CPU at human reading speed (5-7 tokens/second)
    It's energy efficient: more than 50% more efficient than full-precision models

```

sve u svemu, zanimljivo, msm da je lokalno pokretanje llm-ova jako bitno stvar da se moze obzirom da su se llm-ovi bas popularizovali u zadnje vreme

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [March 1, 2025, 11:11pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/8 "2025-03-01T23:11:20Z")

</div>

@disu1950  
Ovaj post je o lokalnim LLM-ovima, sto u prevodu znaci da se modeli izvrsavaju na **lokalnom racunaru** i ne vrse nikakvu komunikaciju preko interneta, osim downloada softwera i samog modela, sto smanjuje rizik da se komunikacija prisluskuje, loguje, tj belezi negde jos, presretne ili nesto tome slicno.  
Deepseek i Gemini o kojima govoris izvrsavaju se na **tudjem serveru** i na taj nacin se **ugrozava privatnost** , sto pokusvamo da izbegnemo izvrsavajuci ih na **svojim masinama**.  
Moje ocekivanje je da ce se iz promptova raditi profiling, izvlaciti interesovanje koristnika i koristiti na svakojake nacine, najucestaliji je da ce se plasirati reklame.

Tipa, pitas ih koja je razlika izmedju QLED i OLED, random se zanimalo i vec sutra krenes da dobijes reklame o raznim tv-ovima i monitorima.

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [April 13, 2025, 8:36pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/10 "2025-04-13T20:36:08Z")

</div>

> <https://github.com/ollama/ollama/issues/1890>
>
> I'd like to have a way to update all downloaded models.
> Right now I have to pul…l each model separately.

Komande za update vec skinutih ollama modela

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [April 13, 2025, 9:14pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/11 "2025-04-13T21:14:51Z")

</div>

![image](https://forum.dmz.rs/uploads/default/original/2X/d/d61b2b327f6474431fcc017f09aabf054a3586af.jpeg)

lista modela sa kojima se igram

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [April 18, 2025, 8:12pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/12 "2025-04-18T20:12:39Z")

</div>

> **[GitHub - ymichael/open-codex: Lightweight coding agent that runs in your...](https://github.com/ymichael/open-codex)**
>
> Lightweight coding agent that runs in your terminal

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [April 27, 2025, 7:48pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/13 "2025-04-27T19:48:38Z")

</div>

> **[GPTMobile | F-Droid - Free and Open Source Android App Repository](https://f-droid.org/en/packages/dev.chungjungsoo.gptmobile/)**
>
> Your all in one chat assistant - Chat with multiple LLMs at once!

> **[maid | F-Droid - Free and Open Source Android App Repository](https://f-droid.org/en/packages/com.danemadsen.maid/)**
>
> Maid is an open source app for interacting with artificial intelligence models.

Mobile apps, koji se mogu povezati na ollama i llama.cpp API.  
Sa port forwardom i API kljucem… ili sigurnije sa hostovanim VPN-om, na fonu mozete imati pristup svojim modelima kao da ste kod kod kuce 🙂

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [May 28, 2025, 11:46am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/14 "2025-05-28T11:46:38Z")

</div>

Jednostavan tutorial za build llama.cpp i skidanje modela sa huggingface

> **[GGUF usage with llama.cpp](https://huggingface.co/docs/hub/en/gguf-llamacpp)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [May 28, 2025, 8:18pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/15 "2025-05-28T20:18:35Z")

</div>

> **[GitHub - SilasMarvin/lsp-ai: LSP-AI is an open-source language server that...](https://github.com/SilasMarvin/lsp-ai/)**
>
> LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.

---

<div class="post-metadata">

**Author:** ![txrpe](https://forum.dmz.rs/user_avatar/forum.dmz.rs/txrpe/32/1860_2.png) [@txrpe](https://forum.dmz.rs/u/txrpe)\
**Post date:** [August 27, 2025, 7:48am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/16 "2025-08-27T07:48:06Z")

</div>

Zanimljiv repozitorijum :^D bas smo juce pricali pre eventa o sistemskim promptovima.

> **[GitHub - asgeirtj/system\_prompts\_leaks: Collection of extracted System Prompts from...](https://github.com/asgeirtj/system_prompts_leaks)**
>
> Collection of extracted System Prompts from popular chatbots like ChatGPT, Claude & Gemini

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [August 27, 2025, 10:49pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/17 "2025-08-27T22:49:09Z")

</div>

> **[GitHub - pythops/tenere: 🤖 TUI interface for LLMs written in Rust](https://github.com/pythops/tenere)**
>
> 🤖 TUI interface for LLMs written in Rust

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [August 28, 2025, 9:52am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/18 "2025-08-28T09:52:04Z")

</div>

> **[GitHub - sigoden/aichat: All-in-one LLM CLI tool featuring Shell Assistant,...](https://github.com/sigoden/aichat)**
>
> All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.

> **[Configuration Guide](https://github.com/sigoden/aichat/wiki/Configuration-Guide#add-client-for-openai-compatible-api-provider)**
>
> All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more. - sigoden/aichat

---

<div class="post-metadata">

**Author:** ![atari](https://forum.dmz.rs/user_avatar/forum.dmz.rs/atari/32/1263_2.png) [@atari](https://forum.dmz.rs/u/atari)\
**Post date:** [August 28, 2025, 11:00am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/19 "2025-08-28T11:00:17Z")

</div>

@coja Koji lokalni model ti generise najbolji kod i/ili daje najsmislenije odgovore na coding related pitanja? Ili dva razlicita modela za ova dva use case-a?

---

<div class="post-metadata">

**Author:** ![coja](https://forum.dmz.rs/user_avatar/forum.dmz.rs/coja/32/1856_2.png) [@coja](https://forum.dmz.rs/u/coja)\
**Post date:** [August 28, 2025, 11:25am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/20 "2025-08-28T11:25:43Z")

</div>

Trenutno za coding koristim qwen3-coder:30b, najzadovoljniji sam odgovorima.  
A za regularna pitanja llama.3.1:8b, gemma3, mistral-small:22b (rade ok za oba)  
Svi sem llama3.1 su malo veci modeli i rade na GPU, llama3.1 mi je bio najudobniji za CPU

---

<div class="post-metadata">

**Author:** ![txrpe](https://forum.dmz.rs/user_avatar/forum.dmz.rs/txrpe/32/1860_2.png) [@txrpe](https://forum.dmz.rs/u/txrpe)\
**Post date:** [September 1, 2025, 10:19am UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/21 "2025-09-01T10:19:13Z")

</div>

System prompt Tumanka - chat bot-a manastira Tumane xD

> **[PrivateBin](https://pastebin.dmz.rs/?35e31769a400c16e#6JJEsnNP6NE96JRbkyvkhAAKv79E35bbsJ92Xa81aVuh)**
>
> Visit this link to see the note. Giving the URL to anyone allows them to access the note, too.

---

<div class="post-metadata">

**Author:** ![venmo](https://forum.dmz.rs/user_avatar/forum.dmz.rs/venmo/32/445_2.png) [@venmo](https://forum.dmz.rs/u/venmo)\
**Post date:** [September 1, 2025, 10:21pm UTC](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938/22 "2025-09-01T22:21:47Z")

</div>

legendo 😃

[Next page](https://forum.dmz.rs/t/llms-marketinski-receno-vestacka-inteligencija-nije/938.md?page=2)