Local AI: Run & Build Your Own (Free)

🚀 Quick start: try it in 30 minutes

Do these in order. Stop whenever you've had enough fun for one day.

1. Check what you have. On the Mac: Apple menu → About This Mac. Note the Chip (Apple M1 or newer, or Intel) and the Memory (8 GB, 16 GB, 24 GB…). On the iPhone: Settings → General → About → Model Name.
2. iPhone: install the free app Locally AI (needs iOS 18.1+) or PocketPal AI (older iPhones too). Download one small model on Wi-Fi.
3. Prove it's offline: turn on Airplane Mode and ask it a question. If it answers, the AI is running on your phone. 🎉
4. Mac: download Ollama, drag it into Applications and open it once. (Needs macOS 14 Sonoma or newer.)
5. Open Terminal (⌘-Space, type Terminal) and run the command for your memory size below. The first run downloads the model (1–2 GB).
6. Type a question at the >>> prompt. Type /bye to quit.
7. Liked it? Start Day 1 of the course below.

ollama run llama3.2:1b

ollama run llama3.2

📱 On your iPhone

These apps download a model once, then run it fully on the phone. Pick one to start; Locally AI or PocketPal AI are the easiest.

Which models fit on a phone?

• Small models only: about 1 to 4 billion parameters (“1B–4B”), downloads of roughly 0.7–3 GB.
• iPhone 15 Pro, any 16 or 17 (about 8 GB memory): try Gemma 3 1B, Llama 3.2 3B, Qwen 3.5 4B or Phi-4 Mini.
• Older iPhones (6 GB or less): stick to 1B–2B models such as Gemma 3 1B, Llama 3.2 1B or Qwen3 1.7B.
• Expect limits: slower replies than cloud apps, short memory of the conversation, no live web knowledge, the phone gets warm and the battery drains faster. If the app crashes, pick a smaller model.

💻 On your Mac

Ollama is the easiest free starting point and it's what the course uses. Your Mac's memory decides which models run well.

Install Ollama and run your first model

1. Go to ollama.com/download and download the Mac app.
2. Open the download, drag Ollama into Applications, then open it. Allow it to add its command-line tool if asked.
3. Open Terminal and run ollama run llama3.2:1b. Wait for the one-time download.
4. Chat at the >>> prompt. /bye quits. Run ollama list to see what you've downloaded.

Pick a model by memory (Apple M1 or newer)

Handy Ollama commands

ollama list            # models you have
ollama pull qwen3:4b   # download without chatting
ollama ps              # what's running right now
ollama rm qwen3:4b     # delete a model to free space

Intel Mac?

• Ollama works on Intel Macs running macOS 14 Sonoma or newer, but only on the processor (no graphics speed-up), so it's slow. Stick to 1B–3B models.
• LM Studio and Apple MLX don't support Intel Macs. For the training days, use the free Google Colab option instead.

Other free Mac tools (optional)

🛠️ Build and train your own

Yes, you can build one yourself. Here's what's realistic at home, for free.

Python or C++?

• Start with Python. Almost every tutorial, library and course uses it, and it's friendly for beginners.
• C/C++ comes later, if you want speed or to see the guts. Read llama.cpp (running models) or Karpathy's llm.c, which trains GPT-2 in plain C. Its simple CPU version is about 1,000 lines in one file, train_gpt2.c.

Words you'll see

• Model / weights: the big file of numbers that is the AI.
• Parameters (1B, 3B, 8B): how many numbers it has. More is usually smarter but needs more memory.
• Quantized / GGUF / 4-bit: squeezed copies of a model so it fits on normal hardware.
• Token: a word piece. Speed is measured in tokens per second.
• Context window: how much text it can keep in mind at once.
• Fine-tuning / LoRA: cheaply adjusting a trained model with your own examples.
• Embedding: text turned into a list of numbers so similar ideas can be found.

🎓 7-day course: about an hour a day

Each day builds on the last. Tick the box when you're done; your progress is saved in this browser only.

FROM llama3.2:1b
PARAMETER temperature 0.3
SYSTEM """You are a patient tutor. Explain things in short, plain steps
and end with one quick question to check my understanding."""

# chat.py: a tiny chat program that talks to your local model
import ollama

MODEL = "llama3.2:1b"   # use any model you have pulled
history = []
print("Say something (press Enter on a blank line to quit).")

while (text := input("\nYou: ").strip()):
    history.append({"role": "user", "content": text})
    reply = ollama.chat(model=MODEL, messages=history).message.content
    history.append({"role": "assistant", "content": reply})
    print("AI:", reply)

# notes_chat.py: ask questions about your own notes, 100% on your computer
import glob, math, ollama

CHAT_MODEL  = "llama3.2:1b"        # or llama3.2 / qwen3:4b with 16 GB+
EMBED_MODEL = "nomic-embed-text"   # turns text into numbers for searching

def chunks(text, size=800):
    buf = ""
    for para in text.split("\n\n"):
        if buf and len(buf) + len(para) > size:
            yield buf
            buf = ""
        buf += para + "\n\n"
    if buf.strip():
        yield buf

def similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))

pieces = []
for path in glob.glob("notes/*.md") + glob.glob("notes/*.txt"):
    for c in chunks(open(path, encoding="utf-8").read()):
        pieces.append((path, c))
vectors = ollama.embed(model=EMBED_MODEL, input=[c for _, c in pieces]).embeddings
print(f"Read {len(pieces)} pieces of your notes. Ask a question (blank line quits).")

while (question := input("\nYou: ").strip()):
    qv = ollama.embed(model=EMBED_MODEL, input=question).embeddings[0]
    ranked = sorted(zip(vectors, pieces), key=lambda vp: similarity(qv, vp[0]), reverse=True)
    best = "\n---\n".join(f"[{path}]\n{text}" for _, (path, text) in ranked[:3])
    messages = [
        {"role": "system", "content": "Answer only from these notes and name the note you used. "
                                      "If the notes don't say, reply: not in my notes.\n\n" + best},
        {"role": "user", "content": question},
    ]
    print("AI:", ollama.chat(model=CHAT_MODEL, messages=messages).message.content)

python train.py config/train_shakespeare_char.py --device=cpu --compile=False --eval_iters=20 --log_interval=1 --block_size=64 --batch_size=12 --n_layer=4 --n_head=4 --n_embd=128 --max_iters=2000 --lr_decay_iters=2000 --dropout=0.0
python sample.py --out_dir=out-shakespeare-char --device=cpu

{"messages": [{"role": "user", "content": "What is a sea breeze?"}, {"role": "assistant", "content": "Breezy answer: land heats faster than water, warm air rises, and cool ocean air rushes in to replace it. 🌊"}]}
{"messages": [{"role": "user", "content": "Why do kites fly?"}, {"role": "assistant", "content": "Breezy answer: moving air pushes up on the tilted kite while the string holds it in place. 🪁"}]}

🔒 Privacy and safety, in plain words

Local AI is private by design, but a few habits keep it that way.

• Download only from trusted places: the App Store, ollama.com, lmstudio.ai, and well-known Hugging Face publishers (Google, Meta, Qwen, Microsoft, ggml-org, unsloth, mlx-community).
• Check the seller name in the App Store before tapping Get. Copycat apps exist.
• Watch for cloud switches. Some apps can also use online models or web search. Leave those off if you want everything to stay on your device.
• Small models make things up. Double-check facts, numbers, medical, legal and money answers.
• Be careful running code from the internet, including code an AI writes for you. Read it first, and keep projects inside their own folder and Python environment.
• Models are big. Delete ones you don't use (ollama rm name) to free disk space.
• Check the license before using a model in anything you sell or offer to others. Most are free for personal use.

📚 Sources and links

Everything above was checked against these pages.

• Ollama download
• Ollama macOS requirements
• Ollama model library
• LM Studio system requirements
• Locally AI
• PocketPal AI (GitHub)
• Google AI Edge Gallery (GitHub)
• Apple: Use Apple Intelligence in Shortcuts
• llama.cpp
• Apple MLX LM
• nanoGPT
• nanochat
• llm.c
• Unsloth free notebooks
• Google Colab