I gave my messy photo archive a spirit guide — an offline, open-weight AI that learns your family, names your places, and finds any memory
---
title: I gave my messy photo archive a spirit guide — an offline, open-weight AI that learns your family, names your places, and finds any memory
published: true
tags: devchallenge, hf26challenge, python, ai
---
This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
Alebrije (JamesDBartlett3/Alebrije) is a local-first archive spirit-guide for your photos, videos, and audio clips. Alebrijes are the brightly colored creatures of Mexican folk art that guide souls where they need to go — this one guides you through thousands of memories that would otherwise sit in folders with names like DCIM_0947.
You point it at a messy directory. It then:
1. Meets your people. You introduce a friend or family member from a sample photo (alebrije person add Abuela --photo abuela.jpg), and Alebrije builds a private people book: the face is embedded locally by open face models (SCRFD detection + AdaFace recognition via onnxruntime) and described in words by the vision model — no cloud, no face database handed to a third party.
2. Recognizes them everywhere, and names every place. Every photo is scanned for faces; each becomes a 512-d vector stored locally. Faces close to someone in your people book are matched automatically, borderline ones are double-checked by Gemma, and recurring strangers cluster into "Unnamed" people you name with one rename. It also tags every photo and video with objects, clothing, animals, food, and the mood of the moment. If a photo carries GPS coordinates, it finds the closest point of interest (a park, museum, landmark — nearest by real haversine distance, via the OpenStreetMap Overpass API) and always appends the municipality chain: Civic Center Park — Denver, Colorado, USA, or with no landmark in range simply City of Golden, Jefferson County, Colorado, USA.
3. Remembers by feeling, not filename. Everything is embedded with an open model into a local SQLite index, so you can search in plain language — "dog on the beach at sunset" — sorted by relevance, newest, media type, or name.
Everything runs on your machine: an open-weight multimodal model (Gemma 3 4B via llama.cpp) looks at your media, open face models (SCRFD + AdaFace via onnxruntime) recognize your people, an open embedding model (nomic-embed-text) indexes it, and a Día de los Muertos–themed local web app — papel picado banner, marigold glow — serves the whole collection to any phone or laptop on your network. Nothing about your family ever leaves your hardware (the only outbound calls, and only for GPS-tagged files, are the two open-data OpenStreetMap geocoders, which receive coordinates, never images).
I built this for the #GhostInTheMachine theme the way Día de los Muertos itself treats it: the dead stay alive as long as their stories are told. A hard drive full of untagged photos is a room full of ancestors waiting to be introduced. Alebrije is the marigold path through it.
Demo
Everything below is from a live run on an ordinary 4-core desktop, CPU only:
!Alebrije web gallery
Six files went in named IMG_4031.png … IMG_4043.jpg. Alebrije tagged all six (captions, objects, clothing, mood), recognized and named a person across four of them from a single sample photo, and reverse-geocoded the two that carried GPS. Then, in plain language:
!Natural-language search
"dog on the beach at sunset" puts the three beach-dog photos on top — including one taken before the archive's newest party photos — because ranking is by meaning, not date.
!Memory detail
The CLI speaks the same language:
!CLI search
And it's the same app on a phone, served from your own desk over the LAN:
!Phone view
Code
JamesDBartlett3/Alebrije
Install and run (Python 3.10+ and pipx are the only prerequisites — Alebrije installs everything else):
pipx install git+https://github.com/JamesDBartlett3/Alebrije.git@v0.2.0
alebrije setup # downloads llama.cpp + the open-weight models, auto-detects your GPU
alebrije init ~/Pictures/family
alebrije up # starts the two local model servers
alebrije scan # index files, read EXIF dates + GPS
alebrije tag # vision-tag everything (CPU: ~1 min/photo, GPU: seconds)
alebrije serve # http://localhost:8799 — also on your LAN
alebrije update # later: checks GitHub releases, upgrades in place
No pipx? git clone + python alebrije.py <command> works straight from the repo — friendliest option on Windows.
How I Built It
- Vision + language: Gemma 3 4B IT (open weights, Gemma license) served by llama.cpp's llama-server with its multimodal projector. Every caption, object list, clothing item, animal, mood, person description, and gray-band identity double-check comes from this one local model.
- Embeddings & search: nomic-embed-text v1.5 (Apache-2.0), via llama.cpp's OpenAI-compatible embeddings endpoint. Vectors live as float32 blobs in SQLite; cosine ranking is a few lines of Python.
- Faces, without the cloud: v0.1 matched people by Gemma descriptions alone — we shipped it, watched it conflate lookalike relatives and miss people across decades, measured the failure, and rebuilt identity on real face embeddings. SCRFD-10GF detects every face (five landmarks each) and AdaFace ir101 — trained specifically to handle low-quality faces, i.e. old family photos — turns each aligned crop into a 512-d identity vector, both on onnxruntime (MIT) on CPU. Detection runs a letterboxed whole-image pass plus overlapping native-resolution tiles on large photos, so faces never get lost to downsampling; alignment crops are always warped from the full-resolution original. Matching uses measured thresholds (same person ≈ 0.56–0.95, different people ≈ 0.0), with Gemma confirming borderline cases and strangers auto-clustering into nameable "Unnamed" people.
- The app itself is nearly all Python standard library — a scanner (with a hand-rolled EXIF parser that pulls capture dates and GPS coordinates out of JPEG APP1 segments), a tagging pipeline, a people book, a vector index, a CLI, and a threaded web server with an embedded single-page UI. The only pip dependencies are onnxruntime and Pillow. No framework, no account.
- Fully self-installing, fully cross-platform. alebrije setup works on Linux, macOS, and Windows: it queries the llama.cpp GitHub releases API and downloads the prebuilt build matching your OS, CPU architecture, and accelerator — NVIDIA (nvidia-smi), AMD ROCm (rocm-smi), Vulkan (vulkaninfo), Apple Metal on any Mac, or plain CPU — then fetches the five open models (three GGUFs from Hugging Face, two face ONNXs from GitHub releases) with resume support. When a GPU is present, alebrije up launches the models with full layer offload (-ngl 999).
- Tests: 182 offline tests (alebrije selftest) cover the scanner (including a synthetic JPEG with hand-built EXIF + GPS binary), the face pipeline (letterbox/tiling math, SCRFD decoding, similarity alignment, threshold bands, clustering, v0.1-database migration), tag coercion, people matching, vector math, search sorting, the bootstrap asset picker for every OS/GPU combo, server lifecycle, the web API (real HTTP on an ephemeral port), and the CLI — all against fake model servers, no network or GPU required.
Why Does Open Innovation Matter?
Your family photos are the last data you'd ever hand to a stranger, and the first data that gets uploaded without thinking the moment you use a cloud "AI photo assistant." Open weights flip that: the same Gemma model a giant cloud product would run is running on your desk, inspectable, with your files never leaving the room.
Openness also made this project possible at all. llama.cpp's OpenAI-compatible server meant the whole app is one urllib call away from vision and embeddings; the GGUF ecosystem meant a 4B model runs on an ordinary 4-core CPU; permissive licenses meant everything here ships without negotiation; and OpenStreetMap's open geodata meant place names without a paid maps API. If the models were closed, an offline family-archive assistant simply couldn't exist — and if they were free-but-closed, it couldn't be audited, fine-tuned, or trusted with something as personal as a memorial archive.
The Spirit of It
The moment it landed: I fed it six files named IMG_40xx and asked, out loud, "dog on the beach at sunset" — and the first three results were exactly the three photos I'd have picked, one of them taken years before the others. A folder became a memory. That's the whole ofrenda: the tech is just the marigold petals; what they guide home is your own past.
Prize Categories
- Gemma (Featured) — Alebrije is built around Gemma 3 4B: every caption, tag, person description, and borderline identity double-check comes from the open-weight Gemma model running locally through llama.cpp.
EmbeddingGemma 2 vs Mac Spotlight: 30/30 vs 5/30, offline on my Mac
EmbeddingGemma 2, Google's new free embedding model, found the right file 30 times out of 30 on my Mac in October 2026, offline, in about 34 ms per question. macOS Spotlight found 5. I used it to build the face-search computer from the movies: type a name and every photo of that person lights up. From 5 clicks on one photo, it found a fictional family 114 times out of 121, with 0 wrong names. This is the written companion to my 6-minute video, with every number, how it works, what broke, and the code.
{% youtube ZELcWmxAOLA %}
The app and every benchmark: AnassKartit/embeddinggemma2-photo-sweep.
Key facts (quick answer)
| Fact | Measured result |
|---|---|
| Right file first, 30 questions, 62 files | EmbeddingGemma 2 30/30, file names 12/30, Spotlight 5/30 (11/30 keywords only) |
| Search speed | 62 files indexed in 5.0 s, about 34 ms per question, offline |
| Answers from your files (Gemma 4 E4B) | 11/11, about 2 s each, 6.5 GB peak memory |
| Find anyone by name (ArcFace, 5 clicks on 1 photo) | 114/121 family appearances, 0 wrong names; 118/121 after 2 more clicks |
| Photo cleanup, no training | 29/29 family photos kept, 1 of 77 junk let through |
| Speed per photo (70 image tokens, Mac GPU) | 85 ms (556 ms at the default 280 tokens) |
| Electricity bills: find, read, add up | 12/12 bills exact; one-prompt total was wrong |
| Vector database needed? | No: numpy, ChromaDB, sqlite-vec, pgvector all 29/30 |
| Cost | $0 on the Mac; the cloud comparison cost $0.19 for 948 calls |
TL;DR
- EmbeddingGemma 2 understood plain questions like "where's my latest electricity bill?": 30/30 vs Spotlight's 5/30.
- Gemma 4 E4B answered from the files it found: 11/11, about 2 s each, nothing left the Mac.
- Name 5 faces in one photo and search by name: 114/121 found, 0 wrong names, 0 names on photos of other people.
- "Max birthday" works too: the name filters first, then EmbeddingGemma 2 ranks the rest by meaning.
- The 6 cloud decision models that can see images sorted junk well, but every photo is uploaded.
- You do not need a vector database or LangChain for one person's files.
How does EmbeddingGemma 2 compare to Mac Spotlight?
It found the right file first 30 times out of 30; Spotlight found it 5 times.
!EmbeddingGemma 2 vs Mac Spotlight: right file first on 30 questions
The test folder is a fictional Downloads folder for one made-up person: 62 files, including three electricity bills from 2024, 2025 and 2026 with dull file names, a boarding pass, a lease, notes, installers, screenshots and receipts. I wrote the 30 questions and the right file for each before running anything.
| Search | Right file first (of 30) | Right file in top 5 |
|---|---|---|
| EmbeddingGemma 2 (offline) | 30 | 30 |
| File-name search | 12 | 15 |
| macOS Spotlight, keywords only (mdfind) | 11 | 12 |
| macOS Spotlight, the question as typed | 5 | 6 |
Why the gap: Spotlight matches words. EmbeddingGemma 2 turns each file and each question into 768 numbers that describe meaning, so "latest electricity bill" lands next to ebill_download.pdf even though no word matches. For "latest", the app re-sorts near ties by date.
Can it answer questions from my files?
Yes. This is RAG in plain words: EmbeddingGemma 2 finds the 3 best files, then a small model reads them and answers.
| Model on the Mac | Right answers | Median time | Peak memory |
|---|---|---|---|
| Gemma 4 E4B (mlx-community/gemma-4-E4B-it-qat-4bit) | 11/11 | 2.0 s | 6.5 GB |
| Qwen3 4B Instruct 2507 (4-bit) | 11/11 | 2.2 s | 3.0 GB |
| Qwen3.5 9B (4-bit) | 11/11 | 4.8 s | 5.9 GB |
Example: "how much was my latest electricity bill?" gave "$131.44" with the file name ebill_download.pdf.
For totals, do not let the model add. With all 12 monthly bills in one prompt, Gemma 4 E4B said $1,477.17, which is wrong. When it read one bill at a time into JSON and plain Python added them, all 12 totals were exact and so was the year ($1,741.64).
How does "find anyone by name" work?
You click 5 faces in one photo and type their names; the app then names every face it is sure about.
!Find anyone by name: the whole family search with 114 of 121 found
1. YuNet (OpenCV) finds every face and 5 points on it (eyes, nose, mouth corners).
2. ArcFace R50 turns each aligned face into 512 numbers. Two photos of the same person land close together, across ages and angles.
3. A face gets a name when it is at least 0.4 similar to the faces you named, and each name is used once per photo.
4. Search: type "Lily", "Max birthday" or "whole family". Names filter first, then EmbeddingGemma 2 ranks the rest.
!Search Lily: 19 photos, each with MATCH FOUND and a score
Measured on 112 WhatsApp-style files with a fictional family of 5 in 29 photos (121 face appearances):
| Setting | Found | Wrong names | Names on other people's photos |
|---|---|---|---|
| ArcFace, 0.4, 5 clicks on 1 photo | 114/121 | 0 | 0 |
| + 2 more clicks (Lily and Emma in a pool photo) | 118/121 (108/111 on photos it never learned from) | 0 | 0 |
| ArcFace, looser 0.3 | 121/121 | 0 | strangers named in 7 photos |
| SFace (the free-for-commercial model) | 115/121 on the benchmark | | |
The looser setting finds everyone but starts naming strangers, so the app stays at 0.4 and asks you for one more click instead.
For your own library, agent/people.py groups every face it finds, lets you name groups, cross out the faces that are not that person, hide groups you don't care about, and then shows the faces it is least sure about. Learning is examples, not retraining: every click is stored and used on the next search. Fingerprints and names stay in a folder on your Mac.
Can it clean the junk off a phone?
Yes, on a test phone with 158 photos squeezed like WhatsApp (1600 px, JPEG quality 70).
!The photo cleanup app running on a Mac
| Check | Result |
|---|---|
| Family photos kept, no training | 29/29 |
| Junk let through, no training | 1 of 77 |
| Speed at Google's phone setting (70 image tokens) | 85 ms per photo |
| One family photo + YuNet/SFace, strict setting | 97.4% of the family found, 0.8% of other families kept |
| You tag 20 photos, then it decides junk or keep | 96.9% (tag 40: 97.8%) |
| Best cloud decision models (Perplexity Decider, Mercury Decide) | 158/158, but every photo is uploaded |
The app never deletes anything: Sweep moves files into _Sweep/<pile>/ and Undo puts them back.
Do I need a vector database or LangChain?
No, not for one person's files.
| Store | Right file first (30) | At 50,000 vectors | True top 10 found |
|---|---|---|---|
| numpy, in memory (what the app uses) | 29 | 6 ms | 100% |
| ChromaDB 1.5.9 | 29 | 11 ms | 92% |
| sqlite-vec 0.1.6 | 29 | 95 ms | 100% |
| Postgres 17 + pgvector, HNSW index | 29 | 22 ms | 64% at default settings |
Chunking is 3 lines of Python: 700-character pieces, 100 characters overlapping, each piece starting with the file name. The full search is about 80 lines. LangChain, LangGraph or ADK would add dependencies and change nothing here. They start to make sense for multi-step agents, like "find every passport photo and move it".
If the library grows, the next step is saving the index in SQLite so a restart only embeds new files. Past a million pieces, sqlite-vec or LanceDB. For a multi-user cloud version, Postgres + pgvector, with the index tuned and measured first.
What broke?
- Other families kept as family. EmbeddingGemma 2 alone knows a photo is a family photo, not whose family, so other families' photos went into the family pile. Two small face models fixed it.
- Looser face matching names strangers. 0.3 found 121/121 but named strangers in 7 photos.
- One-prompt totals. The model added 12 bills wrong; code adds them right.
- Licence. ArcFace weights are non-commercial. YuNet (MIT) and SFace (Apache 2.0) are the commercial-safe pair.
My take
The surprise is how little you need. One free embedding model, one small reading model and 80 lines of Python beat the search built into every Mac, and nothing leaves the laptop. The face search is the part people ask about, and it works because it asks you for 5 clicks instead of guessing. Start with plain numpy and a folder; add a database when the numbers tell you to.
Sources
- Code, data and every result file: AnassKartit/embeddinggemma2-photo-sweep (REVIEW.md has the full method)
- Video, 13 October 2026: youtu.be
- InsightFace buffalo_l (ArcFace R50, non-commercial weights): deepinsight/insightface
- OpenCV Zoo YuNet and SFace: opencv/opencv_zoo
- All numbers measured by me on a MacBook in October 2026. All people in the test photos are fictional (made with Codex), except 40 stranger photos and 25 receipts from public research datasets, fetched by script and not shipped.
More tests like this every week: subscribe on YouTube.
What's the first thing you'd ask your own files?
Mentions
I gave my messy photo archive a spirit guide — an offline, open-weight AI that learns your family, names your places, and finds any memory --- title: I gave my messy photo archive a spirit guide — an offline, open-weight AI that learns your family, names your places, and finds any memory published: true tags: devchallenge, hf26challenge, python, ai --- This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine What I Built Alebrije (JamesDBartlett3/Alebrije) is a local-first archive spirit-guide for your photos, videos, and audio clips. Alebrijes are the brightly colored creatures of Mexican folk art that guide souls where they need to go — this one guides you through thousands of memories that would otherwise sit in folders with names like DCIM_0947. You point it at a messy directory. It then: 1. Meets your people. You introduce a friend or family member from a sample photo (alebrije person add Abuela --photo abuela.jpg), and Alebrije builds a private people book: the face is embedded locally by open face models (SCRFD detection + AdaFace recognition via onnxruntime) and described in words by the vision model — no cloud, no face database handed to a third party. 2. Recognizes them everywhere, and names every place. Every photo is scanned for faces; each becomes a 512-d vector stored locally. Faces close to someone in your people book are matched automatically, borderline ones are double-checked by Gemma, and recurring strangers cluster into "Unnamed" people you name with one rename. It also tags every photo and video with objects, clothing, animals, food, and the mood of the moment. If a photo carries GPS coordinates, it finds the closest point of interest (a park, museum, landmark — nearest by real haversine distance, via the OpenStreetMap Overpass API) and always appends the municipality chain: Civic Center Park — Denver, Colorado, USA, or with no landmark in range simply City of Golden, Jefferson County, Colorado, USA. 3. Remembers by feeling, not filename. Everything is embedded with an open model into a local SQLite index, so you can search in plain language — "dog on the beach at sunset" — sorted by relevance, newest, media type, or name. Everything runs on your machine: an open-weight multimodal model (Gemma 3 4B via llama.cpp) looks at your media, open face models (SCRFD + AdaFace via onnxruntime) recognize your people, an open embedding model (nomic-embed-text) indexes it, and a Día de los Muertos–themed local web app — papel picado banner, marigold glow — serves the whole collection to any phone or laptop on your network. Nothing about your family ever leaves your hardware (the only outbound calls, and only for GPS-tagged files, are the two open-data OpenStreetMap geocoders, which receive coordinates, never images). I built this for the #GhostInTheMachine theme the way Día de los Muertos itself treats it: the dead stay alive as long as their stories are told. A hard drive full of untagged photos is a room full of ancestors waiting to be introduced. Alebrije is the marigold path through it. Demo Everything below is from a live run on an ordinary 4-core desktop, CPU only: !Alebrije web gallery Six files went in named IMG_4031.png … IMG_4043.jpg. Alebrije tagged all six (captions, objects, clothing, mood), recognized and named a person across four of them from a single sample photo, and reverse-geocoded the two that carried GPS. Then, in plain language: !Natural-language search "dog on the beach at sunset" puts the three beach-dog photos on top — including one taken before the archive's newest party photos — because ranking is by meaning, not date. !Memory detail The CLI speaks the same language: !CLI search And it's the same app on a phone, served from your own desk over the LAN: !Phone view Code JamesDBartlett3/Alebrije Install and run (Python 3.10+ and pipx are the only prerequisites — Alebrije installs everything else): pipx install git+https://github.com/JamesDBartlett3/Alebrije.git@v0.2.0 alebrije setup # downloads llama.cpp + the open-weight models, auto-detects your GPU alebrije init ~/Pictures/family alebrije up # starts the two local model servers alebrije scan # index files, read EXIF dates + GPS alebrije tag # vision-tag everything (CPU: ~1 min/photo, GPU: seconds) alebrije serve # http://localhost:8799 — also on your LAN alebrije update # later: checks GitHub releases, upgrades in place No pipx? git clone + python alebrije.py <command> works straight from the repo — friendliest option on Windows. How I Built It - Vision + language: Gemma 3 4B IT (open weights, Gemma license) served by llama.cpp's llama-server with its multimodal projector. Every caption, object list, clothing item, animal, mood, person description, and gray-band identity double-check comes from this one local model. - Embeddings & search: nomic-embed-text v1.5 (Apache-2.0), via llama.cpp's OpenAI-compatible embeddings endpoint. Vectors live as float32 blobs in SQLite; cosine ranking is a few lines of Python. - Faces, without the cloud: v0.1 matched people by Gemma descriptions alone — we shipped it, watched it conflate lookalike relatives and miss people across decades, measured the failure, and rebuilt identity on real face embeddings. SCRFD-10GF detects every face (five landmarks each) and AdaFace ir101 — trained specifically to handle low-quality faces, i.e. old family photos — turns each aligned crop into a 512-d identity vector, both on onnxruntime (MIT) on CPU. Detection runs a letterboxed whole-image pass plus overlapping native-resolution tiles on large photos, so faces never get lost to downsampling; alignment crops are always warped from the full-resolution original. Matching uses measured thresholds (same person ≈ 0.56–0.95, different people ≈ 0.0), with Gemma confirming borderline cases and strangers auto-clustering into nameable "Unnamed" people. - The app itself is nearly all Python standard library — a scanner (with a hand-rolled EXIF parser that pulls capture dates and GPS coordinates out of JPEG APP1 segments), a tagging pipeline, a people book, a vector index, a CLI, and a threaded web server with an embedded single-page UI. The only pip dependencies are onnxruntime and Pillow. No framework, no account. - Fully self-installing, fully cross-platform. alebrije setup works on Linux, macOS, and Windows: it queries the llama.cpp GitHub releases API and downloads the prebuilt build matching your OS, CPU architecture, and accelerator — NVIDIA (nvidia-smi), AMD ROCm (rocm-smi), Vulkan (vulkaninfo), Apple Metal on any Mac, or plain CPU — then fetches the five open models (three GGUFs from Hugging Face, two face ONNXs from GitHub releases) with resume support. When a GPU is present, alebrije up launches the models with full layer offload (-ngl 999). - Tests: 182 offline tests (alebrije selftest) cover the scanner (including a synthetic JPEG with hand-built EXIF + GPS binary), the face pipeline (letterbox/tiling math, SCRFD decoding, similarity alignment, threshold bands, clustering, v0.1-database migration), tag coercion, people matching, vector math, search sorting, the bootstrap asset picker for every OS/GPU combo, server lifecycle, the web API (real HTTP on an ephemeral port), and the CLI — all against fake model servers, no network or GPU required. Why Does Open Innovation Matter? Your family photos are the last data you'd ever hand to a stranger, and the first data that gets uploaded without thinking the moment you use a cloud "AI photo assistant." Open weights flip that: the same Gemma model a giant cloud product would run is running on your desk, inspectable, with your files never leaving the room. Openness also made this project possible at all. llama.cpp's OpenAI-compatible server meant the whole app is one urllib call away from vision and embeddings; the GGUF ecosystem meant a 4B model runs on an ordinary 4-core CPU; permissive licenses meant everything here ships without negotiation; and OpenStreetMap's open geodata meant place names without a paid maps API. If the models were closed, an offline family-archive assistant simply couldn't exist — and if they were free-but-closed, it couldn't be audited, fine-tuned, or trusted with something as personal as a memorial archive. The Spirit of It The moment it landed: I fed it six files named IMG_40xx and asked, out loud, "dog on the beach at sunset" — and the first three results were exactly the three photos I'd have picked, one of them taken years before the others. A folder became a memory. That's the whole ofrenda: the tech is just the marigold petals; what they guide home is your own past. Prize Categories - Gemma (Featured) — Alebrije is built around Gemma 3 4B: every caption, tag, person description, and borderline identity double-check comes from the open-weight Gemma model running locally through llama.cpp.
@Rafalkow @OniszczukAssoc Chatgpt ci soft napisze w 30 min. Powiedz mu, zeby użył deepinsight/insightface
EmbeddingGemma 2 vs Mac Spotlight: 30/30 vs 5/30, offline on my Mac EmbeddingGemma 2, Google's new free embedding model, found the right file 30 times out of 30 on my Mac in October 2026, offline, in about 34 ms per question. macOS Spotlight found 5. I used it to build the face-search computer from the movies: type a name and every photo of that person lights up. From 5 clicks on one photo, it found a fictional family 114 times out of 121, with 0 wrong names. This is the written companion to my 6-minute video, with every number, how it works, what broke, and the code. {% youtube ZELcWmxAOLA %} The app and every benchmark: AnassKartit/embeddinggemma2-photo-sweep. Key facts (quick answer) | Fact | Measured result | |---|---| | Right file first, 30 questions, 62 files | EmbeddingGemma 2 30/30, file names 12/30, Spotlight 5/30 (11/30 keywords only) | | Search speed | 62 files indexed in 5.0 s, about 34 ms per question, offline | | Answers from your files (Gemma 4 E4B) | 11/11, about 2 s each, 6.5 GB peak memory | | Find anyone by name (ArcFace, 5 clicks on 1 photo) | 114/121 family appearances, 0 wrong names; 118/121 after 2 more clicks | | Photo cleanup, no training | 29/29 family photos kept, 1 of 77 junk let through | | Speed per photo (70 image tokens, Mac GPU) | 85 ms (556 ms at the default 280 tokens) | | Electricity bills: find, read, add up | 12/12 bills exact; one-prompt total was wrong | | Vector database needed? | No: numpy, ChromaDB, sqlite-vec, pgvector all 29/30 | | Cost | $0 on the Mac; the cloud comparison cost $0.19 for 948 calls | TL;DR - EmbeddingGemma 2 understood plain questions like "where's my latest electricity bill?": 30/30 vs Spotlight's 5/30. - Gemma 4 E4B answered from the files it found: 11/11, about 2 s each, nothing left the Mac. - Name 5 faces in one photo and search by name: 114/121 found, 0 wrong names, 0 names on photos of other people. - "Max birthday" works too: the name filters first, then EmbeddingGemma 2 ranks the rest by meaning. - The 6 cloud decision models that can see images sorted junk well, but every photo is uploaded. - You do not need a vector database or LangChain for one person's files. How does EmbeddingGemma 2 compare to Mac Spotlight? It found the right file first 30 times out of 30; Spotlight found it 5 times. !EmbeddingGemma 2 vs Mac Spotlight: right file first on 30 questions The test folder is a fictional Downloads folder for one made-up person: 62 files, including three electricity bills from 2024, 2025 and 2026 with dull file names, a boarding pass, a lease, notes, installers, screenshots and receipts. I wrote the 30 questions and the right file for each before running anything. | Search | Right file first (of 30) | Right file in top 5 | |---|---|---| | EmbeddingGemma 2 (offline) | 30 | 30 | | File-name search | 12 | 15 | | macOS Spotlight, keywords only (mdfind) | 11 | 12 | | macOS Spotlight, the question as typed | 5 | 6 | Why the gap: Spotlight matches words. EmbeddingGemma 2 turns each file and each question into 768 numbers that describe meaning, so "latest electricity bill" lands next to ebill_download.pdf even though no word matches. For "latest", the app re-sorts near ties by date. Can it answer questions from my files? Yes. This is RAG in plain words: EmbeddingGemma 2 finds the 3 best files, then a small model reads them and answers. | Model on the Mac | Right answers | Median time | Peak memory | |---|---|---|---| | Gemma 4 E4B (mlx-community/gemma-4-E4B-it-qat-4bit) | 11/11 | 2.0 s | 6.5 GB | | Qwen3 4B Instruct 2507 (4-bit) | 11/11 | 2.2 s | 3.0 GB | | Qwen3.5 9B (4-bit) | 11/11 | 4.8 s | 5.9 GB | Example: "how much was my latest electricity bill?" gave "$131.44" with the file name ebill_download.pdf. For totals, do not let the model add. With all 12 monthly bills in one prompt, Gemma 4 E4B said $1,477.17, which is wrong. When it read one bill at a time into JSON and plain Python added them, all 12 totals were exact and so was the year ($1,741.64). How does "find anyone by name" work? You click 5 faces in one photo and type their names; the app then names every face it is sure about. !Find anyone by name: the whole family search with 114 of 121 found 1. YuNet (OpenCV) finds every face and 5 points on it (eyes, nose, mouth corners). 2. ArcFace R50 turns each aligned face into 512 numbers. Two photos of the same person land close together, across ages and angles. 3. A face gets a name when it is at least 0.4 similar to the faces you named, and each name is used once per photo. 4. Search: type "Lily", "Max birthday" or "whole family". Names filter first, then EmbeddingGemma 2 ranks the rest. !Search Lily: 19 photos, each with MATCH FOUND and a score Measured on 112 WhatsApp-style files with a fictional family of 5 in 29 photos (121 face appearances): | Setting | Found | Wrong names | Names on other people's photos | |---|---|---|---| | ArcFace, 0.4, 5 clicks on 1 photo | 114/121 | 0 | 0 | | + 2 more clicks (Lily and Emma in a pool photo) | 118/121 (108/111 on photos it never learned from) | 0 | 0 | | ArcFace, looser 0.3 | 121/121 | 0 | strangers named in 7 photos | | SFace (the free-for-commercial model) | 115/121 on the benchmark | | | The looser setting finds everyone but starts naming strangers, so the app stays at 0.4 and asks you for one more click instead. For your own library, agent/people.py groups every face it finds, lets you name groups, cross out the faces that are not that person, hide groups you don't care about, and then shows the faces it is least sure about. Learning is examples, not retraining: every click is stored and used on the next search. Fingerprints and names stay in a folder on your Mac. Can it clean the junk off a phone? Yes, on a test phone with 158 photos squeezed like WhatsApp (1600 px, JPEG quality 70). !The photo cleanup app running on a Mac | Check | Result | |---|---| | Family photos kept, no training | 29/29 | | Junk let through, no training | 1 of 77 | | Speed at Google's phone setting (70 image tokens) | 85 ms per photo | | One family photo + YuNet/SFace, strict setting | 97.4% of the family found, 0.8% of other families kept | | You tag 20 photos, then it decides junk or keep | 96.9% (tag 40: 97.8%) | | Best cloud decision models (Perplexity Decider, Mercury Decide) | 158/158, but every photo is uploaded | The app never deletes anything: Sweep moves files into _Sweep/<pile>/ and Undo puts them back. Do I need a vector database or LangChain? No, not for one person's files. | Store | Right file first (30) | At 50,000 vectors | True top 10 found | |---|---|---|---| | numpy, in memory (what the app uses) | 29 | 6 ms | 100% | | ChromaDB 1.5.9 | 29 | 11 ms | 92% | | sqlite-vec 0.1.6 | 29 | 95 ms | 100% | | Postgres 17 + pgvector, HNSW index | 29 | 22 ms | 64% at default settings | Chunking is 3 lines of Python: 700-character pieces, 100 characters overlapping, each piece starting with the file name. The full search is about 80 lines. LangChain, LangGraph or ADK would add dependencies and change nothing here. They start to make sense for multi-step agents, like "find every passport photo and move it". If the library grows, the next step is saving the index in SQLite so a restart only embeds new files. Past a million pieces, sqlite-vec or LanceDB. For a multi-user cloud version, Postgres + pgvector, with the index tuned and measured first. What broke? - Other families kept as family. EmbeddingGemma 2 alone knows a photo is a family photo, not whose family, so other families' photos went into the family pile. Two small face models fixed it. - Looser face matching names strangers. 0.3 found 121/121 but named strangers in 7 photos. - One-prompt totals. The model added 12 bills wrong; code adds them right. - Licence. ArcFace weights are non-commercial. YuNet (MIT) and SFace (Apache 2.0) are the commercial-safe pair. My take The surprise is how little you need. One free embedding model, one small reading model and 80 lines of Python beat the search built into every Mac, and nothing leaves the laptop. The face search is the part people ask about, and it works because it asks you for 5 clicks instead of guessing. Start with plain numpy and a folder; add a database when the numbers tell you to. Sources - Code, data and every result file: AnassKartit/embeddinggemma2-photo-sweep (REVIEW.md has the full method) - Video, 13 October 2026: youtu.be - InsightFace buffalo_l (ArcFace R50, non-commercial weights): deepinsight/insightface - OpenCV Zoo YuNet and SFace: opencv/opencv_zoo - All numbers measured by me on a MacBook in October 2026. All people in the test photos are fictional (made with Codex), except 40 stranger photos and 25 receipts from public research datasets, fetched by script and not shipped. More tests like this every week: subscribe on YouTube. What's the first thing you'd ask your own files?