ageitgey

Group Photos by Face in Python: One Album per Person --- title: "Group Photos by Face in Python: One Album per Person" published: true tags: python, api, computervision, machinelearning canonical_url: ai-engine.net cover_image: ai-engine.net --- Phone galleries, family archives, a photographer's backlog: where are all the photos of this person? Here is how to group photos by face in Python: detect every face, match it in a face repository, and sort the library into one album per person. No model to download, no dlib to compile. Every number below comes from a live run on 24 photos from two outings, a camping trip and a beach picnic. > Want to sort your own library? Get a key for the Face Analyzer API. Step 1: create a face repository import io import os import shutil import requests from concurrent.futures import ThreadPoolExecutor from PIL import Image, ImageOps BASE = "https://faceanalyzer-ai.p.rapidapi.com" HEADERS = { "x-rapidapi-host": "faceanalyzer-ai.p.rapidapi.com", "x-rapidapi-key": "YOUR_API_KEY", } LIBRARY = "my-photo-library" MIN_SIMILARITY = 90 r = requests.post( f"{BASE}/create-facial-repository", headers=HEADERS, data={"repository_id": LIBRARY}, ) print(r.json()) Step 2: detect and crop every face def detect_faces(path): """Return the photo path and the bounding box of every face in it.""" with open(path, "rb") as f: r = requests.post(f"{BASE}/faceanalysis", headers=HEADERS, files={"image": f}) return path, [face["boundingBox"] for face in r.json()["body"]["faces"]] def crop_face(path, box, margin=0.4): """Cut one face out of the photo, with a margin, as JPEG bytes.""" im = ImageOps.exif_transpose(Image.open(path)).convert("RGB") w, h = im.size x1, y1 = box["topLeft"]["x"] * w, box["topLeft"]["y"] * h x2, y2 = box["bottomRight"]["x"] * w, box["bottomRight"]["y"] * h mx, my = (x2 - x1) * margin, (y2 - y1) * margin face = im.crop(( int(max(0, x1 - mx)), int(max(0, y1 - my)), int(min(w, x2 + mx)), int(min(h, y2 + my)), )) buf = io.BytesIO() face.save(buf, "JPEG", quality=92) return buf.getvalue() The repository search looks at the main face of the image it receives, so each face is sent as its own crop. Step 3: match each face or start a new person def who_is(face_jpg): """Return the label of the closest known person, or None for a new face.""" r = requests.post( f"{BASE}/search-face-in-repository", headers=HEADERS, files={"image": ("face.jpg", face_jpg)}, data={"repository_id": LIBRARY}, ) matches = [ m for m in r.json()["body"].get("FaceMatches", []) if m["Similarity"] >= MIN_SIMILARITY ] if not matches: return None return max(matches, key=lambda m: m["Similarity"])["ExternalImageId"] def remember(face_jpg, person): """Save the face under the person's label, so later photos match it too.""" requests.post( f"{BASE}/save-face-in-repository", headers=HEADERS, files={"image": ("face.jpg", face_jpg)}, data={"repository_id": LIBRARY, "external_id": person, "max_faces": "1"}, ) people = {} # person label -> names of the photos they appear in def sort_photos(folder): """Detect faces in parallel, then match each face to a person in order.""" paths = sorted(os.path.join(folder, n) for n in os.listdir(folder)) with ThreadPoolExecutor(max_workers=4) as pool: detected = list(pool.map(detect_faces, paths)) for path, boxes in detected: for box in boxes: face_jpg = crop_face(path, box) person = who_is(face_jpg) if person is None: person = f"person_{len(people) + 1}" people[person] = set() remember(face_jpg, person) people[person].add(path) return sum(len(boxes) for _, boxes in detected) faces = sort_photos("photos") print(faces, "faces,", len(people), "face groups") # 54 faces, 16 face groups Step 4: one album per person MIN_PHOTOS = 3 def build_albums(out_dir="albums"): """Copy each photo into the album of every person in it.""" shutil.rmtree(out_dir, ignore_errors=True) for person, paths in people.items(): album = person if len(paths) >= MIN_PHOTOS else "unsorted" os.makedirs(os.path.join(out_dir, album), exist_ok=True) for path in paths: shutil.copy(path, os.path.join(out_dir, album, os.path.basename(path))) for album in sorted(os.listdir(out_dir)): print(album, len(os.listdir(os.path.join(out_dir, album))), "photos") build_albums() person_10 6 photos person_12 4 photos person_13 4 photos person_14 5 photos person_2 6 photos person_3 6 photos person_5 7 photos person_6 5 photos person_9 3 photos unsorted 7 photos Nine albums for the nine people: the two campers and the seven picnic friends. Every album contains a single person, checked by eye. Faces seen once or twice (a head turned away, a passer-by out of focus) go to unsorted. !Nine person albums built by a face recognition API from a 24-photo library Step 5: new photos join the right album before = {person: set(paths) for person, paths in people.items()} sort_photos("new_photos") for person, paths in people.items(): for path in sorted(paths - before.get(person, set())): print(os.path.basename(path), "->", person) build_albums() IMG_5064932.jpg -> person_3 IMG_8021122.jpg -> person_9 Under 3 seconds, no re-clustering. Why not face_recognition + DBSCAN? | | face_recognition + DBSCAN | Face Analyzer API | |---|---|---| | Install | dlib source build (CMake, C++) | requests and Pillow | | Last release | 1.3.0, February 2020 | hosted | | New photos | re-cluster, labels can change | one search per face, stable labels | | Tuning | eps and min_samples | one similarity threshold | | Children | README warns it mixes them up at 0.6 | same code for every face | Production tips - One repository per user, deleted when they turn the feature off - Face grouping is biometric processing under GDPR Article 9: make it opt-in - Let users merge and rename albums - Run the first pass as a background job Full guide: group photos by face on ai-engine.net. Sources: face_recognition, dlib on PyPI, GDPR Article 9, test photos from Pexels (Kamaji Ogino, Thirdman). 👉 Grab a key from the Face Analyzer API page and sort your own photos.