Arthur-Ficial

处理文档的办公族，你每月为云端付费，却不敢传敏感文件，想在1台电脑切换多个本地大模型却卡住了。 它解决本地部署难、切换麻烦的问题，模型跑在你的显卡上。 分3步衔接：引擎跑模型，接口接模型，界面切模型。 第 1 步 · 本地推理引擎：vllm-project/vllm —— 把模型加载到显卡上高效推理。 第 2 步 · 模型服务接口：Arthur-Ficial/apfel —— 把引擎包装成统一接口。 第 3 步 · 本地对话界面：huggingface/chat-ui —— 提供网页对话，随时切换模型。 适合有显卡想离线用的人 / 不适合只有核显的人。 值得先跑通1个模型再加第二个，注意显存。 你最想先本地跑哪个模型处理文档？ #工具推荐 #开源项目

[ Video ] Testing Apple Intelligence on my 16GB MacBook Pro This is the 8th episode in the series where I’m testing a local AI model that runs on a 16GB MacBook Pro. Today I’m doing something special: I picked the model that Apple ships with Apple Intelligence (often called the Apple Foundation Model or AFM). It is already on your machine (Mac, iPhone, iPad) once you turn Apple Intelligence on. Apple does not distribute it the usual way (no Hugging Face weights, traditionally opaque), and there was no straightforward chat interface for it until recently. I used an open-source tool apfel to get a normal chat interface: Arthur-Ficial/apfel It installs easily (Homebrew) and it provides a CLI and an OpenAI-compatible local server. No extra model download is needed. I Ran The Two Standard Tests First, the short question “What is a dolphin?” The answer appeared almost immediately. Across all the models I’ve run on this 16GB MacBook Pro, this was the fastest generation. Second, the coding task: generate a small utility that counts words in files inside a folder. It defaulted to Python and produced a working script very quickly. Because this is a rather unusual mode, a few technical notes. According to what I could find online, it is roughly a 3 billion parameter model. The space it occupies on disk is about 8 GB (Apple is opaque about the exact breakdown). Also, there is one caveat: some requests can be routed to Private Cloud Compute, so it is not always possible to know with high accuracy which ones stay fully on-device. On newer hardware (M3 and later) Apple can use a larger on-device variant that takes more space (up to around 12 GB). Even the smaller model was impressive for speed and basic usability on a 16GB machine. If you have other models that fit in 16GB and want them tested the same way, leave a comment. I read them. And if you like this kind of content, like, share and subscribe to my YouTube channel!

I asked Claude to build something nice on Apple Intelligence (i.e. all local inference) and it made a nice mystery app in Swift, a game to solve a crime! No API keys, all local; built on top of Arthur-Ficial/apfel See the Pawprint repo: y-a-v-a/pawprints t.co

here's the repository if anybody wants to dig deeper - Arthur-Ficial/apfel