LLaMA-Factory

A model as the judge: build preference pairs with an API, then train with DPO on a rented GPU --- title: "A model as the judge: build preference pairs with an API, then train with DPO on a rented GPU" published: true description: "Have DeepSeek write two answers to each of your questions and pick the one that follows your principles better, in both orders, keep only consistent picks, and train a LoRA adapter with DPO in LLaMA-Factory on one rented GPU." tags: ai, python, machinelearning, tutorial cover_image: dev-to-uploads.s3.us-east-2.amazonaws.com --- Short answer: write a short list of principles and a set of questions, have an API model write two answers to each question, have it pick the answer that follows your principles better (twice, once in each order), keep only the pairs where both picks agree, and train a LoRA adapter on those chosen and rejected pairs with DPO in LLaMA-Factory on a rented GPU. Generate the pairs on your own machine; rent the GPU only for training; copy the adapter off, then destroy the instance. Disclosure: I work on CloudGPU. The endpoint (https://cloudgpu.app/v1) and the GPU rentals below are ours. Each step ends with a "What to check" line, because details (versions, template names, model access) change. Where the idea comes from, and what this guide does not claim Using a model's judgments as the training signal is usually called RL from AI Feedback (RLAIF). One public paper on it, "RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback" (Lee et al., arXiv, submitted 2023-09-01: paper), describes training a reward model on preferences generated by an off-the-shelf LLM and then using reinforcement learning. This guide keeps only the "a model states a preference between two answers" idea and skips the rest: it trains directly on the pairs with Direct Preference Optimization (paper), using LLaMA-Factory. That makes this a small, single-developer adaptation. It is our own construction, not a result from those sources. A model's pick can be wrong, and we make no claim that the adapter you train will be better than the base model. That is why the last step is a comparison you read yourself, with a rule for when to throw the round away. Read these things first 1. DeepSeek's terms. The answers and the picks in this guide come from DeepSeek, and this guide covers DeepSeek only. DeepSeek's Terms of Use (Last Update: March 27, 2026) say in section 4.2(3) that you may apply the Outputs of the Services to "a wide range of use cases, including personal use, academic research, derivative product development, training other models (such as model distillation), etc., as long as such usage is legal and adhere to these Terms." Read the terms yourself. This guide does not cover any other provider, and it does not tell you what any other provider's terms allow; if you want to use another model's outputs for training, read that provider's own terms first. 2. Labelling. Section 3.1 of the same terms says that if you publish or disseminate outputs, you must clearly indicate that the output content is generated by artificial intelligence. The chosen and rejected answers are outputs. If you publish the pairs, a model card or examples from them, label them as AI-generated; the script below writes a DATASET_NOTE.md you can keep next to the file. 3. Your data. The questions you write must be ones you have the right to use. CloudGPU's acceptable use policy (section 1.8) prohibits training on datasets you do not have rights to use. Also read the terms. 4. The base model's license. Each base model below has its own license. Open the link, read it, and confirm for yourself that your use is allowed. What you need - Python 3 and the openai package on your own machine: pip install openai. - A CloudGPU account with balance, and an API key (the API page has a "Get your API key" link). Keep it in an environment variable, never in code: export CLOUDGPU_API_KEY="paste-your-key-here" - GPU instances are billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance (terms). Rates are on the pricing page. Payment is by card or USDT (payments). - A Hugging Face account and a read token if you choose Llama or Gemma (both are gated, see the table below). What to check: python3 -c "import os; print(bool(os.environ.get('CLOUDGPU_API_KEY')))" prints True in the shell where you will run the scripts, and python3 -c "import openai" raises no error. API calls are billed per token from the same balance as GPU rentals; the current rate for every model is in the table on the API page. This post does not repeat rates. Step 1: write the principles and the questions (on your own machine) Create three plain-text files in one folder. principles.txt is a short list, in your own words, of what a better answer looks like for your use case. An example to replace with yours: 1. Answer only what the question asks. 2. If the answer depends on information you were not given, say what is missing instead of guessing. 3. Do not invent names, numbers, links or quotes. 4. Keep the answer under 150 words unless the question asks for more. prompts.txt is the training questions, one per line. Start with ten lines to test the script, then add more; the more varied the questions are, the more the judge has to compare. heldout.txt is 20 or so questions in the same style that you will not use for training. Write them separately, and do not copy lines between the files. What to check: wc -l prompts.txt heldout.txt shows the line counts you expect, and no line of heldout.txt appears in prompts.txt (grep -x -F -f heldout.txt prompts.txt prints nothing). Step 2: write two answers, let the model pick, keep consistent picks For each question the script asks deepseek-v4.1-flash for two answers at a high temperature, then asks it twice which answer follows your principles better: once with the answers in one order and once in the other. A pair is kept only if both picks choose the same answer. Judges can lean towards whichever answer comes first, or towards the longer one; the two orders and the length count the script prints are our own suggestions for spotting that, not claims from a source. The model ID comes from the live model table on the API page (checked 2026-10-27); the same table lists deepseek-v4-flash and deepseek-v4-pro. The OpenAI SDK only needs base_url and the key changed. # make_pairs.py import json, os, sys, time from concurrent.futures import ThreadPoolExecutor from pathlib import Path from openai import OpenAI client = OpenAI(base_url="https://cloudgpu.app/v1", api_key=os.environ["CLOUDGPU_API_KEY"]) MODEL = "deepseek-v4.1-flash" PRINCIPLES = Path("principles.txt").read_text(encoding="utf-8").strip() JUDGE = """You compare two answers to the same question. Judge them ONLY against these principles: {principles} Question: {question} Answer A: {a} Answer B: {b} Reply with exactly one letter: A if Answer A follows the principles better, B if Answer B follows them better.""" def chat(prompt, temperature): for attempt in range(3): try: resp = client.chat.completions.create( model=MODEL, temperature=temperature, messages=[{"role": "user", "content": prompt}], ) return resp.choices[0].message.content.strip() except Exception as err: # network error, rate limit print(f"attempt {attempt + 1} failed: {err}", file=sys.stderr) time.sleep(2 ** attempt) return None def verdict(question, first, second): text = chat(JUDGE.format(principles=PRINCIPLES, question=question, a=first, b=second), 0) letter = (text or "").strip().strip(".*").upper() return letter if letter in ("A", "B") else None def make_pair(question): one = chat(question, 1.0) two = chat(question, 1.0) if not one or not two or one == two: return None v1 = verdict(question, one, two) # `one` is shown first v2 = verdict(question, two, one) # `two` is shown first if v1 is None or v2 is None: return None winner1 = "one" if v1 == "A" else "two" winner2 = "two" if v2 == "A" else "one" if winner1 != winner2: # the judge changed its mind when the order changed return None chosen, rejected = (one, two) if winner1 == "one" else (two, one) return {"instruction": question, "input": "", "chosen": chosen, "rejected": rejected} if __name__ == "__main__": questions = [q.strip() for q in Path("prompts.txt").read_text(encoding="utf-8").splitlines() if q.strip()] with ThreadPoolExecutor(max_workers=4) as pool: results = list(pool.map(make_pair, questions)) pairs = [p for p in results if p and max(len(p["chosen"]), len(p["rejected"])) <= 3000] Path("dpo_pairs.json").write_text(json.dumps(pairs, ensure_ascii=False, indent=2), encoding="utf-8") Path("DATASET_NOTE.md").write_text( "AI-generated data: the answers in dpo_pairs.json were generated, and the preferences between " "them chosen, with deepseek-v4.1-flash from questions written by the dataset author. " "Review before use.\n", encoding="utf-8") longer = sum(len(p["chosen"]) > len(p["rejected"]) for p in pairs) print("questions:", len(questions), "pairs kept:", len(pairs), "chosen is the longer answer:", longer) Run it first on the ten-line prompts.txt: python3 make_pairs.py. Each question takes four API calls (two answers, two picks), so a long prompts.txt makes many calls; lower max_workers if you see rate-limit errors. What to check: dpo_pairs.json and DATASET_NOTE.md exist, and the script prints how many pairs it kept and how many times the chosen answer was the longer one. If the chosen answer is the longer one in nearly every pair, the judge may be choosing length, not your principles: tighten the principles, or drop pairs where the lengths differ a lot. If it keeps very few pairs, the two orders disagree often; read why in a few cases before you change anything. A 401 means the key variable is not set in this shell. Step 3: read the pairs yourself The format is LLaMA-Factory's preference format: per its data preparation docs, a preference dataset needs a better response in the chosen column and a worse one in rejected. A row looks like this (illustrative, written by hand, not model output): { "instruction": "Does the config file need a restart to take effect?", "input": "", "chosen": "The documentation you gave me does not say. Check the section on reloading before relying on it.", "rejected": "Yes, always restart the service after editing the config file." } Read a sample: python3 -c "import json,random; [print(r['instruction'], '\nCHOSEN:', r['chosen'], '\nREJECTED:', r['rejected'], '\n') for r in random.sample(json.load(open('dpo_pairs.json')), 5)]" What to check: for each sample, ask whether you agree that chosen follows your principles better than rejected. If you disagree with several, the judge prompt or the principles are the problem, not the training: fix them and run Step 2 again. Once you are happy with the ten-line run, put all your questions in prompts.txt and run it again. Step 4: rent a GPU On Deploy, choose a card. The CloudGPU docs say two things that matter here: every template is an image with root SSH, and for anything a template does not cover you can rent a raw GPU (Deploy → Raw GPU) and install what you need over SSH. For this guide, pick a template that includes PyTorch, or choose Raw GPU and install it yourself. Pick a card with enough memory for your base model: the LLaMA-Factory README has a rough memory table (marked as estimates), and nvidia-smi shows real use during the first training steps. Connect with the SSH details of your running instance. Then upload the pairs from your own machine (scp takes the port with a capital -P): scp -P <ssh-port> dpo_pairs.json <ssh-user>@<ssh-host>:/workspace/ What to check: connect with ssh -p <ssh-port> <ssh-user>@<ssh-host>, then run nvidia-smi (a GPU is listed) and python3 -c "import torch; print(torch.cuda.is_available())" (prints True). If torch is missing, install PyTorch with CUDA from the PyTorch install page first. Step 5: install LLaMA-Factory and register the preference dataset These are the install commands from the LLaMA-Factory README (read 2026-10-27; the repository name is spelled LlamaFactory there). Don't count on a pre-installed copy being current: check, and install if needed. cd /workspace git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git cd LlamaFactory pip install -e . pip install -r requirements/metrics.txt Copy the pairs into data/ and add an entry to data/dataset_info.json. The "ranking": true line is what tells LLaMA-Factory this is a preference dataset: cp /workspace/dpo_pairs.json data/ python3 - <<'EOF' import json p = "data/dataset_info.json" info = json.load(open(p, encoding="utf-8")) info["judge_pairs"] = { "file_name": "dpo_pairs.json", "ranking": True, "columns": {"prompt": "instruction", "query": "input", "chosen": "chosen", "rejected": "rejected"}, } json.dump(info, open(p, "w", encoding="utf-8"), ensure_ascii=False, indent=2) EOF What to check: llamafactory-cli help prints the command list, and grep -A 3 judge_pairs data/dataset_info.json shows your entry with "ranking": true. Step 6: choose a base model and train with DPO Small models are enough to see the pipeline work. Pick one row; the template name must match the model. | Base model | LLaMA-Factory template | License (read it yourself) | Access | |---|---|---|---| | Llama 3.2 3B Instruct (meta-llama/Llama-3.2-3B-Instruct) | llama3 | Llama 3.2 Community License | Gated on Hugging Face: accept the license, use a token | | Gemma 3 4B IT (google/gemma-3-4b-it) | gemma3 | Gemma Terms of Use | Gated on Hugging Face: accept the terms, use a token | | Mistral 7B Instruct v0.3 (mistralai/Mistral-7B-Instruct-v0.3) | mistral | License shown on the model card | The model card listed no gate when we looked (2026-10-27) | Template names are from the LLaMA-Factory README's model table; license and access information was read on 2026-10-27 and can change. The Llama and Gemma licenses have conditions that can apply if you distribute or make available a model built on them. Confirm the license for your use yourself before you train or share anything. For a gated model, request access on its Hugging Face page, then give the instance a fine-grained read token for just this run (you will revoke it at the end): export HF_TOKEN="paste-a-read-token-here" Put this in a file named judge_dpo.yaml in the LlamaFactory folder. The keys follow the official DPO LoRA example in the repo's examples/train_lora/ folder; the model, template and dataset lines are the parts you change: ### model model_name_or_path: mistralai/Mistral-7B-Instruct-v0.3 ### method stage: dpo do_train: true finetuning_type: lora lora_rank: 8 lora_target: all pref_beta: 0.1 pref_loss: sigmoid ### dataset dataset: judge_pairs template: mistral cutoff_len: 2048 preprocessing_num_workers: 4 ### output output_dir: saves/mistral-7b/lora/judge-dpo logging_steps: 10 save_steps: 100 plot_loss: true overwrite_output_dir: true ### train per_device_train_batch_size: 1 gradient_accumulation_steps: 8 learning_rate: 5.0e-6 num_train_epochs: 3.0 lr_scheduler_type: cosine warmup_ratio: 0.1 bf16: true Train: llamafactory-cli train judge_dpo.yaml Open the LLaMA-Factory examples folder and compare your YAML with the current DPO example: key names and good values change between releases. pref_loss: sigmoid is the plain DPO loss in that example. What to check: the log prints a training loss every 10 steps; saves/mistral-7b/lora/judge-dpo contains adapter_model.safetensors and adapter_config.json. If the base model download fails with a 401 or 403, the Hugging Face access request or token is the problem. An out-of-memory error means a smaller cutoff_len, a smaller model or a card with more memory; nvidia-smi shows the real use. Step 7: compare the base model and the adapter on questions it never saw A quick test in the terminal (no server is started). Create two small files in the LlamaFactory folder. infer_base.yaml: model_name_or_path: mistralai/Mistral-7B-Instruct-v0.3 template: mistral infer_backend: huggingface and infer_dpo.yaml: model_name_or_path: mistralai/Mistral-7B-Instruct-v0.3 adapter_name_or_path: saves/mistral-7b/lora/judge-dpo template: mistral infer_backend: huggingface Ask each one the questions in heldout.txt (copy that file to the instance first, or paste the questions): llamafactory-cli chat infer_base.yaml llamafactory-cli chat infer_dpo.yaml Read both sets of answers against your own principles.txt. A lower training loss is not the test; your reading of questions the adapter never trained on is. What to check: for each held-out question, write down which answer follows your principles better, or "can't tell". Stop rule: if you cannot say that the adapter's answers follow your principles better more often than the base model's, throw this round away. Change the principles, the questions or the judge prompt and start again from Step 2; do not publish the adapter or write that it improved anything. Step 8: copy the adapter off, then destroy the instance Before you destroy anything, copy out what you want to keep: destroying the instance permanently deletes its local storage. The docs have a section called "What to save before you destroy an instance"; read it in the CloudGPU docs. From your own machine: rsync -avP -e "ssh -p <ssh-port>" \ <ssh-user>@<ssh-host>:/workspace/LlamaFactory/saves/mistral-7b/lora/judge-dpo/ ./adapter-judge-dpo/ Check that adapter_model.safetensors is in ./adapter-judge-dpo/, revoke the Hugging Face token you pasted on the instance (if you used one), then destroy the instance from your Dashboard (the docs: "You can destroy an instance at any time"). The minutes you spent copying are billed like any other minute, and billing stops once the instance is destroyed. What to check: the instance no longer appears as running in your Dashboard, and ./adapter-judge-dpo/ still opens on your machine. Privacy and access, in plain terms - Step 2 sends your questions, the principles, and the candidate answers to a model API. Don't send text you would not want processed by a model service, and read the privacy policy for what the service handles. - You keep ownership of what you put on the GPU instance. CloudGPU's terms (section 5) say it does not access the contents of your instance except where needed to investigate a credible report of a policy violation, where compelled by valid legal process, or to protect the safety of users or third parties. Local instance storage is deleted when you destroy the instance. - This guide does not start a server. If you later serve your model from an instance, use your own endpoint with your own API key: the terms (section 6) say instance service URLs are not authenticated by default, so add authentication yourself before you share the URL. Sources All read on 2026-10-27 (UTC+8), except the RLAIF paper, read on 2026-11-01 (UTC+8). - Lee et al., "RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback" (arXiv 2309.00267, submitted 2023-09-01, read 2026-11-01 UTC+8): paper. Direct Preference Optimization: paper. - DeepSeek, Terms of Use, Last Update March 27, 2026: cdn.deepseek.com, sections 3.1 and 4.2. - LLaMA-Factory: README and data preparation docs. - CloudGPU: API page, terms (sections 4 to 6), acceptable use policy (section 1.8, last updated April 22, 2026), privacy policy, docs. - Licenses: Llama 3.2, Gemma, Mistral 7B Instruct v0.3 model card.

GIAO DIỆN WEB DỰNG SẴN ĐỂ FINE-TUNE HƠN 100 DÒNG LLM: LLAMA-FACTORY BIẾN VIỆC HUẤN LUYỆN MÔ HÌNH THÀNH TRẢI NGHIỆM NO-CODE Việc tinh chỉnh mô hình ngôn ngữ lớn (LLM) thường đòi hỏi kiến thức sâu sắc về lập trình PyTorch, các tham số siêu dữ liệu phức tạp và câu lệnh terminal dài dòng. Rất nhiều kỹ sư dữ liệu và nhà nghiên cứu muốn có một công cụ trực quan để quản lý các đợt huấn luyện, theo dõi biểu đồ loss và đánh giá kết quả tức thì. LLaMA-Factory là framework mã nguồn mở thống nhất toàn diện cho quá trình tinh chỉnh (Fine-tuning) hơn 100 mô hình ngôn ngữ lớn khác nhau, nổi tiếng với giao diện đồ họa LLaMA Board trực quan không cần viết code. Tech Stack: Được xây dựng trên nền tảng Python, PyTorch, Hugging Face Transformers và PEFT, giao diện web trực quan được phát triển bằng Gradio. Tích hợp sâu các công nghệ tăng tốc DeepSpeed, FlashAttention-2 và Unsloth. Tại sao nên dùng? - Giao diện đồ họa LLaMA Board toàn diện: Cấu hình toàn bộ quá trình train (từ chọn mô hình nền Llama 3, Qwen, Mistral, nạp dataset đến chỉnh learning rate) hoàn toàn trên trình duyệt web chỉ bằng vài cú nhấp chuột. - Hỗ trợ đầy đủ các phương pháp huấn luyện tiên tiến: Pre-training, Supervised Fine-Tuning (SFT), Reward Modeling, PPO, DPO, ORPO và KTO với kỹ thuật nén QLoRA 4-bit giúp tiết kiệm tối đa VRAM. - Chat thử nghiệm và xuất mô hình ngay trên giao diện: Sau khi huấn luyện xong, bạn có thể trò chuyện trực tiếp với mô hình mới ngay trên web để đánh giá câu trả lời và xuất thẳng ra định dạng GGUF hoặc vLLM để triển khai production. Dự án sở hữu hơn 42.000 stars trên GitHub và là một trong những công cụ fine-tuning phổ biến và thân thiện với người dùng nhất trong toàn bộ cộng đồng mã nguồn mở. Link tham khảo: hiyouga/LLaMA-Factory #OpenSourceVN #VNOSS #opensourcevn #vnoss #DevCommunity #LlamaFactory #LLM #FineTuning #AIWorkflow #DeepLearning #Python

Let a small model critique and revise its own answers, then fine-tune on the revisions (on a rented GPU) --- title: "Let a small model critique and revise its own answers, then fine-tune on the revisions (on a rented GPU)" published: true description: "A step-by-step self-critique loop with a small base model on one rented GPU: write principles, generate, critique, revise, filter, train a LoRA adapter with LLaMA-Factory, and check it on questions the model never saw." tags: ai, python, machinelearning, tutorial cover_image: dev-to-uploads.s3.us-east-2.amazonaws.com --- Short answer: write a short list of principles and a set of questions, have a small model answer them, critique its own answers against your principles and revise them, filter the revisions with a script and read a sample yourself, train a LoRA adapter on the result with LLaMA-Factory, then compare the base model and the adapter on questions you held back. Everything runs on one rented GPU; nothing is sent to a third-party API, and the guide starts no server. Disclosure: I work on CloudGPU, and the GPU rental in this guide is ours. Each step links the documentation it follows and ends with a "What to check" line, because versions, template names and model access change. Where the idea comes from, and what this guide does not claim Anthropic's Constitutional AI work (2022-12-15) describes a supervised phase in which the model samples answers, critiques them against a list of principles, revises them, and is then fine-tuned on the revisions: research page, paper. OpenAI's deliberative alignment post (2024-12-20) says its training data is generated automatically from written specifications, without human-labeled completions: post. This guide is a small, single-developer version of that pattern. It is our own adaptation, not a result from those sources, and we make no claim that the adapter you train will be better than the base model. A model's critique of its own answer can be wrong, and a revision can be worse than the original. That is why the last step is a comparison you read yourself, with a rule for when to throw the round away. Read these things first 1. Your base model's license. You will train on text a base model wrote, so the license of that model matters. The walkthrough uses Mistral 7B Instruct v0.3, whose model card lists the Apache-2.0 license (read 2026-10-27). Two other small models have conditions that touch this exact workflow. The Llama 3.2 Community License says that if you use the Llama Materials "or any outputs or results of the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available", the model name must begin with "Llama". The Gemma Terms of Use define Model Derivatives to include models created by "methods based on the generation of synthetic data Outputs by Gemma for training that model". Read the current text of the license of whichever model you use; this is an excerpt, not legal advice. 2. Your data. The questions you write must be ones you have the right to use. CloudGPU's acceptable use policy (section 1.8) prohibits training on datasets you do not have rights to use. Also read the terms. 3. Labelling. This guide generates data with your own model, not with an API. If you would rather use an API as the generator, the guide on generating a training set with DeepSeek, from this same series, uses DeepSeek only. DeepSeek's Terms of Use (Last Update: March 27, 2026) say in section 3.1 that if you publish or disseminate outputs, you must clearly indicate that the content is generated by artificial intelligence. If you publish any dataset or example from this guide, label it AI-generated as well. What you need - A CloudGPU account with balance. GPU instances are billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance (terms). Rates are on the pricing page. Payment is by card or USDT (payments). - A text editor and a terminal on your own machine. You write three small files there before you rent anything. What to check: you can open a terminal and run ssh -V and scp (both print a usage line or version). Step 1: write the principles and the questions (on your own machine) Create three plain-text files in one folder. principles.txt is a short list, in your own words, of what a good answer looks like for your use case. An example to replace with yours: 1. Answer only what the question asks. 2. If the answer depends on information you were not given, say what is missing instead of guessing. 3. Do not invent names, numbers, links or quotes. 4. Keep the answer under 150 words unless the question asks for more. prompts.txt is the training questions, one per line, as many as you can write (a few hundred is a reasonable start; the more varied they are, the more the loop has to work with). heldout.txt is 20 or so questions in the same style that you will not use for training. Write them separately, and do not copy lines between the files. What to check: wc -l prompts.txt heldout.txt shows both files non-empty, and comm -12 <(sort prompts.txt) <(sort heldout.txt) prints nothing (no question appears in both). Step 2: rent a GPU On Deploy, choose a card. The CloudGPU docs say two things that matter here: every template is an image with root SSH, and for anything a template does not cover you can rent a raw GPU (Deploy → Raw GPU) and install what you need over SSH. For this guide, pick a template that includes PyTorch, or choose Raw GPU and install it yourself. A 7B model in 16-bit needs a card with enough memory for the weights plus the generation cache; nvidia-smi shows real use, and a smaller model or a card with more memory is the fix for an out-of-memory error. Connect with the SSH details of your running instance. Then upload your three files (scp takes the port with a capital -P): scp -P <ssh-port> principles.txt prompts.txt heldout.txt <ssh-user>@<ssh-host>:/workspace/ What to check: connect with ssh -p <ssh-port> <ssh-user>@<ssh-host>, then ls /workspace lists your three files, nvidia-smi lists a GPU, and python3 -c "import torch; print(torch.cuda.is_available())" prints True. If torch is missing, install PyTorch with CUDA from the PyTorch install page first. Step 3: install LLaMA-Factory These are the install commands from the LLaMA-Factory README (read 2026-10-27; the repository name is spelled LlamaFactory there). The repository's pyproject.toml lists torch, transformers, accelerate and peft as dependencies, which is all the scripts below use. The docs FAQ lists a LLaMA-Factory template, but don't count on it being installed or current: check, and install if needed. cd /workspace git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git cd LlamaFactory pip install -e . pip install -r requirements/metrics.txt What to check: llamafactory-cli help prints the command list, and python3 -c "import transformers, peft; print(transformers.__version__, peft.__version__)" prints two versions without an error. Step 4: generate, critique, revise Two files. common.py loads the base model and answers a list of prompts in batches (the transformers docs say to pad on the left for batched generation, and the chat templating docs describe apply_chat_template): # /workspace/common.py import torch from transformers import AutoModelForCausalLM, AutoTokenizer MODEL = "mistralai/Mistral-7B-Instruct-v0.3" def load(): tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left") if tok.pad_token is None: tok.pad_token = tok.eos_token model = AutoModelForCausalLM.from_pretrained( MODEL, torch_dtype=torch.bfloat16, device_map="auto" ) model.eval() return tok, model def chat(tok, model, prompts, max_new_tokens=400, sample=True, batch=8): outs = [] for i in range(0, len(prompts), batch): texts = [ tok.apply_chat_template( [{"role": "user", "content": p}], tokenize=False, add_generation_prompt=True, ) for p in prompts[i : i + batch] ] enc = tok(texts, return_tensors="pt", padding=True, add_special_tokens=False) enc = enc.to(model.device) kw = {"do_sample": True, "temperature": 0.7, "top_p": 0.9} if sample else {"do_sample": False} with torch.no_grad(): gen = model.generate( **enc, max_new_tokens=max_new_tokens, pad_token_id=tok.pad_token_id, **kw ) for row in gen[:, enc["input_ids"].shape[1]:]: outs.append(tok.decode(row, skip_special_tokens=True).strip()) return outs loop.py runs the three passes: answer, critique against your principles, revise. Principles go inside the user message so the same scripts work with chat templates that have no system role. # /workspace/loop.py import json from common import load, chat NONE = "NONE" principles = open("/workspace/principles.txt", encoding="utf-8").read().strip() prompts = [l.strip() for l in open("/workspace/prompts.txt", encoding="utf-8") if l.strip()] tok, model = load() answers = chat(tok, model, prompts) critique_in = [ "Principles:\n" + principles + "\n\nQuestion:\n" + p + "\n\nAnswer:\n" + a + "\n\nIn at most three short bullet points, say where the answer breaks a principle. " "If it breaks none, reply with exactly " + NONE + "." for p, a in zip(prompts, answers) ] critiques = chat(tok, model, critique_in, max_new_tokens=200) todo = [i for i, c in enumerate(critiques) if c.strip().strip(".").upper() != NONE] revise_in = [ "Principles:\n" + principles + "\n\nQuestion:\n" + prompts[i] + "\n\nAnswer:\n" + answers[i] + "\n\nProblems found:\n" + critiques[i] + "\n\nRewrite the answer so that it follows the principles and fixes these problems. " "Reply with the rewritten answer only." for i in todo ] revised = chat(tok, model, revise_in) revised_by_index = dict(zip(todo, revised)) with open("/workspace/loop_raw.jsonl", "w", encoding="utf-8") as f: for i in range(len(prompts)): rec = {"prompt": prompts[i], "answer": answers[i], "critique": critiques[i], "revised": revised_by_index.get(i)} f.write(json.dumps(rec, ensure_ascii=False) + "\n") with open("/workspace/loop_revised.jsonl", "w", encoding="utf-8") as f: for i in todo: rec = {"prompt": prompts[i], "answer": answers[i], "critique": critiques[i], "revised": revised_by_index[i]} f.write(json.dumps(rec, ensure_ascii=False) + "\n") print("prompts:", len(prompts), "revised rows:", len(todo)) Run it from /workspace: cd /workspace python3 loop.py If your transformers version prints a warning that torch_dtype is deprecated, replace it with dtype in common.py. What to check: the script ends by printing the number of prompts and revised rows. Open loop_revised.jsonl and read five rows yourself: does the critique point at a real problem, and is the revised answer actually better than the first one? If critiques are vague or revisions are not better, stop here and change your principles or your prompts before you go on. Step 5: filter and read a sample The loop writes everything the model said, including its mistakes. Filter before you train. This script drops rows where the revision is empty, identical to the first answer, very long, repeated, or contains words that show the model talked about the task instead of answering: # /workspace/filter.py import json import random LEAK = ("principles", "critique", "rewritten", "revised answer", "problems found") rows, seen = [], set() for line in open("/workspace/loop_revised.jsonl", encoding="utf-8"): r = json.loads(line) rev = (r["revised"] or "").strip() if not rev or rev == r["answer"].strip(): continue if len(rev) > 2500 or len(rev) < 20: continue low = rev.lower() if any(w in low for w in LEAK): continue key = " ".join(low.split()) if key in seen: continue seen.add(key) rows.append({"instruction": r["prompt"], "input": "", "output": rev}) json.dump(rows, open("/workspace/selfloop_sft.json", "w", encoding="utf-8"), ensure_ascii=False, indent=2) random.seed(1) with open("/workspace/sample_to_read.md", "w", encoding="utf-8") as f: for r in random.sample(rows, min(30, len(rows))): f.write("## " + r["instruction"] + "\n\n" + r["output"] + "\n\n") print("kept:", len(rows)) python3 filter.py Then read sample_to_read.md yourself. Edit selfloop_sft.json by hand and remove any row you would not want the model to learn from, or tighten the filter and run it again. If you publish the filtered file or examples from it, label them as AI-generated. What to check: the script prints how many rows it kept. If that number is very small, the loop is not producing usable revisions for your prompts; go back to Step 1. If you read the sample and many rows are wrong or off-topic, do not train on them. Step 6: register the data and train a LoRA adapter Copy the dataset into LLaMA-Factory's data/ folder and register it in data/dataset_info.json, which is how LLaMA-Factory finds custom datasets: cd /workspace/LlamaFactory cp /workspace/selfloop_sft.json data/ python3 - <<'EOF' import json p = "data/dataset_info.json" info = json.load(open(p, encoding="utf-8")) info["selfloop_sft"] = { "file_name": "selfloop_sft.json", "columns": {"prompt": "instruction", "query": "input", "response": "output"}, } json.dump(info, open(p, "w", encoding="utf-8"), ensure_ascii=False, indent=2) EOF Save this as selfloop_lora.yaml in the LlamaFactory folder. The keys follow the official LoRA example in the repo's examples/train_lora/ folder; the model, template and dataset lines are the parts you change (the mistral template name is from the README's model table): ### model model_name_or_path: mistralai/Mistral-7B-Instruct-v0.3 ### method stage: sft do_train: true finetuning_type: lora lora_rank: 8 lora_target: all ### dataset dataset: selfloop_sft template: mistral cutoff_len: 2048 preprocessing_num_workers: 4 ### output output_dir: saves/mistral7b/lora/selfloop logging_steps: 10 save_steps: 100 plot_loss: true overwrite_output_dir: true ### train per_device_train_batch_size: 1 gradient_accumulation_steps: 8 learning_rate: 1.0e-4 num_train_epochs: 3.0 lr_scheduler_type: cosine warmup_ratio: 0.1 bf16: true ### eval val_size: 0.1 per_device_eval_batch_size: 1 eval_strategy: steps eval_steps: 50 llamafactory-cli train selfloop_lora.yaml Compare your YAML with the current LoRA example in the LLaMA-Factory examples folder: key names change between releases. What to check: the log prints a training loss every 10 steps and an evaluation loss every 50; saves/mistral7b/lora/selfloop contains adapter_model.safetensors and adapter_config.json. Make sure no earlier script is still holding the GPU (nvidia-smi shows what holds memory). If you choose another base model, change MODEL in common.py, model_name_or_path, and template together, and set a Hugging Face token for a gated model. Step 7: compare on questions the model never saw compare.py answers your held-out questions with the base model and with the adapter, greedily (no sampling, so the two columns are comparable), and writes both next to each other: # /workspace/compare.py from peft import PeftModel from common import load, chat ADAPTER = "/workspace/LlamaFactory/saves/mistral7b/lora/selfloop" qs = [l.strip() for l in open("/workspace/heldout.txt", encoding="utf-8") if l.strip()] tok, model = load() base = chat(tok, model, qs, sample=False) tuned_model = PeftModel.from_pretrained(model, ADAPTER) tuned_model.eval() tuned = chat(tok, tuned_model, qs, sample=False) with open("/workspace/compare.md", "w", encoding="utf-8") as f: for q, b, t in zip(qs, base, tuned): f.write("## " + q + "\n\n**Base:**\n\n" + b + "\n\n**Adapter:**\n\n" + t + "\n\n") print("questions:", len(qs)) cd /workspace python3 compare.py Read compare.md against your principles, question by question. Decide for yourself what "better" means, and write it down before you read. Stop rule: if the adapter is not clearly better on your principles, do not keep it. Change the principles, the prompts or the filter and run the loop again; each round uses more GPU minutes, so decide in advance when to stop. What to check: the script prints the number of questions and compare.md exists with two answers per question. Remember that 20 questions is a small test, and that you wrote the principles that decide the outcome. Step 8: copy the adapter off and destroy the instance Before you destroy anything, copy what you want to keep. The docs have a section called "What to save before you destroy an instance"; read it in the CloudGPU docs. From your own machine: rsync -avP -e "ssh -p <ssh-port>" \ <ssh-user>@<ssh-host>:/workspace/LlamaFactory/saves/mistral7b/lora/selfloop/ ./adapter-selfloop/ scp -P <ssh-port> <ssh-user>@<ssh-host>:/workspace/compare.md ./ Check that adapter_model.safetensors is in ./adapter-selfloop/, then destroy the instance from your Dashboard (the docs: "You can destroy an instance at any time"). The minutes spent copying are billed like any other minute, and billing stops once the instance is destroyed. What to check: the instance no longer appears as running in your Dashboard, and ./adapter-selfloop/ opens on your machine. Privacy and access, in plain terms - The guide generates and trains on the instance itself and does not call an API, so your questions and the model's answers stay on that instance and on your machine. No server is started, so nothing listens for outside requests. If you later serve the model, put your own API key in front of it before you expose it. - Per CloudGPU's terms, destroying an instance deletes its local storage; copy what you need first. - If you used a Hugging Face token for a gated base model, revoke it when you are done. Sources All read on 2026-10-27 (UTC+8). - Anthropic, 2022-12-15: Constitutional AI: Harmlessness from AI Feedback and the paper. - OpenAI, 2024-12-20: Deliberative alignment and the paper. - Licenses: Mistral 7B Instruct v0.3 model card, Llama 3.2 Community License, Gemma Terms of Use. - DeepSeek: Terms of Use (Last Update: March 27, 2026). - Tools: LLaMA-Factory, transformers text generation tutorial, chat templating. - CloudGPU: terms, acceptable use policy, docs, pricing.

Generate a training set with DeepSeek, then fine-tune a small model on a rented GPU --- title: "Generate a training set with DeepSeek, then fine-tune a small model on a rented GPU" published: true description: "A step-by-step LoRA tutorial: generate instruction data with DeepSeek through the OpenAI SDK, filter it, train a small Llama, Gemma or Mistral model with LLaMA-Factory on a rented GPU, download the adapter and destroy the instance." tags: ai, python, machinelearning, tutorial cover_image: dev-to-uploads.s3.us-east-2.amazonaws.com --- Short answer: write a few documentation pages yourself, have DeepSeek turn them into question-and-answer pairs through an OpenAI-compatible endpoint, filter the pairs with a short script, then train a LoRA adapter on a small base model with LLaMA-Factory on a rented GPU. Generate and filter on your own machine; rent the GPU only for training; download the adapter, then destroy the instance. Disclosure: I work on CloudGPU. The endpoint (https://cloudgpu.app/v1) and the GPU rentals below are ours. Each step links the official documentation it follows and ends with a "What to check" line, because the details (versions, template names, model access) change. Why this pattern? AI labs describe using models to write or judge training data from written rules. This is the small, single-developer version of that idea. It is our own analogy, not a claim from those papers. Read these three things first 1. DeepSeek's terms. This tutorial generates data with DeepSeek only. DeepSeek's Terms of Use (Last Update: March 27, 2026) say in section 4.2(3) that you may apply the Outputs of the Services to "a wide range of use cases, including personal use, academic research, derivative product development, training other models (such as model distillation), etc., as long as such usage is legal and adhere to these Terms." Section 3.1 adds that if you publish or disseminate outputs, you must "clearly indicate that the output content is generated by artificial intelligence". So if you publish the dataset, a model card or examples from it, label them as AI-generated. Read the terms yourself, and read CloudGPU's terms and acceptable use policy, which apply to the gateway. 2. Rights to your source text. The acceptable use policy lists "Training models on datasets you do not have rights to use" under large-scale copyright infringement (section 1.8). In this tutorial the generator only sees documentation you wrote yourself. 3. The base model's license. Each base model below has its own license. Open the link, read it, and confirm for yourself that your use is allowed. What you need - Python 3 and the openai package on your own machine: pip install openai. - A CloudGPU account with balance, and an API key (the API page has a "Get your API key" link). Keep it in an environment variable, never in code: export CLOUDGPU_API_KEY="paste-your-key-here" - Three to ten Markdown files of your own documentation, with ## section headings, in a docs/ folder. A README of your own project works. - A Hugging Face account and a read token if you choose Llama or Gemma (both are gated, see the table below). What to check: python3 -c "import os; print(bool(os.environ.get('CLOUDGPU_API_KEY')))" prints True in the shell where you will run the scripts, and python3 -c "import openai" raises no error. API calls are billed per token from the same balance as GPU rentals; the current rate for every model is in the table on the API page. This post does not repeat rates. Step 1: generate question-and-answer pairs with DeepSeek The script splits each Markdown file at ## headings, asks deepseek-v4.1-flash for a handful of pairs per section, and writes one JSON object per line. The model ID comes from the live model table on the API page (checked 2026-10-27); the same table lists deepseek-v4-flash and deepseek-v4-pro. The OpenAI SDK only needs base_url and the key changed. # gen_data.py import json, os, re, sys, time from concurrent.futures import ThreadPoolExecutor from pathlib import Path from openai import OpenAI client = OpenAI(base_url="https://cloudgpu.app/v1", api_key=os.environ["CLOUDGPU_API_KEY"]) MODEL = "deepseek-v4.1-flash" PAIRS_PER_SECTION = 6 PROMPT = """You write training examples for a support assistant. Use ONLY the documentation below. Do not invent features or settings. Write {n} varied question-and-answer pairs that a user might ask about it. Return ONLY a JSON array. Each item has the keys "instruction" (the question), "input" (extra context, or an empty string) and "output" (the answer, 1-4 sentences). Documentation: {chunk} """ def sections(folder): for path in sorted(Path(folder).glob("*.md")): for part in re.split(r"\n(?=## )", path.read_text(encoding="utf-8")): if len(part.strip()) > 200: yield path.name, part.strip() def parse(text): fence = "`" * 3 # the model may wrap its JSON in a code fence text = text.strip().removeprefix(fence + "json").removeprefix(fence).removesuffix(fence).strip() rows = json.loads(text) keys = ("instruction", "input", "output") return [r for r in rows if isinstance(r, dict) and all(isinstance(r.get(k), str) for k in keys)] def ask(item): name, chunk = item for attempt in range(3): try: resp = client.chat.completions.create( model=MODEL, temperature=0.8, messages=[{"role": "user", "content": PROMPT.format(n=PAIRS_PER_SECTION, chunk=chunk)}], ) return parse(resp.choices[0].message.content) except Exception as err: # network error, rate limit, bad JSON print(f"{name}: attempt {attempt + 1} failed: {err}", file=sys.stderr) time.sleep(2 ** attempt) return [] if __name__ == "__main__": with ThreadPoolExecutor(max_workers=4) as pool: results = list(pool.map(ask, sections("docs"))) with open("raw.jsonl", "w", encoding="utf-8") as f: for rows in results: for row in rows: f.write(json.dumps(row, ensure_ascii=False) + "\n") print("sections:", len(results), "pairs:", sum(len(r) for r in results)) Run python3 gen_data.py. The four-worker pool and the retry loop keep you inside the default rate limits listed on the API page; check those limits there and lower max_workers if you see rate-limit errors. What to check: raw.jsonl exists and has lines; the script prints how many sections and pairs it got. If a section failed three times it is skipped and the error is on stderr. A 401 means the key variable is not set in this shell. Step 2: filter and de-duplicate Model-written data needs a filter before training. This script drops empty rows, answers that are very short or very long, boilerplate such as "as an AI", and near-duplicate questions (token overlap of 0.8 or more). Tune the numbers for your data. # filter_data.py import json, re from pathlib import Path def tokens(s): return set(re.sub(r"\W+", " ", s.lower()).split()) def jaccard(a, b): union = a | b return len(a & b) / len(union) if union else 0.0 kept, seen = [], [] dropped = {"empty": 0, "length": 0, "boilerplate": 0, "duplicate": 0} for line in Path("raw.jsonl").read_text(encoding="utf-8").splitlines(): row = json.loads(line) ins, inp, out = (row[k].strip() for k in ("instruction", "input", "output")) if not ins or not out: dropped["empty"] += 1 elif not 20 <= len(out) <= 1200: dropped["length"] += 1 elif re.search(r"\bas an ai\b|\blanguage model\b", out, re.I): dropped["boilerplate"] += 1 elif any(jaccard(tokens(ins), t) >= 0.8 for t in seen): dropped["duplicate"] += 1 else: seen.append(tokens(ins)) kept.append({"instruction": ins, "input": inp, "output": out}) Path("synth_docs_qa.json").write_text(json.dumps(kept, ensure_ascii=False, indent=2), encoding="utf-8") Path("DATASET_NOTE.md").write_text( "AI-generated data: the examples in synth_docs_qa.json were generated with deepseek-v4.1-flash " "from documentation written by the dataset author. Review before use.\n", encoding="utf-8") print("kept", len(kept), "dropped", dropped) The output is in LLaMA-Factory's Alpaca format: instruction, optional input, and output, per the data preparation docs. A row looks like this (illustrative, written by hand, not model output): { "instruction": "How do I change the output folder?", "input": "", "output": "Pass --out followed by a path, for example --out ./build." } Then read a sample yourself. Filters catch format problems, not wrong answers: python3 -c "import json,random; [print(r['instruction'],'->',r['output'],'\n') for r in random.sample(json.load(open('synth_docs_qa.json')), 10)]" What to check: DATASET_NOTE.md is the label DeepSeek's terms ask for if you publish the data: keep it next to the file. Compare ten sampled answers with your documentation; delete rows that are wrong, or tighten the prompt and regenerate. If the filter drops most rows, the model is probably not returning clean JSON: print one raw response and adjust parse. If your data is multi-turn, LLaMA-Factory also reads a conversation format; the same docs page shows the dataset_info.json entry for it. Step 3: rent a GPU On Deploy, choose a card. The CloudGPU docs say two things that matter here: every template is an image with root SSH, and for anything a template does not cover you can rent a raw GPU (Deploy → Raw GPU) and install what you need over SSH. For this tutorial, pick a template that includes PyTorch, or choose Raw GPU and install it yourself. Pick a card with enough memory for your base model: the LLaMA-Factory README has a rough memory table (marked as estimates), and nvidia-smi shows real use during the first training steps. GPU instances are billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance (terms). Rates are on the pricing page. Connect with the SSH details of your running instance. Then upload the dataset from your own machine (scp takes the port with a capital -P): scp -P <ssh-port> synth_docs_qa.json <ssh-user>@<ssh-host>:/workspace/ What to check: connect with ssh -p <ssh-port> <ssh-user>@<ssh-host>, then run nvidia-smi (a GPU is listed) and python3 -c "import torch; print(torch.cuda.is_available())" (prints True). If torch is missing, install PyTorch with CUDA from the PyTorch install page first. Step 4: install LLaMA-Factory and register the dataset These are the install commands from the LLaMA-Factory README (read 2026-10-27; the repository name is spelled LlamaFactory there). The docs FAQ lists a LLaMA-Factory template, but don't count on it being installed or current: check, and install if needed. cd /workspace git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git cd LlamaFactory pip install -e . pip install -r requirements/metrics.txt Copy the dataset into data/ and add an entry to data/dataset_info.json, which is how LLaMA-Factory finds custom datasets: cp /workspace/synth_docs_qa.json data/ python3 - <<'EOF' import json p = "data/dataset_info.json" info = json.load(open(p, encoding="utf-8")) info["synth_docs_qa"] = { "file_name": "synth_docs_qa.json", "columns": {"prompt": "instruction", "query": "input", "response": "output"}, } json.dump(info, open(p, "w", encoding="utf-8"), ensure_ascii=False, indent=2) EOF What to check: llamafactory-cli help prints the command list, and grep synth_docs_qa data/dataset_info.json finds your entry. Step 5: choose a base model and train a LoRA adapter Small models are enough to see the pipeline work. Pick one row; the template name must match the model. | Base model | LLaMA-Factory template | License (read it yourself) | Access | |---|---|---|---| | Llama 3.2 3B Instruct (meta-llama/Llama-3.2-3B-Instruct) | llama3 | Llama 3.2 Community License | Gated on Hugging Face: accept the license, use a token | | Gemma 3 4B IT (google/gemma-3-4b-it) | gemma3 | Gemma Terms of Use | Gated on Hugging Face: accept the terms, use a token | | Mistral 7B Instruct v0.3 (mistralai/Mistral-7B-Instruct-v0.3) | mistral | License shown on the model card | The model card listed no gate when we looked (2026-10-27) | Template names are from the LLaMA-Factory README's model table; license and access information was read on 2026-10-27 and can change. The Llama license has conditions that apply if you distribute or make available a model built on it. Confirm the license for your use yourself before you train or share anything. For a gated model, request access on its Hugging Face page, then give the instance a fine-grained read token for just this run (you will revoke it at the end): export HF_TOKEN="paste-a-read-token-here" Save this as synth_lora.yaml in the LlamaFactory folder. The keys follow the official LoRA example in the repo's examples/train_lora/ folder; the model, template and dataset lines are the parts you change: ### model model_name_or_path: meta-llama/Llama-3.2-3B-Instruct ### method stage: sft do_train: true finetuning_type: lora lora_rank: 8 lora_target: all ### dataset dataset: synth_docs_qa template: llama3 cutoff_len: 2048 preprocessing_num_workers: 4 ### output output_dir: saves/llama32-3b/lora/synth logging_steps: 10 save_steps: 100 plot_loss: true overwrite_output_dir: true ### train per_device_train_batch_size: 1 gradient_accumulation_steps: 8 learning_rate: 1.0e-4 num_train_epochs: 3.0 lr_scheduler_type: cosine warmup_ratio: 0.1 bf16: true ### eval val_size: 0.1 per_device_eval_batch_size: 1 eval_strategy: steps eval_steps: 50 Train: llamafactory-cli train synth_lora.yaml Open the LLaMA-Factory examples folder and compare your YAML with the current LoRA example: key names change between releases. What to check: the log prints a training loss every 10 steps and, with val_size set, an evaluation loss every 50; saves/llama32-3b/lora/synth contains adapter_model.safetensors and adapter_config.json. If the base model download fails with a 401 or 403, the Hugging Face access request or token is the problem. An out-of-memory error means a smaller cutoff_len, a smaller model or a card with more memory; nvidia-smi shows the real use. Unsloth (optional). The LLaMA-Factory README lists Unsloth among its supported add-ons and describes a use_unsloth: true argument for LoRA. Unsloth has its own documentation and install steps, and supports a specific list of models, so check that your base model is on its list before you add it. Step 6: try the adapter, copy it off, destroy the instance A quick test in the terminal (no server is started). Save as infer.yaml: model_name_or_path: meta-llama/Llama-3.2-3B-Instruct adapter_name_or_path: saves/llama32-3b/lora/synth template: llama3 infer_backend: huggingface llamafactory-cli chat infer.yaml Ask ten questions that are not in your training data and that you wrote yourself, and compare the answers with your documentation. That is your real test; the evaluation loss is not. Now copy the adapter to your own machine before you destroy anything: destroying the instance permanently deletes its local storage. From your own machine: rsync -avP -e "ssh -p <ssh-port>" \ <ssh-user>@<ssh-host>:/workspace/LlamaFactory/saves/llama32-3b/lora/synth/ ./adapter-synth/ Check that adapter_model.safetensors is in ./adapter-synth/, revoke the Hugging Face token you pasted on the instance, then destroy the instance from your Dashboard (the docs: "You can destroy an instance at any time"). The minutes you spent copying are billed like any other minute, and billing stops once the instance is destroyed. What to check: the instance no longer appears as running in your dashboard, and ./adapter-synth/ still opens on your machine. Privacy and access, in plain terms - Generation (step 1) sends the documentation sections you give it to a model API. Don't send text you would not want processed by a model service, and read the privacy policy for what the service handles. - You keep ownership of what you put on the GPU instance. CloudGPU's terms (section 5) say it does not access the contents of your instance except where needed to investigate a credible report of a policy violation, where compelled by valid legal process, or to protect the safety of users or third parties. Local instance storage is deleted when you destroy the instance. - This tutorial does not start a server. If you later serve your model from an instance, use your own endpoint with your own API key: the terms (section 6) say instance service URLs are not authenticated by default, so add authentication yourself before you share the URL. Sources All read on 2026-10-27 (UTC+8). - DeepSeek, Terms of Use, Last Update March 27, 2026: cdn.deepseek.com, sections 3.1 and 4.2. - LLaMA-Factory: README and data preparation docs. - CloudGPU: API page, terms (sections 4 to 6), acceptable use policy (section 1.8, last updated April 22, 2026), privacy policy. - Licenses: Llama 3.2, Gemma, Mistral 7B Instruct v0.3 model card.

想微调大模型，却被训练配置劝退？🧠 LlamaFactory 提供微调工具和 Web UI，可实验 LoRA 等方法。跑训练仍需相应算力。🛠️ hiyouga/LlamaFactory #FineTuning #浇朋友 #蓝朋友 #互关 #fo

7.5 万颗星。想微调自己的模型，这是最省事的入口：100+ 大模型和视觉模型统一成一套配置，改几行参数就开跑。 自己有一个垂直模型，和只能租通用模型，长期看是两种生意。 hiyouga/LlamaFactory