kkdai

將原來 Python Gemini ADK 的框架改成 Go ADK ，資源少超多的。 這支 Bot 還能直接聽語音、幫你做網頁摘要，甚至連 GitHub 的 Issues 都能自動彙整。 kkdai/linebot-helper-go

[AI in Action] Rewriting a Daily LINE Bot from Python to Go 1.26 on Cloud Run: 1/4 Resource Usage, Google ADK v1.9.0, and Zero-D !LINE Bot Helper: Python to Go 1.26 & Cloud Run Architecture (ADK v1.9.0) Background I have a primary LINE Bot that I use heavily every day: kkdai/linebot-helper-python. Initially, it was just a small tool where you could "drop a URL and get a Traditional Chinese summary," but over the past year and a half, I've gradually integrated various new Gemini capabilities into it: - Multi-source Content Loading and Document Conversion: A three-layer web crawler Fallback Chain (HTTP $\rightarrow$ Firecrawl $\rightarrow$ SingleFile Headless Chromium), YouTube video summaries, automatic conversion of PDF / Word / PPT / Excel / EPUB to Markdown, and one-click generation of FB / LinkedIn / Threads viral social media copy Carousels. - Bookmarks and Grounding Deep Research Reports: /save, /list, and /search bookmark management. Clicking "📄 Detailed Research Report" on a card combines Google Search Grounding to write a complete Markdown report, rendered as an independent HTML page (/reports/{id}) and permanently saved in Cloud Firestore. - Google Maps Grounding and Gemini Batch API Gourmet Big Data Analysis: Send LINE location coordinates to search for nearby gas stations, parking lots, and restaurants. It can also pull 50+ latest reviews for specific restaurants and feed them into the Gemini Batch API (enjoying a 50% cost discount) for deep analysis of signature dishes and pitfalls, pushed back to LINE via Webhook or background polling. - Agentic Vision Visual Annotation: After uploading an image, gemini-3-flash-preview writes Python code to draw bounding boxes, which are then saved as images and pushed back to the chat room. - LINE Voice Message TTS and LIFF Real-time Two-way Voice Assistant: Voice messages are automatically transcribed and paired with gemini-3.1-flash-tts-preview + ffmpeg to generate M4A voice readings. Opening LIFF allows two-way streaming of gemini-3.1-flash-live-preview via WebSocket, supporting hands-free VAD, Barge-in interruption, and Live Tool Calling. The features are great, but the cost is obvious: the resident memory for Python + FastAPI + a large number of document parsing and AI packages kept increasing. To ensure background tasks and multi-concurrency didn't lag, this Cloud Run service was unknowingly scaled up to 4 vCPU / 4 GiB Memory (with --no-cpu-throttling enabled). As someone who has been writing Go for a long time, seeing a personal daily assistant consume 4 vCPU / 4 GiB was becoming hard to stomach. So, I decided to do one thing: rewrite the entire linebot-helper-python 100% into the latest Go 1.26 version (kkdai/linebot-helper-go), deploy it to Cloud Run in the same GCP project, and ensure it shares Firestore data with the old version for zero-downtime switching at any time. - * * 1. Setting Rules Before Rewriting: Breaking Down 11 Milestones with GitHub Issues Moving a nearly 90KB main.py plus a dozen modules from Python to Go can easily result in an unmaintainable maze if boundaries aren't defined and testing isn't done along the way. Therefore, after creating kkdai/linebot-helper-go, the first task wasn't writing code, but mapping out all features from the Python version. I opened Tracking Issue #12 and 11 sub-task Issues (#1 ~ #11) on GitHub, with each Issue corresponding to an independent package and complete unit tests: | Milestone Issue | Corresponding Go Package | Core Responsibilities and Compatibility Highlights | | --- | --- | --- | | #1 Project Skeleton and Dockerfile | main.go, internal/config | Go 1.26 net/http, 100% environment variable compatibility with Python version, Multi-stage Dockerfile (integrating Chromium / single-file / ffmpeg) | | #2 Gemini Dual-channel Client | internal/gemini | Supports both Vertex AI (ADC) and Gemini Developer API (GOOGLE_AI_API_KEY), covering Grounding, Code Execution, and TTS | | #3 Firestore Persistence Layer | internal/firestore, internal/services | Fully compatible with Python version's chat_sessions, bookmarks, reports, and batch_jobs collection schemas; auto-downgrades to in-memory storage without credentials | | #4 LINE Messaging API and Flex | internal/line | HMAC-SHA256 signature verification, Reply / Push / Loading Animation, Social copy and Bookmark Flex Carousel assembly | | #5 URL Crawler and Multi-format Doc Conversion | internal/loader | HTTP (including PTT over18=1) $\rightarrow$ Firecrawl $\rightarrow$ SingleFile three-layer time-budget Fallback, and PDF / DOCX / PPTX / XLSX / EPUB / ODF to Markdown | | #6 AI Toolset | internal/ai | Summaries, viral social copy, Grounding research reports, YouTube analysis, Agentic Vision, GitHub Issues aggregation, Maps search | | #7 ADK Orchestrator and Specialized Agents | internal/agents | Integrates official google.golang.org/adk v1.9.0, establishing a Root Orchestrator and 5 specialized Agents: Chat / Content / Location / Vision / GitHub | | #8 Asynchronous Webhook and Postback | internal/handlers | Webhook returns 200 OK instantly, background Goroutines handle events, 9 types of Postback dispatching, and /reports/{id}, /images/{id}, /audio/{id} endpoints | | #9 Gemini Batch API Gourmet Analysis | internal/services/batch_service.go | JSONL batch job upload, Static HMAC Webhook, Dynamic RS256 JWKS Webhook verification, and 60s background polling | | #10 LIFF Real-time Voice Assistant | internal/voice, static/liff | /ws/voice/{session_id} two-way WebSocket Relay, Push-to-Talk / hands-free VAD, Barge-in, Session Resumption, and Live Tool Calling | | #11 Cloud Run Deployment and Verification | Cloud Run (us-central1) | Deployed using existing Python version environment variables, stress tested, and container resource specs downsized | The overall architecture is shown in the diagram below: flowchart TB subgraph Client["LINE Client and LIFF"] LINE["LINE App Chatroom\n(Text / URL / Image / Voice / Location / Postback)"] LIFF["LIFF Real-time Voice Assistant\n(/liff/ + WebSocket PCM 16kHz/24kHz)"] end subgraph CloudRun["Google Cloud Run: linebot-helper-go (Go 1.26, 1 vCPU / 1 GiB)"] Mux["Go 1.26 net/http Server\n(HMAC Verification + Immediate 200 OK)"] VoiceRelay["internal/voice\nGemini Live BidiGenerateContent Relay"] Orch["internal/agents (google.golang.org/adk v1.9.0)\nRoot Orchestrator + 5 Sub-Agents"] Loader["internal/loader\nHTTP -> Firecrawl -> SingleFile\n+ PDF/Office/EPUB to Markdown"] BatchSvc["internal/services/batch_service\nGemini Batch API + JWKS/HMAC Webhook + 60s Poller"] end subgraph GCP["Google Cloud & AI Backend"] Vertex["Vertex AI / Gemini Developer API\n(Search Grounding / Maps / Vision / TTS / Live)"] FS[("Cloud Firestore\nchat_sessions | bookmarks\nreports | batch_jobs")] end LINE -->|"POST /"| Mux LIFF <-->|"GET /ws/voice/{id}"| VoiceRelay Mux --> Orch Mux --> Loader Mux --> BatchSvc VoiceRelay <-->|"WebSocket"| Vertex VoiceRelay -.->|"delegate_to_orchestrator"| Orch Orch --> Vertex Orch --> Loader Orch <--> FS BatchSvc <--> Vertex BatchSvc <--> FS - * * 2. Five Engineering Designs Particularly Noteworthy During the Rewriting Process 1. Dual-track Gemini Client for Vertex AI and Developer API In this LINE Bot, not all features use the same Gemini endpoint: - General Chat, Grounding Search, Google Maps Grounding, Image Analysis: Uses Vertex AI (using Application Default Credentials of the Cloud Run service account, no API Key required). - Native TTS Voice Synthesis (gemini-3.1-flash-tts-preview), Gemini Batch API, and Gemini Live API WebSocket: Uses Gemini Developer API (using GOOGLE_AI_API_KEY). - Agentic Vision (gemini-3-flash-preview + Code Execution): Prioritizes Developer API, with automatic Fallback to Vertex AI if no Key is set. In the Go version's internal/gemini/client.go, we converged these two paths under the same gemini.Client. Even converting the Raw PCM (24kHz, 16-bit, mono) returned by TTS into LINE-supported audio/mp4 (.m4a) via ffmpeg and calculating audio milliseconds is handled seamlessly within the same module. 2. 100% Firestore Schema Compatibility with the Python Version (Key to Zero-Downtime Switching) The biggest fear when rewriting a service is that "after switching, user chat history, half-saved bookmarks, or previously generated research report URLs will all become invalid." To prevent this, we strictly aligned the 4 Firestore Collection names and field structures in internal/firestore and internal/services with the Python version: - chat_sessions: Stores multi-turn chat history and the last GPS coordinates sent by the user (last_location). - bookmarks: Stores /save bookmarks, viral social copy candidate bookmarks (status: candidate $\rightarrow$ saved), and 7-day cached report_id and report_generated_at. - reports: Stores the fully rendered HTML of Grounding deep research reports (supporting permanent retrieval via /reports/{report_id}). - batch_jobs: Stores the mapping between Gemini Batch API task IDs and LINE user_id. This means the Python version and Go version can simultaneously read and write to the same Firestore online. The moment the LINE Webhook is pointed to the new version, all old bookmarks and report links remain completely unaffected; if I ever want to switch back to the Python version, I only need to change one line of the Webhook URL. 3. Native Go Implementation for Multi-format Document to Markdown Conversion (Saying Goodbye to Heavy Python Packages) The Python version used packages like anydoc to convert PDF, DOCX, PPTX, XLSX, ODT, EPUB, CSV, and RTF to Markdown before feeding them to Gemini for summarization. These packages in Python often pull in a long list of C extensions and dependencies. In the Go version internal/loader/doc_converter.go, since .docx, .pptx, .xlsx, .odt, and .epub are essentially ZIP-packaged XML files, we used the Go standard library's archive/zip + encoding/xml along with a lightweight PDF text extractor. With less than a few hundred lines of pure Go code, we can cleanly convert all mainstream office documents and e-book formats into structured Markdown. It compiles fast and has almost zero extra memory overhead at runtime! - * * 3. Facing the Architecture Honestly: Python Version's "Conceptual ADK" vs. Go Version's Actual Implementation of google.golang.org/adk v1.9.0 While checking the code, we found an interesting historical trace: If you open the beginning of the original Python version's agents/orchestrator.py, you'll see a comment saying: > Orchestrator Agent - Main control Agent based on Google ADK But if you look at the Python version's requirements.txt and the entire source code, you'll find that google-adk isn't actually installed, and there isn't a single line of from google.adk import ...! The Python version at that time merely "borrowed the hierarchical concept of ADK," manually creating Orchestrator, ChatAgent, ContentAgent, etc., using pure Python classes, with the underlying layer calling google-genai directly. For this Go rewrite, we decided to do it for real—directly introducing the latest official Google google.golang.org/adk v1.9.0 in go.mod (paired with google.golang.org/genai v1.57.0): | Comparison Item | Original Python Version (linebot-helper-python) | New Go Version (linebot-helper-go) | | --- | --- | --- | | ADK Package Dependency | ❌ None (not installed) | ✅ google.golang.org/adk v1.9.0 | | Agent Definition | Custom Python Class | Uses llmagent.New(llmagent.Config{...}) to create agent.Agent | | Sub-Agents Hierarchy | Manual object attributes | Root orchestrator mounts 5 specialized sub-agents via SubAgents: []agent.Agent{...} | | Tools Binding | Manual method calls | Uses functiontool.New() to register type-safe Go function tools (load_url_content, summarize_youtube_video, search_nearby_places) | | Model Interface | Direct call to google.genai.Client | Implements google.golang.org/adk/model.LLM and adkgemini.NewModel (Vertex AI Backend) | Next Step: From "Building an ADK Agent Tree" to "Fully Driven by ADK Runner" (Issue #13) To ensure the first migration version passes all existing test cases 100%, ProcessText currently retains the original fast intent routing. However, we have already planned the following 5 exciting ADK upgrades in Issue #13 ([RFC] Google ADK (v1.9.0) Deep Integration and Architecture Evolution Proposal): 1. Replace Hardcoded Regex Routing with runner.Runner + SubAgents / agenttool: When a user enters a composite command in LINE (e.g., "Help me find the restaurant mentioned in this article and save it to bookmarks"), ADK will automatically transfer or call multiple Sub-Agents in parallel. 2. Add BookmarkAgent and Research Report functiontool: Users won't need to remember /save or /search slash commands; they can simply say "Find the bookmark about MCP I saved last time and make a research report" to trigger it. 3. Implement adk/session.Service for Cloud Firestore: Wrap the existing firestore.KVStore into a standard session.Service, allowing ADK Runner to natively manage multi-turn conversation events and states. 4. Introduce workflowagents (sequentialagent / parallelagent): Deconstruct deep research reports into a sequential workflow of "Extraction $\rightarrow$ Google Search Verification $\rightarrow$ Report Writing"; change viral copy for FB / LinkedIn / Threads to parallel Sub-Agents for simultaneous output. 5. Connect LIFF Real-time Voice Assistant's delegate_to_orchestrator directly to ADK Runner. - * * 4. Cloud Run Deployment and Slimming Down: Machine Specs Cut Directly to 1/4! After completing all 10 core packages and unit tests (go vet ./... && go test ./... all passed), we deployed linebot-helper-go directly to Cloud Run. Let's look at the container resource configuration comparison between the Python and Go versions on the same GCP project: | Cloud Run Service | Runtime Environment | CPU | Memory | Concurrency | CPU Allocation Mode | Relative Resource Cost | | --- | --- | --- | --- | --- | --- | --- | | linebot-helper-python | Python 3.11 + FastAPI | 4 vCPU | 4 GiB | 80 | Always allocated (--no-cpu-throttling) | 100% (Baseline) | | linebot-helper-go | Go 1.26 Native Compilation | 1 vCPU | 1 GiB | 160 | Always allocated (--no-cpu-throttling) | 25% (75% Savings) | You might ask: The resident memory of the Go compiled executable is only 20–30 MB, why not just use 512 MiB instead of 1 vCPU / 1 GiB? Both numbers have practical engineering considerations behind them: 1. Why 1 vCPU? Because the LINE Bot needs to return 200 OK instantly upon receiving a Webhook and continue running Gemini generation and research report rendering in background Goroutines. Plus, there's a background Ticker polling the Gemini Batch API every 60 seconds. Therefore, --no-cpu-throttling (CPU always allocated) must be enabled. In Cloud Run specifications, the minimum CPU threshold for enabling --no-cpu-throttling is 1 vCPU. 2. Why 1 GiB Memory? The Go program itself is indeed under 30 MB, but don't forget our 3rd layer Fallback for the URL crawler has single-file + Headless Chromium installed in the container (used to deal with dynamic rendering sites that even Firecrawl blocks), plus voice message synthesis requires calling ffmpeg. When Headless Chromium is occasionally awakened, it instantly consumes 300–400 MB of memory. Keeping 1 GiB ensures Chromium never OOMs (Out of Memory) upon startup while cutting machine costs to 1/4 of the original Python version! The deployment command is very clean: gcloud run deploy linebot-helper-go \ --source . \ --project <YOUR_GCP_PROJECT_ID> \ --region us-central1 \ --cpu 1 \ --memory 1Gi \ --allow-unauthenticated \ --no-cpu-throttling - * * 5. Seven Devilish Details Most Likely to be Tripped Over or Missed in Cloud Deployment Moving a LINE Bot from local testing to Cloud Run, or switching from an old Python service to a new Go service, "successful compilation" often only accounts for 60% of the work. The real traps are usually hidden in Cloud Run's runtime lifecycle, network proxies, container sandboxing, and external endpoint settings. Here are the 7 key blind spots most easily overlooked during this deployment process: Omission 1: Forgot to add --no-cpu-throttling, or passed r.Context() into a background Goroutine LINE Webhook specifications require the server to return 200 OK quickly after receiving an event. Our heavy tasks (such as Grounding deep research reports taking 1–2 minutes, Agentic Vision writing code to draw, voice transcription plus TTS, and the Gemini Batch API Poller polling every 60 seconds) all return 200 OK first, then run go func() in the background and use PushMessage to push the result. There are two traps here that both beginners and experts are extremely likely to fall into: 1. Cloud Run "freezes the CPU" by default the moment the HTTP Response is sent: If you don't add --no-cpu-throttling, as soon as handleWebhookCallback returns 200 OK, Cloud Run will drop the container CPU to near 0 within tens of milliseconds. Your background Goroutine will be stuck in mid-air like it was paused, only moving a few steps when the next HTTP request comes in minutes later. The user will never receive the push! 2. Background Goroutines must never use r.Context(): In Go's net/http, r.Context() is triggered as Canceled the moment the HTTP Handler ends (sends 200 OK)! If you casually write go s.handleMessageEvent(r.Context(), event), background requests calling Vertex AI or Firestore will die directly with a context canceled error. // ❌ Incorrect: r.Context() will be canceled the moment 200 OK is returned; background tasks will fail. go s.handleMessageEvent(r.Context(), eventCopy) // ✅ Correct: Detach from the HTTP Request lifecycle, use an independent Background Context (set Timeout internally). go s.handleMessageEvent(context.Background(), eventCopy) Omission 2: Vertex AI GOOGLE_CLOUD_LOCATION=global Domain Concatenation Trap When reusing old environment variables, we saw GOOGLE_CLOUD_LOCATION=global (because many Gemini 3.x Preview models in Vertex AI are under the global location). If you write a Vertex AI REST Client in Go and follow the habit for general regions by using fmt.Sprintf("https://%s-aiplatform.googleapis.com", location), when location is "global", the resulting URL becomes: > ❌ https://global-aiplatform.googleapis.com (This domain doesn't exist in GCP; DNS lookup will fail!) In Vertex AI specifications, only regional endpoints (like us-central1, asia-east1) have a prefix; when the Location is global, you must switch to the prefix-less https://aiplatform.googleapis.com: func (c *Client) vertexEndpointBase() string { if c.VertexBaseURL != "" { return strings.TrimRight(c.VertexBaseURL, "/") } if c.Location == "" || c.Location == "global" { return "https://aiplatform.googleapis.com" } return fmt.Sprintf("https://%s-aiplatform.googleapis.com", c.Location) } Omission 3: Moving Environment Variables Using --set-env-vars Leading to Character Truncation or Secret Leakage When moving over a dozen environment variables from linebot-helper-python to linebot-helper-go, never manually string together --set-env-vars="KEY1=VAL1,KEY2=VAL2...": 1. If any Token or Prompt string contains a comma , or equals sign =, gcloud's parameter parsing will split at the wrong position or even error out. 2. Putting environment variables directly in the command line parameters leaves the entire Secret string in the terminal history or conversation logs. The safest way: Directly export the temporary /tmp/env.yaml locally using gcloud run services describe --format=json, feed it in during deployment using --env-vars-file=/tmp/env.yaml, and rm -f /tmp/env.yaml immediately after deployment. This way, no Secrets are printed, and all special characters are 100% preserved. Omission 4: Ignoring Cloud Run Reverse Proxy's X-Forwarded-Proto, Leading to LINE Rejecting Images and Audio This LINE Bot needs to dynamically assemble "public URLs pointing to its own server" in four places: 1. Research report page: https://<host>/reports/{report_id} 2. Agentic Vision annotated image: https://<host>/images/{image_id} 3. TTS voice synthesis audio file: https://<host>/audio/{audio_id} 4. Gemini Batch API Dynamic Webhook: https://<host>/api/gemini-callback/dynamic Note: LINE Messaging API's ImageMessage and AudioMessage strictly require URLs to be https://. In the Cloud Run architecture, HTTPS TLS is terminated at the outer Google Front End (GFE), so r.TLS is nil when it reaches the Go http.Request inside the container. If you don't read the X-Forwarded-Proto and X-Forwarded-Host headers, the assembled http:// URL will be rejected by the LINE Push API! Omission 5: Pursuing Minimalist Go Images (scratch / distroless) but Forgetting External Executables and Chromium Sandbox Restrictions When writing Go services, we often use FROM gcr.io/distroless/static to compress the Docker Image to under 20MB. But doing this for this multi-functional LINE Bot will immediately hit a wall: 1. Missing ffmpeg: After receiving PCM from Gemini for TTS synthesis, calling exec.Command("ffmpeg", ...) to convert to .m4a will report executable file not found. 2. Missing Headless Chromium, Chinese Fonts, and Container Sandbox Flags: The 3rd layer Fallback of the crawler uses single-file to drive Headless Chromium. In Cloud Run's container sandbox environment, if you don't add --no-sandbox, --disable-gpu, and --disable-dev-shm-usage (because the container's default /dev/shm shared memory is small) to Chromium, or if the base image doesn't have fonts-noto-cjk installed (resulting in garbled text on captured Chinese web pages) and ca-certificates (HTTPS certificate verification failure), the crawler will silently crash in the cloud. Therefore, our Dockerfile uses golang:1.26-bookworm for compilation, and the Runtime stage uses debian:bookworm-slim with explicit installation of ca-certificates, ffmpeg, chromium, fonts-noto-cjk, and @single-file/cli. Omission 6: State Splitting Between In-Memory Caching and Cloud Run Auto-scaling (Multi-Instance) Although we have persisted the four core states—chat history, bookmarks, research reports, and Batch Job mappings—to Cloud Firestore, some binary data with a "lifecycle of only 5 minutes" is stored in a Go in-memory map: - Images just uploaded by users, waiting for the "Identify Image / Agentic Vision" Quick Reply selection (imageTempStore). - PNGs just drawn with bounding boxes, waiting for the LINE server to pull them (/images/{image_id}). - M4A audio files just synthesized, waiting for the LINE server to pull them (/audio/{audio_id}). This works perfectly under a single instance, but once Cloud Run scales to 2 or more instances due to a burst in traffic, state splitting occurs: - User sends an image to Instance A (image stored in A's memory). - 3 seconds later, the user clicks the "Agentic Vision" button, and the Webhook hits Instance B $\rightarrow$ B can't find the image and replies "⚠️ Image expired, please resend!" - Or Instance A generates /audio/{id} and calls the LINE Reply API, but when the LINE media server comes to download the audio, it hits Instance B, gets a 404, and the user only sees an unplayable voice bubble in the chat room. Solution: Since a single Go instance can easily handle high concurrency (we set containerConcurrency to 160), for personal or small-to-medium assistants, it's recommended to limit --max-instances to 1 (if multi-instance high availability is needed in the future, the caching for /images and /audio must be rewritten to Cloud Storage or Firestore). Omission 7: Only Changing the Messaging API Webhook During Switching, but Missing Firebase Hosting WebSocket Restrictions and LIFF Endpoint When preparing to switch from the old service to the new one, the following three peripheral settings are most commonly missed: | Checkpoint | Why is it easily missed? | Actual Impact and Correct Practice | | --- | --- | --- | | 1. Is the existing Webhook hidden behind Firebase Hosting? | When querying existing settings via LINE API (GET /v2/bot/channel/webhook/endpoint), we found the original Webhook pointed to *.firebaseapp.com (rewritten to Cloud Run by firebase.json). | Firebase Hosting does not support WebSocket proxying! If the LIFF voice assistant (/ws/voice/{id}) goes through the Firebase Hosting domain, the WebSocket handshake will fail. Therefore, when switching, point directly to the Cloud Run URL or ensure LIFF uses a direct connection. | | 2. LIFF Endpoint URL of the LINE Login Channel | LIFF App settings are not in the Messaging API Channel but in the LINE Login Channel tab under the same Provider. | If you only change the Webhook and not the LIFF Endpoint, the chat room text will be handled by the new Go version, but opening the voice assistant will still connect back to the old Python version. | | 3. Gemini Batch API Static Webhook / Eventarc | If Eventarc or a project-level static Webhook is set in GCP to point to /api/gemini-callback/static. | Remember to update the target URL as well (while the Dynamic Webhook /api/gemini-callback/dynamic will automatically capture the Host of the incoming request, requiring no manual modification). | - * * 6. Inventory Before Switching: Identifying Three "Zombie Endpoints" from GCP Logs and Planning to Incorporate n8n Schedules! While writing the "Switching from Python Version" steps in the README, I originally listed: > "Change external schedules calling /hn, /hf, and /urls to point to the new service." But then I thought: Wait, are these three endpoints actually still being called by my n8n? We immediately used gcloud logging read to pull execution records for the linebot-helper-python and n8n services in the GCP project, and found a big surprise: - /hn, /hf, and /urls were endpoints written in the first version in November 2024, intended for external schedules to POST URL arrays for the LINE Bot to summarize and push. - But the n8n service currently in the cloud (running four active workflows: "RSS to Sheet", "Daily Tasks (Fetch Papers/Share Videos)", "Daily LINE Sender", and "AI Trends Automated Newsletter") had already been changed to run Firecrawl internally in n8n, call Gemini for summaries, assemble Flex JSON, and directly hit https://api.line.me/v2/bot/message/push! - In other words, the number of calls to /hn, /hf, and /urls on linebot-helper-python from n8n over the past few months was 0! Even more interestingly, while checking n8n's Debug Logs, we accidentally caught n8n failing to push YouTube new video summaries at 6 AM because of word count or thumbnail format issues in the Flex Carousel assembled by the Javascript node, with the LINE API returning: Workflow execution finished with error: A message (messages[0]) in the request body is invalid This discovery led us to make two immediate decisions: Decision 1: Immediately remove /hn, /hf, and /urls from linebot-helper-go Since no external services are calling them, keeping them on the public Cloud Run only adds three attack surfaces without signature protection. After removing them from server.go and redeploying, hitting /hn, /hf, or /urls online returns 404 Not Found. When switching, I only need to change the Webhook URL and LIFF Endpoint URL in the LINE Developers Console! Decision 2: Open Issue #14 ~ Issue #19 to incorporate n8n workflows into linebot-helper-go! Since linebot-helper-go is already a high-speed Go server always resident at 1 vCPU / 1 GiB (--no-cpu-throttling), and it has a built-in three-layer crawler loader.Loader, strongly-typed LINE Flex Builder, Gemini Batch API, Firestore BookmarkService, and ReportStore, why maintain a separate n8n + Postgres database? More importantly—if these daily schedules are all moved to linebot-helper-go and pushed through the same LINE Official Account, a perfect closed loop of "Push $\rightarrow$ Interaction" can be achieved: - Issue #14 (Batch Task 1/5): Establish a protected POST /api/tasks/{task_name} schedule endpoint and a Firestore feed_items deduplication queue (replacing the old method of comparing rows in Google Sheets), unifying pushes using the main official account. - Issue #15 (Batch Task 2/5): Fetch Hugging Face Daily Papers every day at 10:00. When pushing paper summary cards, directly attach "🔖 Save Bookmark" and "📄 Detailed Research Report" buttons, allowing one-click generation of Grounding deep web reports for good papers. - Issue #16 (Batch Task 3/5): Inspect YouTube subscription channel new videos every day at 06:00. Use Go's strongly-typed Flex Builder to completely solve n8n's 400 Invalid Request Body issue, and attach a "📝 Detailed Summary" Postback button directly to the card. - Issue #17 (Batch Task 4/5): Integrate multi-source RSS inspection from HN, Hatena IT, Google Dev Blog, LY Blog, Steam, etc., with daily 06:00 featured pushes. - Issue #18 (Batch Task 5/5): Aggregate weekly featured content and bookmarks every Friday at 17:00, rendering them directly into a beautiful /reports/{id} web version AI Trends Weekly pushed to LINE. - * * 7. One Last Security Reminder: Don't Leave Real Cloud Run URLs in Public Git History After writing the deployment records and switching tutorial, I did one very important cleanup task: replaced all occurrences of .run.app real service URLs, GCP Project Numbers, and LIFF IDs in README.md and GitHub Issue comments with <YOUR_CLOUD_RUN_SERVICE_URL> placeholders, and rewrote the Git Commit history to eliminate traces. The reason is simple: The LINE Bot's Cloud Run service must be set to --allow-unauthenticated to receive Webhooks from the LINE platform. Although our POST / has strict X-Line-Signature verification (requests without a Channel Secret signature are blocked with a 400), writing the real .run.app URL directly in a public GitHub Repo's README still easily attracts unnecessary automated crawler scans and traffic noise. Sharing architecture and implementation with public code while keeping real endpoints only in your own configuration backend is the most reassuring way to maintain a personal AI assistant in the long run. - * * 8. Cloud Run Deployment and Migration Self-Checklist If you also plan to rewrite an existing Python LINE Bot into Go and move it to Cloud Run, you can use this checklist to tick off items before switching traffic: - <input type="checkbox" disabled> 1. CPU Throttling Setting: As long as there are background Goroutines (Webhook returns 200 OK followed by background push, or background periodic Ticker), be sure to add --no-cpu-throttling (and set CPU to at least 1). - <input type="checkbox" disabled> 2. Context Lifecycle Decoupling: Background Goroutines are prohibited from passing in r.Context(); always use context.Background() with an independent context.WithTimeout. - <input type="checkbox" disabled> 3. Vertex AI global Domain Judgment: If GOOGLE_CLOUD_LOCATION=global, confirm the Endpoint Base URL is https://aiplatform.googleapis.com instead of global-aiplatform.googleapis.com. - <input type="checkbox" disabled> 4. External Binary Tools and Fonts: If ffmpeg, single-file, or Headless Chromium are called inside the container, confirm the final stage Docker Image includes the corresponding executables, ca-certificates, fonts-noto-cjk, and Chromium's --no-sandbox --disable-dev-shm-usage parameters. - <input type="checkbox" disabled> 5. Reverse Proxy HTTPS Headers: When dynamically generating /images/{id}, /audio/{id}, and /reports/{id} URLs, prioritize reading X-Forwarded-Proto and X-Forwarded-Host to ensure media URLs sent to LINE are always https://. - <input type="checkbox" disabled> 6. In-Memory Caching and Instance Count: If there are short-lived memory maps (like cached images/audio), it's recommended to set --max-instances=1 and increase concurrency, or switch to GCS / Firestore to avoid state splitting across multiple instances. - <input type="checkbox" disabled> 7. Dual-channel LINE Setting Synchronization: In addition to updating the Messaging API Webhook URL, don't forget to update the LIFF Endpoint URL of the LINE Login Channel under the same Provider, and confirm the WebSocket path is not intercepted by a CDN / Firebase Hosting that doesn't support WS. - <input type="checkbox" disabled> 8. Clean Up Obsolete Endpoints and Public Information: Confirm useless old endpoints (like /hn, /hf, /urls) from Cloud Logging and remove them, and ensure the public Repo and Git history do not contain real .run.app URLs and keys. - * * Summary and Open Source Project Links From this rewriting journey from kkdai/linebot-helper-python to kkdai/linebot-helper-go, here are the most valuable takeaways: 1. Open Issues Before Writing Code: 11 milestone Issues made the large-scale language rewrite highly controllable, with each module independently passing unit tests before being layered. 2. Shared Data Layer is the Foundation for Zero-Downtime Migration: As long as the Firestore Collection Schemas are aligned, services in the two languages can be switched painlessly at any time. 3. Go's Resource Efficiency on Cloud Run is Extremely High: Even with Headless Chromium and ffmpeg mounted, switching from Python's 4 vCPU / 4 GiB to Go's 1 vCPU / 1 GiB cut the bill to 25%, with even faster response times. 4. Check Cloud Logging Before Switching: Real traffic logs don't lie; you can see at a glance which endpoints are already retired and which external n8n workflows are actually better suited to be incorporated into the same service. The complete source code and subsequent ADK v1.9.0 / Batch Tasks evolution plans are all open-sourced on GitHub: kkdai/linebot-helper-go. Feel free to refer to it and exchange ideas!

[Agent Platform in Action (7)] When ADK Go Graph Workflow Meets Google Research Harness (Part 1): Two-Layer Recursive Agent Fram !Harness² Part 1 Architecture: Current Architecture vs c1..c8 Editable Harness Tree Foreword: After the AI Agent Goes Online, What is the Next Bottleneck? Welcome back to the "Agent Platform in Action" series! In the previous six articles, we took the LINE AI Card Assistant (kkdai/line-card-assistant) through the complete five stages from a standalone prototype to production-grade cloud deployment: 1. [[Agent Platform in Action (1)] Building LINE AI Card Assistant with Go and Google ADK 2.0 Graph Workflow](/2026/10/16/adk-graph-workflow-go/): Using google.golang.org/adk/v2/workflow to build a 13-node directed graph featuring Vision OCR, AddFanOut / AddFanIn parallel background investigation, and NewRequestInputEvent HITL interaction loops. 2. [[Agent Platform in Action (2)] From Monolithic Cloud Run to Google Cloud Agent Runtime](/2026/10/17/adk-graph-workflow-go-2/): Completed Phase 1 (PR #10), standardizing the project into an agents-cli (adk_go) dual-channel architecture. 3. [[Agent Platform in Action (3)] Implementing Cross-Instance Business Card Recognition and Pause/Resume with ADK 2.0 GCS Artifact Service and Agent Platform Sessions](/2026/10/20/adk-graph-workflow-go-3/): Completed the first half of Phase 2 (PR #11), solving the HITL "amnesia" of stateless containers and the issue of binary image data explosion. 4. [[Agent Platform in Action (4)] Completing the Architecture Puzzle with Cloud Run LINE Webhook Gateway, Agent Runtime, and Cloud SQL PostgreSQL](/2026/10/21/adk-graph-workflow-go-4/): Completed the second half of Phase 2 & Phase 3 (PR #12), upgrading the business card database to Cloud SQL PostgreSQL and achieving two-layer stateless decoupling. 5. [[Agent Platform in Action (5)] Establishing Automated Evaluation Benchmarks with agents-cli eval and ADK 2.0 Go](/2026/10/22/adk-graph-workflow-go-5/): Completed Phase 4 (PR #13), establishing a golden test set and a quality flywheel. 6. [[Agent Platform in Action (6)] Building a Full-Stack Observable LINE AI Card Assistant with agents-cli deploy, Terraform IaC, GitHub Actions CI/CD, and Cloud Trace](/2026/10/24/adk-graph-workflow-go-6/): Completed Phase 5 (PR #14), adding OpenTelemetry Spans (workflow.node.*) to all 13 Graph Nodes and deploying to the cloud via Terraform and GitHub Actions. Additionally, we have planned advanced Graph Workflow primitives for future ADK v2.6+ (such as Parallel dynamic dispatch, MultiRoute multi-way fan-out, reflection-repair loops, and strongly typed nodes) into the Future Roadmap Issue #15. However, now that our Agent possesses Phase 4 Automated Evaluation and Phase 5 Full-Node Tracing (OpenTelemetry Spans), a deeper question emerges: > When evaluations find that the Agent performs poorly in certain real-world cases (e.g., Japanese-English bilingual cards, searching for colloquial nicknames in the Taiwan tech industry, multi-clause colloquial HITL modifications, or financial compliance icebreaker letters), must engineers manually check logs, manually modify Prompt strings in the Go source code, and then recompile and redeploy every single time? Can we let a Meta-Agent (Engineer Agent) automatically read our evaluation results and OpenTelemetry execution trajectories, diagnose the reasons for lost points, write new Skills and Long-Term Memory itself, and complete a closed-loop self-evolution? This is precisely the core proposition addressed by Google Research's recently published and open-sourced google-research/harness2 (_Harness²: Recursive Harnessing for LLM Agents_, arXiv:2610.14224)! This series will consist of three articles (corresponding to Issue #16, Issue #17, Issue #18, and PR #19), guiding you through the complete integration of Google Research Harness² into our Go ADK 2.0 project: - Part 1 (This article, Issue #16): What is Google Harness²? What can it do? Inventorying the existing architecture before modification and decoupling hardcoded configurations in the Go binary into a c1..c8 editable Harness file tree. - Part 2 (Issue #17): Encapsulating Phase 4's eval/datasets/ and Phase 5's OpenTelemetry Spans into a LineCardAdapter for Harness², and analyzing why "Zero-Leakage Barriers" and "High-Resolution Behavioral Trajectories" are both indispensable. - Part 3 (Issue #18): Complete testing tutorial, disclosure of Parallel vs Sequential + LongTermMEMORY two-layer recursive experimental data (accuracy jumping from 62.5% to 100.0%), and outlook on the future development of Agents. - * * 1. What is Google Research Harness²? What can it do? 1. From "Fine-tuning Model Weights" to "Optimizing Agent Harnessing" In the era of Large Language Models (LLMs), the success of an AI Agent is often determined not just by the underlying model (like Gemini 2.5 Flash / Pro) itself, but by the "Agent Harness" layer wrapped around the model—namely, System Prompts, formatting and safety Guardrails, Long-Term Memory, Tool Descriptions, Sub-Agent settings, and On-Demand Skills. Past automated prompt optimizations (like PromptBreeder, TextGrad, or DSPy) mostly targeted mutations of a single Prompt string. However, in a real enterprise-grade Agent, if you stuff all edge-case patching rules into the same System Prompt, it quickly leads to "Context Bloat": the number of Always-On Tokens increases, which not only squeezes the model's Output Token Budget but also causes different rules to interfere with each other! Google Research's Harness² (_Recursive Harnessing for LLM Agents_, arXiv:2610.14224) breaks this limitation. It defines the entire Agent's equipment as a structured c1..c8 Editable Component Tree and automatically evolves this file tree through Two-Level Recursion: flowchart TB subgraph Outer["Outer Recursion: Harnessing-Level Recursion (Experience accumulation across rounds k=1..K)"] Reflector["Experience Reflector\n(ReflectorPrompt)\nDistilling success and failure patterns across rounds"] Memory["c3: LongTermMEMORY.md\nPersisting domain experience across iterations"] Reflector -->|Write long-term rules| Memory end subgraph Inner["Inner Recursion: Task-Level Recursion (Closed-loop diagnosis and patching within a single round)"] Propose["1. Propose (Diagnostic Proposal)\nRead trajectory.md\nLocate behavioral defects and propose N candidate Patches"] Probe["2. Probe (Sandbox Probing)\nExecute N candidate Harnesses in parallel\nCompare trajectory and score changes before and after modification"] Compose["3. Compose (Conflict Resolution and Synthesis)\nRetain positive-gain Patches, discard negative interference\nSynthesize the next-generation best Harness H^(k)"] Propose --> Probe --> Compose end Memory --> Propose Compose --> Reflector 2. The c1..c8 Eight Harness Components and the 24KB Budget Limit In google-research/harness2/harness.py, a standard Agent Harness consists of 4 Always-On Components (c1..c4) and 4 On-Demand Components (c5..c8): | Component Code | File Path Rule | Loading Mode | Responsibility in Harness² | | --- | --- | --- | --- | | c1 | systemprompt.md | Always-On | Defines core roles, node responsibilities, and indexing rules for triggering c6 skills (Skill Trigger Lines). | | c2 | guardrails.md | Always-On | Defines non-violable output formats (e.g., pure JSON constraints), E.164 normalization invariants, and security/PII guardrails. | | c3 | LongTermMEMORY.md | Always-On | Domain lessons and industry compliance memories written across rounds by the outer Experience Reflector. | | c4 | tool_descriptions/*.md | Always-On | Tool parameter descriptions, search strategies, and normalization details (e.g., Japan long-distance prefix (0) cleaning rules). | | c5 | sub_agents/<name>/AGENT.md | On-Demand | Exclusive Instructions and YAML Frontmatter (mode: subagent) for delegated sub-agents (e.g., company_enrich). | | c6 | skills/<name>/SKILL.md | On-Demand | Professional skills dynamically injected only for specific domains or tasks (e.g., Japanese card OCR, Hsinchu Science Park colloquial search expansion). | | c7 | scripts/*.py | On-Demand | Auxiliary verification or data conversion scripts. | | c8 | plugin/*.ts | On-Demand | Runtime lifecycle and OpenTelemetry diagnostic hooks. | > [!IMPORTANT] Why distinguish between c1..c4 (Always-On) and c5..c8 (On-Demand)? Harness² sets an iron rule in OPENCODE_SURFACE: The total byte count of c1..c4 always-on files must never exceed 24,000 bytes (AlwaysOnMaxBytes = 24000)! When Harness² finds that a failure case requires lengthy domain knowledge (e.g., a department title mapping table for Japanese corporate cards or a colloquial abbreviation dictionary for Taiwan semiconductor parks), it won't force these rules into systemprompt.md causing Context explosion. Instead, it will automatically create an on-demand c6 skill file (skills/<name>/SKILL.md) and leave only a lightweight Trigger Line in systemprompt.md! 3. How Does Two-Level Recursion Work? The Harness² evolution engine (harness2/core/recursion.py) includes two levels of recursion: 1. Level 1 (Inner): Task-Level Recursion (Propose $\rightarrow$ Probe $\rightarrow$ Compose): - Propose (Diagnostic Proposal): The optimizer (Harvester LLM) observes tasks that lost points in the benchmark, the final output (Deliverable), and the execution trajectory (trajectory.md). It diagnoses which step the Agent made a wrong decision on and proposes multiple independent candidate patches (Candidate Patches, each being a set of additions or modifications to c1..c8). - Probe (Sandbox Probing): The system mounts each candidate Patch into a sandbox and runs the task once, collecting new scores and new execution trajectories to form a "Before vs. After" Behavioral Contrast. - Compose (Conflict Resolution and Synthesis): The optimizer compares the actual performance of all candidate Patches, merges modifications that truly bring positive score improvements into the new generation Harness $H^{(k)}$, and discards ineffective or side-effect-inducing modifications. 2. Level 2 (Outer): Harnessing-Level Recursion (Experience Reflector): - Between multiple iterations ($k = 1 \dots K$), the outer ReflectorPrompt reviews the evolution history from $H^{(0)} \to H^{(1)} \to \dots \to H^{(k)}$: Which types of modifications succeeded? Which cross-domain constraints (e.g., financial industry compliance headers and personal data protection regulations) need to be remembered at a global level? - Distilled high-level experiences are written into c3: LongTermMEMORY.md, guiding the next round of Propose and Agent execution. - * * 2. Inventorying the Current Architecture: Why Phase 1~5 Go Implementation Cannot Be Directly Optimized by Harness²? Let's compare the left side (Before: Hardcoded Go Binary Architecture) of the architecture diagram at the beginning of the article and review the system architecture we completed in Phase 1 to Phase 5: flowchart LR subgraph Before["Before: Phase 1~5 Solid Compilation Architecture (Hardcoded Go Binary)"] direction TB N1["nodes_ingest.go\nHardcoded VisionOCR / CleanAndDedup /\nCompanyEnrich / ApplyEdit Prompts"] N2["nodes_search.go\nHardcoded QueryIntent /\nEmailDraft Prompts"] N3["graph.go\nFixed SubAgent Instruction"] E["eval/datasets/basic-dataset.json\nPhase 4 Golden Test Set"] T["telemetry.go\nPhase 5 OpenTelemetry 13-Node Spans"] end Problem["Pain Points:\n1. Prompts hardcoded in .go string constants; re-go build required for every adjustment\n2. Lack of on-demand c6 Skills mechanism\n3. Phase 4 Eval and Phase 5 OTel Spans not yet connected into an automated optimization loop"] Before --> Problem In the code before modification: 1. All Prompt templates for VisionOCRNode, CompanyEnrichNode, ApplyEditNode (located in internal/workflow/nodes_ingest.go), as well as QueryIntentNode and EmailDraftNode (located in internal/workflow/nodes_search.go), are hardcoded as fmt.Sprintf(...) string constants in the Go code. 2. The llmagent.Config{Instruction: ...} for CompanyEnrichAgent (located in internal/workflow/nodes_ingest.go) is also a fixed string hardcoded in a Go function. 3. When we want the Harness² Python evolution loop to modify system prompts, add a Japanese business card recognition skill (skills/jp_card_ocr/SKILL.md), or adjust phone number normalization guidelines (tool_descriptions/tool_guidance.md), it is impossible for Harness² to rewrite our .go source code and go build again for every single task! Therefore, the first step in introducing Harness² (Issue #16) is to completely decouple the Go compilation phase from the Harness equipment phase: - Go Binary is responsible for "Graph Topology and Dual-Channel Execution Engine": RouterNode, parallel Fan-Out/Fan-In, HITLConfirmNode pause/resume, Cloud SQL and GCS Artifact access, OpenTelemetry instrumentation. - External c1..c8 Harness file tree is responsible for the "Evolvable Brain and Skill Equipment": Dynamically hot-reloaded at runtime via the environment variable LINE_CARD_HARNESS_DIR! - * * 3. Implementation Part 1: Building internal/harness Loader and harness/base/ Initial Equipment Tree To make the Go ADK 2.0 Graph Workflow natively understand the Harness² c1..c8 specification, we added two core directories to the project: - harness/base/: Stores the initial unevolved ($k=0$) c1..c8 baseline file tree. - internal/harness/harness.go: Responsible for loading the c1..c8 file tree, validating OPENCODE_SURFACE 24KB limits and YAML Frontmatter rules, and dynamically combining always-on context with on-demand skills for each Graph Node. 1. Establishing the harness/base/ Baseline c1..c8 File Tree We extracted the basic settings previously scattered in nodes_ingest.go and nodes_search.go into the harness/base/ directory: harness/base/ ├── systemprompt.md # c1: Core roles and overview of Graph Node responsibilities ├── guardrails.md # c2: JSON output format and phone normalization guardrails ├── LongTermMEMORY.md # c3: Cross-round long-term memory (initially empty, to be written by Reflector) ├── tool_descriptions/ │ └── tool_guidance.md # c4: Guidance for google_search and format_company_profile ├── sub_agents/ │ └── company_enrich/ │ └── AGENT.md # c5: CompanyEnrichAgent delegated sub-agent definition ├── skills/ # c6: On-demand skills directory (to be generated by Propose-Probe-Compose evolution) ├── scripts/ # c7: Auxiliary scripts directory └── plugin/ # c8: Diagnostic plugin directory For example, harness/base/sub_agents/company_enrich/AGENT.md uses a YAML Frontmatter format fully compatible with google-research/harness2: --- mode: subagent description: Searches Google to summarize company background and industry tags. --- You are a professional business analysis assistant. Please use the google_search tool to query the company name provided by the user, then call the format_company_profile tool to return a two-sentence Traditional Chinese company profile and 3 to 5 technical or industry tags (e.g., AI, Cloud, SaaS, FinTech, Semiconductor). 2. Implementing internal/harness/harness.go: 24KB Always-On Budget Validation and On-Demand Skill Injection In internal/harness/harness.go, we implemented the validator Validate() and node prompt synthesizer ComposeNodePrompt() fully aligned with the Python version of OPENCODE_SURFACE.validate(): // AlwaysOnMaxBytes matches google-research/harness2 OPENCODE_SURFACE.always_on_max (24,000 bytes). const AlwaysOnMaxBytes = 24000 // AlwaysOnFiles lists the four c1..c4 always-on files mounted into the agent context. var AlwaysOnFiles = []string{ "systemprompt.md", "guardrails.md", "LongTermMEMORY.md", "tool_descriptions/tool_guidance.md", } // Validate enforces Google Research Harness² OPENCODE_SURFACE invariants on the c1..c8 tree. func (h *Harness) Validate() error { var problems []string for _, reqFile := range AlwaysOnFiles { if strings.TrimSpace(h.Files[reqFile]) == "" { problems = append(problems, fmt.Sprintf("missing or empty always-on component %q", reqFile)) } } if total := h.AlwaysOnBytes(); total > AlwaysOnMaxBytes { problems = append(problems, fmt.Sprintf( "always-on c1..c4 byte count %d exceeds cap %d (every token here competes with the agent's output budget)", total, AlwaysOnMaxBytes, )) } combinedAlwaysOn := h.SystemPrompt + "\n" + h.Guardrails for dirName, skill := range h.Skills { if skill.Name != dirName { problems = append(problems, fmt.Sprintf("%s frontmatter name %q must match directory name %q", skill.RelPath, skill.Name, dirName)) } if strings.TrimSpace(skill.Description) == "" { problems = append(problems, fmt.Sprintf("%s must include a non-empty description in YAML frontmatter", skill.RelPath)) } if !strings.Contains(combinedAlwaysOn, dirName) { problems = append(problems, fmt.Sprintf("skill %q is never referenced in systemprompt.md or guardrails.md; add a trigger line naming `%s`", dirName, dirName)) } } // ... } There are two very critical engineering details here: 1. Mandatory Skill Trigger Reference Check: If Harness² adds a skill in skills/jp_card_ocr/SKILL.md but forgets to write when to trigger jp_card_ocr in systemprompt.md or guardrails.md, Validate() will directly report an error and intercept it, avoiding the creation of "Orphan Skills" that will never be called. 2. Node-Level Dynamic Skill Matching (MatchSkills + ComposeNodePrompt): When the Graph Workflow executes VisionOCRNode, CompanyEnrichNode, ApplyEditNode, QueryIntentNode, or EmailDraftNode, it only injects c6 skills related to that node and automatically attaches adk.harness.skills and adk.harness.always_on_bytes attributes to the OpenTelemetry Span: // composeHarnessPrompt enriches a node's base prompt with the active Harness² c1..c8 tree // and records the active harness metadata onto the node's OpenTelemetry span. func composeHarnessPrompt(span trace.Span, nodeName, basePrompt, taskInput string) (string, *harness.Harness) { h := harness.LoadActive() composed, skills := h.ComposeNodePrompt(nodeName, basePrompt, taskInput) if span != nil { span.SetAttributes( AttrHarnessRoot.String(h.RootDir), AttrHarnessAlwaysOnBytes.Int(h.AlwaysOnBytes()), ) if len(skills) > 0 { span.SetAttributes(AttrHarnessSkills.StringSlice(skills)) } } return composed, h } We can directly use the CLI to verify the always-on byte count and structural validity of harness/base: $ go run ./cmd/harness-runner --validate-only --harness-dir=harness/base OK (1599 always-on bytes) - * * 4. Summary and Next Article Preview In this article (Part 1), we completed the goals of Issue #16: 1. Gained a deep understanding of how Google Research Harness² (arXiv:2610.14224) automatically evolves Agent capabilities through a c1..c8 editable file tree and two-level recursion (Task-level Propose-Probe-Compose + Harnessing-level Experience Reflector), without recompiling the main program or bloating the always-on context. 2. Inventoried the architectural bottlenecks of Phase 1~5 where prompts and normalization logic were hardcoded in the Go binary. 3. Implemented internal/harness/harness.go and harness/base/, allowing the 13 nodes of the ADK 2.0 Go Graph Workflow to support dynamic mounting of the c1..c8 equipment tree via LINE_CARD_HARNESS_DIR. With a hot-reloadable c1..c8 equipment tree in place, the next article [[Agent Platform in Action (8)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 2): Turning Phase 4 Eval Datasets and Phase 5 OpenTelemetry Spans into LineCardAdapter](/2026/10/26/adk-graph-workflow-go-8/) will enter Issue #17: revealing why we need to encapsulate Phase 4 test sets and Phase 5 OpenTelemetry Spans into a LineCardAdapter, and how the "Zero-Leakage Barrier" prevents the AI optimizer from cheating!

[Agent Platform in Practice (2)] From Monolithic Cloud Run to Google Cloud Agent Runtime: Refactoring LINE AI Business Card King !LINE Card Assistant + Google Cloud Agent Platform Architecture Recap In the previous article, "[[Agent Platform in Practice (1)] Building a LINE AI Business Card Assistant with Go and Google ADK 2.0 Graph Workflow](/2026/10/16/adk-graph-workflow-go/)", we used Golang (google.golang.org/adk/v2/workflow) and the LINE Messaging API SDK v8 to build line-card-assistant (LINE AI Business Card Assistant) from scratch. We successfully implemented and deployed the four core capabilities of ADK 2.0 Graph Workflow—Deterministic Router, AddFanOut / AddFanIn for parallel background investigation and aggregation, and NewRequestInputEvent for Human-in-the-Loop (HITL) pause and loop correction—to Google Cloud Run. However, as we evolve this monolithic LINE Bot into a Production-grade Agent Platform, we immediately encounter several architectural questions: 1. Where is the cross-instance HITL Session state stored? In the first version, we used session.InMemoryService() with local SQLite. When Cloud Run switches container instances due to auto-scaling or redeployment, the HITL pause state (Checkpoint) waiting for the user to click "✅ Confirm Save" disappears due to memory reset. 2. How can this business card Agent be called by other Agents (Agent-to-Agent, A2A)? If the company has another "CRM Sales Assistant Agent" or "Calendar Scheduling Agent" that wants to query contacts in our Business Card Assistant or ask it to write an icebreaker email via a standard A2A protocol, a monolithic server tied to the LINE Webhook /webhook entry point cannot directly provide a standard Agent interface. 3. How to systematically evaluate and observe the entire Graph? After changing a line of Prompt or adjusting a node, how do we ensure that OCR accuracy, Fan-Out parallel tool call trajectories (Tool Trajectory), and search quality haven't degraded? To bridge the gap between "writing an Agent prototype" and "operating an enterprise-grade Agent," Google officially launched a new command-line and skill toolkit—agents-cli (Google Agents CLI), serving as the unified development entry point for the Google Cloud Agent Platform (now integrated as Gemini Enterprise Agent Platform). In this second episode of the series, we will dive deep into the agents-cli official development guide, answer frequently asked questions about Agent Runtime scaling (Scale-to-Zero) and billing mechanisms, and walk you through how we completed the Phase 1 architecture upgrade (PR #10): refactoring the project into a dual-channel architecture that complies with adk_go + agent_runtime specifications without breaking the original LINE Bot functionality! * 1. In-depth Analysis of Google Cloud Agent Platform and the agents-cli Official Guide Before diving into the code, let's clarify Google's latest technical landscape for the Agent Platform through two core documents: "Getting Started - agents-cli" and "The Lifecycle - agents-cli". 1. Product Terminology Comparison: What are Agent Platform and Agent Runtime? If you've followed Google Cloud's AI product line over the past year, you might have heard of Vertex AI Agent Engine, Reasoning Engine, Agent Platform, or Gemini Enterprise Agent Platform (GEAP). In the latest agents-cli v1.10+ specifications, the official documentation has clearly converged these terms and underlying resources: | Common Developer Terms | Google Cloud Official Positioning & Underlying Resources | agents-cli Corresponding Parameter | | --- | --- | --- | | Agent Platform / Gemini Platform | Google Cloud's overall platform for hosting enterprise-grade AI Agents (covering Runtime, Sessions, Memory Bank, Agent Gateway, Agent Registry, and Eval) | Unified operation via agents-cli | | Agent Runtime / Agent Engine | A fully managed container execution environment designed specifically for AI Agents. Underlying GCP resource path: projects/{project}/locations/{location}/reasoningEngines/{id} | --deployment-target agent_runtime | | Agent Platform Sessions | A cloud-managed conversation and state persistence service (ADK automatically binds VertexAiEngineSessionService) | --session-type agent_platform_sessions | | Gemini Enterprise / Agent Registry | An enterprise Agent publishing and directory center, allowing deployed Agents to be discovered and called directly in the Gemini Enterprise interface or via A2A | agents-cli publish gemini | * 2. The Seven-Stage Development Lifecycle of agents-cli In the past, we were used to a "write code → go test → gcloud run deploy" workflow. However, in "The Lifecycle" document, agents-cli defines a Seven-Stage Quality Flywheel that integrates Coding Agent Skills (supporting Antigravity CLI, Gemini CLI, Claude Code, etc.): flowchart LR P0["Phase 0-1\nUnderstand & Recipes\n(.agents-cli-spec.md)"] --> P2["Phase 2\nScaffold\n(create / enhance)"] P2 --> P3["Phase 3\nBuild & Smoke Test\n(run / playground)"] P3 --> P4["Phase 4\nEvaluate\n(eval run Quality Flywheel)"] P4 -->|Iterative Correction| P3 P4 --> P5["Phase 5\nDeploy\n(agent_runtime / cloud_run)"] P5 --> P6["Phase 6-7\nPublish & Observe\n(Cloud Trace / BigQuery)"] 1. Phase 0 & 1 — Understand & Study Recipes: Before writing a single line of code, document the Agent's core goals, external tools, security boundaries, and deployment targets in the project root's .agents-cli-spec.md as the Single Source of Truth for the entire development cycle. 2. Phase 2 — Scaffold: Use agents-cli scaffold create or run agents-cli scaffold enhance . on an existing project to automatically generate the directory structure, configuration files (agents-cli-manifest.yaml), A2A endpoints, Session switchers, and OpenTelemetry observation modules that comply with official standards. - Supports two major official built-in templates: adk (Python) and adk_go (Golang)! 3. Phase 3 — Build & Playground: - Go projects only need to focus on implementing and exporting NewRootAgent(ctx) in app/agent.go. - During development, you can use agents-cli run "your test question" for a quick terminal smoke test, or run agents-cli playground (or make playground) to open a browser-based visual interactive interface for multi-turn conversation testing. 4. Phase 4 — Evaluate (The core of the Quality Flywheel): The official guide emphasizes: "Unit Test (go test) measures code contracts and type correctness; while agents-cli eval measures Agent behavior quality (Tool Trajectory, response quality, and safety)." 5. Phase 5 — Deploy: Use agents-cli deploy to package and deploy the project to agent_runtime, cloud_run, or gke, and automatically record the deployed resource ID in deployment_metadata.json. 6. Phase 6 & 7 — Publish & Observe: Register to the enterprise directory via agents-cli publish gemini, and combine with OpenTelemetry to automatically export all Graph node latencies, LLM Prompt/Response, and Token consumption to Cloud Trace, Cloud Logging, and BigQuery Agent Analytics. * 3. Agent Runtime vs. Cloud Run vs. GKE: Which Deployment Target Should I Choose? agents-cli supports three containerized deployment targets (all of which build based on the Dockerfile in the project root, making Go a first-class citizen on all three platforms). We have summarized the comparison based on the official Deployment Decision Matrix: | Evaluation Dimension | Agent Runtime (agent_runtime) | Cloud Run (cloud_run) | GKE (gke) | | --- | --- | --- | --- | | Infrastructure Management | Fully managed (built for Agents), automatically built from Dockerfile and mounted to reasoningEngines | Containerized serverless hosting, requires manual configuration of network and service parameters | GKE Autopilot, provides full K8s control but with the highest operational complexity | | Session State Management | Native built-in Agent Platform Sessions (automatically injects GOOGLE_CLOUD_AGENT_ENGINE_ID at runtime and enables cloud persistent Sessions) | In-Memory by default, requires additional Cloud SQL setup or manual connection to managed Sessions | In-Memory by default, requires additional Cloud SQL setup or managed Sessions | | API & Access Interface | Built-in :query / :streamQuery, Console Playground, and /api/{container_path} HTTP Passthrough and A2A | Native public or IAM-protected HTTPS URL, best for receiving external Webhooks (like LINE / Slack) | Custom K8s Ingress / Gateway | | Enterprise Integration | Native support for Managed OAuth 2.0 User Consent, Agent Identity, Agent Gateway, and PSC-I Private Networking | Supports VPC Egress, IAP, but does not support managed Agent OAuth flows | Full K8s network control | | Billing Model | Billed based on actual vCPU-hours + Memory-hours used (no compute fees when scaled to 0) | Billed based on Request duration or Instance seconds | Billed based on K8s Node Pod resources | * 4. Key Clarification: Will Instances Moved to Agent Runtime Scale to Zero? When planning to migrate line-card-assistant to Agent Runtime, a common concern is: > "According to agents-cli specifications, will instances moved to Agent Runtime never scale to 0 and constantly burn money?" The answer is: Agent Runtime supports Scale to Zero, and agents-cli adopts a clever dual-mode design: "Scale to 0 by default for development, and pre-warm 1 instance by default for production!" According to the agents-cli official deployment specifications (Sizing a Deployment): 1. When using agents-cli deploy directly (Default Scale to Zero): - The CLI's default resource allocation is identical to Cloud Run: --cpu 1, --memory 4Gi, --concurrency 8, --min-instances 0, --max-instances 10. - This means when there is no traffic to the Agent Runtime, the number of instances will automatically scale down to 0. Since Agent Runtime is billed based on actual vCPU-hours and Memory-hours, no compute fees are generated when idling at 0! 2. When using Terraform (service.tf) for production deployment (Default min_instances = 1): - When you generate Terraform IaC (deployment/terraform/single-project/service.tf) for a production environment, the template defaults min_instances to 1 to avoid cold starts for the first user. - If you want to save money in production and can tolerate a few seconds of cold start, you can specify agents-cli deploy --min-instances 0 during deployment or change min_instances back to 0 in service.tf. 💡 Why is a "Dual-layer Decoupled Architecture" best for a LINE Bot with Agent Runtime? Understanding the characteristics of Agent Runtime, you'll find that splitting the system into two layers is a perfect match: - Outer Layer: LINE Webhook Gateway (Cloud Run, min-instances=0): Since the LINE server sends Webhooks with only an X-Line-Signature HMAC signature, it cannot carry a Google Cloud IAM OAuth Token to call an IAM-protected Agent Runtime API. Use a lightweight Go Cloud Run service dedicated to "verifying the signature within 1 second, returning HTTP 200, and triggering the LINE loading animation." It scales to 0 when no one is sending business cards. - Inner Layer: Core Brain (Agent Runtime, min-instances=0): Focuses on running the ADK 2.0 Graph Workflow, managing cloud persistent VertexAiEngineSessionService (ensuring HITL pause states aren't lost across containers!), and simultaneously exposing A2A (/.well-known/agent-card.json) and the Agent Engine Playground to the enterprise! * 2. Planning the Migration Blueprint: Creating a 5-Phase Roadmap Using GitHub CLI To complete this architectural evolution methodically, we used the gh CLI in the GitHub Repository (kkdai/line-card-assistant) to establish 5 Milestones (Phases), 8 implementation Issues, and an overview tracking list Roadmap Issue #9: - Phase 1: agents-cli Environment and adk_go Project Scaffolding (The focus of this article!) - <input type="checkbox" disabled checked>#1 Upgrade google-agents-cli to 1.10+ and create .agents-cli-spec.md and agents-cli-manifest.yaml - <input type="checkbox" disabled checked>#2 Introduce adk_go + agent_runtime standard launcher architecture (app/agent.go, A2A, /app-info, Sessions, and Observability) - Phase 2: ADK Go Graph Workflow Cloud Storage and Persistence Adaptation - <input type="checkbox" disabled>#3 Deepen ADK Go v2.6+ Graph Workflow and NewRootAgent state management - <input type="checkbox" disabled>#4 Integrate Agent Platform Sessions (session/vertexai) and GCS Artifact Service to support cross-instance HITL pause/resume - <input type="checkbox" disabled>#5 Expand business card database (CardStore) to support Cloud SQL / Firestore cloud persistent storage - Phase 3: LINE Webhook Gateway and Agent Runtime Decoupled Architecture - <input type="checkbox" disabled>#6 Implement LINE Webhook Gateway (Cloud Run) and Agent Runtime dual-layer decoupled call architecture - Phase 4: agents-cli eval Automated Evaluation and Benchmarking - <input type="checkbox" disabled>#7 Establish agents-cli eval evaluation datasets and quality flywheel (OCR, Fan-Out/Fan-In, HITL loops, Hybrid Search) - Phase 5: Agent Runtime Official Deployment, IaC, and Observability - <input type="checkbox" disabled>#8 Deploy to Agent Runtime via agents-cli deploy and establish Terraform IaC, CI/CD, and Cloud Trace observability * 3. Phase 1 Hands-on Breakdown: Transforming an Existing Go Project into an adk_go + agent_runtime Dual-Channel Architecture In Phase 1 (corresponding to the merged PR #10), our goal was to: retain the original cmd/server/main.go independent LINE Webhook server while introducing the standard agents-cli (adk_go + agent_runtime) architecture in the project root, allowing the same codebase to possess both superpowers! Let's look at the key files changed in this upgrade and the new features they bring: line-card-assistant/ ├── .agents-cli-spec.md # [New] Phase 0 Architecture Spec (Single Source of Truth) ├── agents-cli-manifest.yaml # [New] agents-cli v1.10.0 project config (adk_go + agent_runtime) ├── deployment_metadata.json # [New] Records remote Agent Runtime resource ID and A2A status ├── main.go # [New] Agent Runtime / ADK WebUI / A2A standard HTTP entry point ├── apploader.go # [New] Loader supporting /ui/?app=app and rootAgent.Name() dual aliases ├── appurl.go # [New] Automatically calculates Cloud Run / Agent Runtime public URL (for A2A Card) ├── observability.go # [New] OpenTelemetry Cloud Trace / Logging / Metrics auto-mounting ├── app/ │ ├── agent.go # [New] Standard export interface: NewRootAgent(ctx) returns 12-node Graph Workflow │ └── agent_test.go # [New] Verifies Root Agent execution and search output directly via ADK Runner ├── appinfo/ │ └── appinfo.go # [New] GET /app-info endpoint, auto-parses Root Workflow and embedded CompanyEnrichAgent ├── sessions/ │ └── sessions.go # [New] Auto-switches between VertexAiEngineSessionService and local InMemoryService ├── cmd/server/main.go # [Retained] Original independent LINE Bot Webhook server └── internal/workflow/ # [Enhanced] Graph Workflow nodes upgraded to support dual-channel input and streaming Content output * 1. Creating agents-cli-manifest.yaml and .agents-cli-spec.md To make agents-cli recognize a Go project, the key is the agents-cli-manifest.yaml in the root directory: # Managed by agents-cli. Do not edit manually. acli_version: "1.10.0" project_name: "line-card-assistant" root_agent_name: "line_card_assistant" base_template: "adk_go" language: "go" framework: "google-adk" deployment_target: "agent_runtime" agent_directory: "app" region: "asia-east1" agent_guidance_filename: "GEMINI.md" is_a2a: true prototype: true generated_at: "2026-10-17T16:12:00Z" After configuration, running agents-cli info in the project root allows the CLI to immediately identify this as a Go project that supports A2A and targets deployment to the asia-east1 Agent Runtime: $ agents-cli info CLI version: 1.10.0 Project root: /Users/evanlin/Documents/line-card-assistant Project name: line-card-assistant Root agent name: line_card_assistant Language: go Deployment target: agent_runtime Agent directory: app Region: asia-east1 A2A: yes * 2. Encapsulating the Standard Root Agent Entry Point: app/agent.go In the adk_go template specification, all Agent logic should be exported via NewRootAgent(ctx context.Context) (agent.Agent, error) under the app package. Since we already used ADK 2.0's workflowagent.New to encapsulate the 12-node Graph Workflow into a standard agent.Agent in the first article, in app/agent.go, we only need to initialize the underlying CardStore (SQLite or Memory) and model.LLM (auto-detecting Vertex AI ADC or Gemini API Key), then call workflow.NewCardAssistantAgent for a seamless connection! package app import ( "context" "log/slog" "os" "strings" "github.com/kkdai/line-card-assistant/internal/config" "github.com/kkdai/line-card-assistant/internal/store" "github.com/kkdai/line-card-assistant/internal/workflow" "google.golang.org/adk/v2/agent" "google.golang.org/adk/v2/model" ) const RootAgentName = workflow.WorkflowName // "line_card_assistant" // NewRootAgent constructs the root ADK v2 Graph Workflow agent for Agent Runtime, // ADK WebUI playground, A2A server, and evaluation CLI. func NewRootAgent(ctx context.Context) (agent.Agent, error) { cfg := config.Load() var cardStore store.CardStore if strings.TrimSpace(cfg.SQLitePath) != "" { sqliteStore, err := store.NewSQLiteStore(cfg.SQLitePath) if err != nil { slog.Warn("failed to open SQLite store, falling back to in-memory store", "err", err) cardStore = store.NewMemoryStore() } else { cardStore = sqliteStore } } else { cardStore = store.NewMemoryStore() } var llm model.LLM if cfg.UseVertexAI || cfg.GoogleAPIKey != "" || os.Getenv("GOOGLE_CLOUD_AGENT_ENGINE_ID") != "" { realLLM, err := workflow.NewGeminiModel(ctx, cfg.GeminiModel, cfg.GoogleAPIKey, cfg.UseVertexAI, cfg.GoogleCloudProject, cfg.GoogleCloudLocation) if err != nil { slog.Warn("failed to initialize Gemini model, falling back to mock LLM", "err", err) llm = workflow.NewMockLLM() } else { llm = realLLM } } else { llm = workflow.NewMockLLM() } return NewRootAgentWithDeps(cardStore, llm) } func NewRootAgentWithDeps(cardStore store.CardStore, llm model.LLM) (agent.Agent, error) { return workflow.NewCardAssistantAgent(cardStore, llm) } Note a clever detail here: when deployed in Agent Runtime, the system automatically injects the GOOGLE_CLOUD_AGENT_ENGINE_ID environment variable. When running unit tests (go test ./...) without an API Key, it automatically falls back to workflow.NewMockLLM(), ensuring offline tests remain 100% fast and deterministic! * 3. Agent Runtime Standard HTTP Server and A2A Endpoints (Root main.go) Next, let's look at the new main.go in the project root. When the container is deployed to Agent Runtime (or executed locally via make playground), this entry point mounts five standard services: 1. OpenTelemetry Observability (setupObservability): Automatically exports Traces, Logs, and Metrics to Google Cloud Trace and Cloud Logging. 2. Session Service Auto-switching (sessions.SessionService()): In sessions/sessions.go, if VERTEX_AI_SEARCH_ENGINE_ID (Agent Engine ID) is detected, it automatically enables the cloud persistent NewVertexAiEngineSessionService; for local development, it seamlessly uses session.InMemoryService(). 3. ADK REST API and WebUI (/api/ and /ui/): Directly starts the ADK built-in streaming conversation API and web debugging interface via api.NewLauncher() and webui.NewLauncher(). 4. A2A (Agent-to-Agent) Protocol Server: Automatically exposes this Agent's capability description card (AgentCard) at /.well-known/agent-card.json and provides a JSON-RPC call endpoint at /a2a/app! 5. Agent Structure Introspection Endpoint (GET /app-info): Analyzes the entire Agent Tree via appinfo/appinfo.go. It not only lists the Root Graph Workflow but also registers the CompanyEnrichAgent (gemini-2.5-flash)—which is embedded in CompanyEnrichNode and equipped with geminitool.GoogleSearch{}—into the /app-info response. This allows agents-cli eval and the Google Cloud Console Playground to clearly see all sub-agents and tools within the system! * 4. Key Transformation: Enabling Graph Workflow to Support Both "LINE Webhook" and "ADK WebUI / A2A" Dual Channels! In the first version, our RouterNode assumed all input came from the LINE Webhook Handler, which pre-filled StateKeyInputType or StateKeyImageBytes into the Session State. Furthermore, terminal nodes (like FormatCarouselNode, BuildPreviewNode, SaveCardNode) only wrote results to StateKeyReplyText and StateKeyFlexCarouselJSON for the LINE Bot to retrieve and call ReplyMessage. However, when we open the ADK WebUI (make playground), the Agent Engine Console Playground, or talk directly to the Root Agent via agents-cli run "Find someone who does AI for me", the input follows the standard ADK ctx.UserContent() (containing text Part.Text or uploaded image Part.InlineData), and the output is read from ev.Content in the event stream! Therefore, we made two fully backward-compatible enhancements in internal/workflow/nodes_ingest.go and internal/workflow/nodes_search.go: (1) RouterNode Automatically Parses Images and Text from ctx.UserContent() // When uploading a business card image directly from ADK WebUI / A2A / Agent Engine Playground, // automatically extract InlineData.Data from ctx.UserContent().Parts and direct to RouteIngest! if len(getBytesState(ctx, StateKeyImageBytes)) == 0 && ctx.UserContent() != nil { for _, part := range ctx.UserContent().Parts { if part != nil && part.InlineData != nil && len(part.InlineData.Data) > 0 { setState(ctx, ev, StateKeyImageBytes, part.InlineData.Data) if getStringState(ctx, StateKeyInputType) == "" { setState(ctx, ev, StateKeyInputType, RouteIngest) } break } } } // If it's text input and the input parameter is empty, automatically extract query text or commands from ctx.UserContent().Parts if inputType == "" { sTrim, _ := input.(string) sTrim = strings.TrimSpace(sTrim) if sTrim == "" && ctx.UserContent() != nil { for _, part := range ctx.UserContent().Parts { if part != nil && strings.TrimSpace(part.Text) != "" { sTrim = strings.TrimSpace(part.Text) break } } } // Automatically determine if it's searching for a card (RouteSearch) or generating an icebreaker (action=draft_email&card_id=...) ... } (2) Terminal Nodes Synchronously Write to ev.Content In BuildPreviewNode, SaveCardNode, FormatCarouselNode, and EmailDraftNode, in addition to writing to StateDelta for the LINE Bot, we added a line: ev.Content = genai.NewContentFromText(flexJSON, genai.RoleModel) This way: - Sending or searching cards from LINE: The LINE Webhook still gets the Flex JSON from StateDelta to render the beautiful LINE card. - Testing from ADK WebUI (make playground) or Agent Engine Playground: Without needing a LINE account, you can upload a business card image or type "Find someone I met at Cloud Summit who does AI" directly in the browser. You'll see the full execution trajectory (Graph Node transitions) and the output Flex JSON / icebreaker draft directly in the conversation window! We also added a TestRootAgentDirectSearchExecution unit test in app/agent_test.go, using the standard ADK runner.Runner to send a genai.Content query to app.NewRootAgent, verifying that the business card Carousel JSON can be obtained directly from the Event Stream without going through the LINE Engine! * 5. Multi-binary Dockerfile: One Image for Both Agent Runtime and LINE Webhook Finally, to allow the same container image to be deployed to Agent Runtime via agents-cli deploy and to Cloud Run as an independent LINE Bot server, we updated the Dockerfile to compile two static executables in a multi-stage build: # Build the Agent Runtime / ADK WebUI / A2A entrypoint binary (root main.go) RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o /out/agent . # Also build the standalone LINE Bot Webhook server binary (cmd/server/main.go) RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o /out/line-card-assistant ./cmd/server - The default ENTRYPOINT ["/agent"] starts the Agent Runtime standard server (listening on 0.0.0.0:8080). - If it needs to run as a monolithic LINE Webhook Server on Cloud Run, simply specify the Container Command as /bin/line-card-assistant! * 4. Summary and Teaser for the Next Step Through the Phase 1 transformation (PR #10), our LINE AI Business Card Assistant (line-card-assistant) has officially evolved from a simple LINE Bot application into an adk_go Agent that complies with Google Cloud Agent Platform standards: 1. Full support for the agents-cli ecosystem: Equipped with agents-cli-manifest.yaml and .agents-cli-spec.md. 2. Dual-channel execution capability: Can interact via LINE chat and directly through the ADK WebUI via make playground, be called by other Agents via the A2A protocol, or undergo multimodal business card recognition and search testing in the Agent Engine Playground. 3. Cloud-native ready: Built-in VertexAiEngineSessionService auto-switching and OpenTelemetry Cloud Trace tracking capabilities. In the next article of this series, we will enter Phase 2 & Phase 3, where we will connect HITL pause states and image artifacts to cloud persistent storage and refactor the system into a dual-layer decoupled architecture of "Cloud Run LINE Webhook Gateway + Agent Runtime Core Brain." Stay tuned! * Reference Resources and Further Reading - First article in the series: [[Agent Platform in Practice (1)] Building a LINE AI Business Card Assistant with Go and Google ADK 2.0 Graph Workflow: From Fan-Out Parallel Retrieval to HITL Pause Loop Correction](/2026/10/16/adk-graph-workflow-go/) - Open source code and Roadmap for this project: - GitHub Repository: kkdai/line-card-assistant - Agent Runtime Migration Roadmap: Issue #9 - Phase 1 Implementation Pull Request: PR #10: feat(phase1): scaffold adk_go Agent Runtime project structure and app.NewRootAgent - Google agents-cli and Agent Platform official documentation: - Getting Started - agents-cli - The Lifecycle - agents-cli - Google ADK Graph Workflow official documentation: - Graph Workflows in ADK: Everything You Need to Know (Google Cloud Blog) - Graph-based agent workflows (adk.dev/graphs)

名片助手该在哪一步等你确认？ Evan Lin的示例：OCR后并行查重、搜公司、从近期聊天推断会面背景；等分支结束，再整理预览。 修改后重新预览，确认才保存，之后仅生成邮件草稿。 把人工确认放在补全之后、存档之前。 固定源码解读，未实测。 github.com/kkdai/line-card-assistant/blob/19296ec…

[Agent Platform in Action (1)] Building a LINE AI Business Card Master with Go and Google ADK 2.0: From Fan-Out Parallel Retrieval to HITL Pause !image-20261017135128284 Recap During the AI Agent development craze over the past year, the first Agent many developers wrote often looked like this: writing an incredibly long System Instruction, then stuffing five or six Tools into the same LLMAgent, and praying the model would obediently follow steps A → B → C → D. It looks impressive as a Demo initially, but as soon as it hits a production environment, problems immediately surface: - The more steps, the higher the rate of losing control: When a Prompt is stuffed with "First do OCR, then check the database for duplicates, simultaneously search online for what this company does, then assemble a preview card to ask the user, if the user wants to change the job title then regenerate the preview, and only write to the DB and write an icebreaker email after confirmation," the model occasionally skips the duplicate check or writes directly to the database before waiting for user confirmation. - Burning money and waiting in places where Tokens shouldn't be spent: Logic that is 100% deterministic, such as "shunting based on LINE event type" or "normalizing phone number formats," is not only slow when handed to an LLM but also prone to errors. - Independent tasks are forced to execute in a queue: Checking the database, Google searching company backgrounds, and comparing recent conversation records are clearly independent of each other, yet they become sequential waits due to the linear conversation loop of a single Agent. To solve this pain point, Google officially introduced a heavyweight feature in Agent Development Kit (ADK) 2.0 —— Graph Workflows, with simultaneous support for Python and Go (google.golang.org/adk/v2/workflow)! After seeing this update, I immediately decided to use my most familiar Golang + LINE Messaging API SDK v8 to build a complete open-source project line-card-assistant (LINE AI Business Card Master). I put all the core weapons of ADK 2.0 Graph Workflow (AddRoute conditional routing, AddFanOut / AddFanIn parallel processing and aggregation, NewRequestInputEvent Human-in-the-Loop pause/wait and loop correction) through a real-world battle test and deployed it on Google Cloud Run. This article will first guide you through two core technical documents that are must-reads for Google's official ADK Graph Workflows, and then break down step-by-step how we used Go code to implement these graph workflow patterns in a LINE Bot! - * * I. How Google Officially Defines Graph Workflow? Analysis of Two Must-Read Guides Before looking at the code, it is strongly recommended to understand the core philosophy of the ADK team's design for Graph Workflows. Google recently released two very critical developer guides: 1. Google Cloud Official Blog: Graph Workflows in ADK: Everything You Need to Know (Written by Annie Wang and Shangjie Chen of the ADK team) 2. ADK Official Technical Documentation: Graph-based agent workflows (adk.dev/graphs) We have condensed and organized the essence of these two official articles as follows: 1. What is "Graph Engineering"? In the article "Graph Workflows in ADK: Everything You Need to Know", the authors use a "Customer Refund Workflow" as an example to point out a core concept: > As workflows become more complex, reliability comes from separating "Workflow Control" from "Model Reasoning." The essence of Graph Engineering is to decompose tasks into independent nodes, connect them with edges, and explicitly decide at each step whether "Code," "Models," or "People" should determine the next step. The article summarizes five of the most commonly used Graph Workflow design patterns in the real world: | Core Pattern | What pain point does it solve? | ADK 2.0 Corresponding Mechanism | | --- | --- | --- | | 1. Fan-out and Fan-in Parallel Aggregation | Multiple independent data collection steps (e.g., checking orders, payments, customer records) slow down overall latency if executed sequentially. | Use AddFanOut to trigger multiple nodes to execute in parallel, then use a JoinNode (AddFanIn) to wait for all branches to complete before aggregating, consuming zero extra LLM calls in between. | | 2. Deterministic Routing vs. Agent Routing | Some shunting rules are black and white (e.g., amount > $100 goes to manual review) and shouldn't be guessed by an LLM; others require understanding natural language intent. | Deterministic Router: Pure code logic sets ev.Routes, 0 Token consumption, 100% testable. <br> Agent Router: Intent is judged by the LLM, but graph edges strictly limit valid jump targets. | | 3. Human-in-the-Loop (HITL) Pause and Resume | High-risk actions (e.g., large refunds, writing to production databases) must be confirmed by a human, but traditional Webhooks are stateless short connections. | A node issues a RequestInput event to freeze the execution state (Checkpoint) and pause the Graph; after receiving an external human response, RerunOnResume wakes up the same node to continue. | | 4. Parallel Workers vs. Fan-out | Distinguishes between "performing three different checks on the same data" and "performing the same processing on N items." | Fan-out: Different nodes perform different responsibilities. <br> Parallel Workers: The same node expands in parallel for array elements. | | 5. Static Graph vs. Dynamic Orchestration | How to balance "architectural visualization/maintainability" with "dynamic loop correction"? | Use static Edges to define clear routing and Conditional Cycles (e.g., generate draft → check/modify → regenerate), making the entire system's state machine clear at a glance. | - * * 2. The Four Core Building Blocks of ADK Go v2.0 (adk.dev/graphs) In the official document "Graph-based agent workflows", the API design for ADK 2.0 in the Go language (google.golang.org/adk/v2/workflow) is shown in detail. To assemble a Graph in Go, you mainly need to master the following four components: 1. Nodes: - workflow.NewFunctionNode / workflow.NewEmittingFunctionNode: Wraps any standard Go function into a graph node, suitable for DB queries, data cleaning, API calls, or assembling LINE Flex Messages. - llmagent.RunLLMAgentAsNode: Encapsulates an llmagent with a specific Prompt and tools (e.g., geminitool.GoogleSearch{}) into a single node, allowing the LLM to focus on doing "one thing" well. 2. State & Data Flow: - In addition to passing strongly-typed Output between nodes, structured data can be written to the Session State (ctx.Session().State()) via ev.Actions.StateDelta for downstream or aggregation nodes to read. 3. Edges & EdgeBuilder: - eb.Add(from, to): Unidirectional straight connection. - eb.AddRoute(from, workflow.StringRoute("route_name", targetNode), ...): Conditional branch routing (can also be used to create Cycles!). - eb.AddFanOut(from, branch1, branch2, ...) and eb.AddFanIn(joinNode, branch1, branch2, ...): Parallel branching and synchronous aggregation. 4. WorkflowAgent: - Finally, call workflowagent.New(workflowagent.Config{Edges: edges}), and the entire Graph becomes a standard ADK agent.Agent, which is then handed over to the runner.Runner for execution and Session management! - * * II. Practical Project Architecture: LINE AI Business Card Master (line-card-assistant) Having understood the official theory, let's see how to apply these concepts to a real-world LINE Bot scenario. Business professionals receive a large number of paper business cards at exhibitions or meetings every day. Traditional business card apps only offer simple OCR recognition but lack features like "What has this company been doing lately?", "At which event did I meet this person?", and "How can I correct recognition errors directly by speaking in the chat room?". Therefore, we designed line-card-assistant, with an overall architecture that fully corresponds to ADK 2.0's Graph Workflow: [LINE Webhook] │ ▼ [Router Node (AddRoutes)] ├── (Received business card photo) ──► [Vision OCR Node] │ │ │ (AddFanOut Parallel Processing) │ ┌──────────────┼──────────────┐ │ ▼ ▼ ▼ │ [DB Dup Check & Validation] [Company Background Search] [Past Interaction Retrieval] │ └──────────────┼──────────────┘ │ (AddFanIn Aggregation) │ ▼ │ [Generate Flex Message Preview Card] │ │ │ ⏸️ NewRequestInputEvent (HITL Pause & Wait) │ │ │ ┌────────────────────┴────────────────────┐ │ ▼ (Click: Confirm Save) ▼ (Click: Modify Field / Add Note) │ [Write to DB + Generate Icebreaker Draft] [Re-entry Loop Correction Node] │ └── (Received query text) ──► [Query Rewrite] ──► [SQL + Vector Hybrid Retrieval] ──► [Flex Carousel Card Rotation] This architecture diagram covers three core paths: 1. Business Card Recognition and Enrichment Pipeline (Ingest Pipeline): Receive photo → Vision OCR → Execute three tasks in parallel (DB duplicate check + Google Search company background + Conversation context inference for meeting occasion) → Aggregate to generate LINE Flex preview card → HITL pause to wait for user confirmation or modification. 2. HITL Loop Correction (Re-entry Loop): When the graph is paused at HITLConfirmNode, if the user enters text directly in LINE (e.g., "_Change job title to Director, met at Cloud Summit_"), the workflow wakes up and takes the RouteEdit branch into ApplyEditNode, updates the fields, and loops back to BuildPreviewNode to regenerate the preview card and pause again, until the user clicks "✅ Confirm Save" to actually write to the database and generate the icebreaker email! 3. Natural Language Hybrid Retrieval Pipeline (Search Pipeline): Receive query text (e.g., "_Help me find people doing AI I met at Cloud Summit last week_") → Query Rewrite structural decomposition → SQL + Keyword/Tag hybrid retrieval → Return LINE Flex Carousel business card rotation. - * * III. Go Code Deep Dive: Writing Graph Workflow into a LINE Bot Next, let's look directly at the Go implementation of internal/workflow/graph.go and various nodes to see how clean the ADK 2.0 API looks in Go. 1. A Global View: Assembling the Graph with EdgeBuilder In internal/workflow/graph.go, we declare and connect all nodes via the BuildEdges function. You can clearly see the topological structure of the entire graph: func BuildEdges(deps *Dependencies) ([]workflow.Edge, error) { // 1. Entry Router Node routerNode, _ := NewRouterNode() // 2. Business Card Recognition and Fan-Out Three Parallel Nodes visionOCRNode, _ := NewVisionOCRNode(deps) cleanAndDedupNode, _ := NewCleanAndDedupNode(deps) companyEnrichNode, _ := NewCompanyEnrichNode(deps) contextMatchNode, _ := NewContextMatchNode(deps) joinPreviewNode := workflow.NewJoinNode("JoinPreviewNode") // 3. Preview Card Generation, HITL Pause, and Confirm/Modify Branch Nodes buildPreviewNode, _ := NewBuildPreviewNode(deps) hitlConfirmNode, _ := NewHITLConfirmNode(deps) saveCardNode, _ := NewSaveCardNode(deps) applyEditNode, _ := NewApplyEditNode(deps) // 4. Natural Language Search and Icebreaker Email Nodes queryIntentNode, _ := NewQueryIntentNode(deps) hybridSearchNode, _ := NewHybridSearchNode(deps) broadSearchNode, _ := NewBroadSearchNode(deps) formatCarouselNode, _ := NewFormatCarouselNode(deps) emailDraftNode, _ := NewEmailDraftNode(deps) eb := workflow.NewEdgeBuilder() // Entry -> Deterministic Router eb.Add(workflow.Start, routerNode) eb.AddRoute(routerNode, workflow.StringRoute(RouteImage, visionOCRNode), workflow.StringRoute(RouteText, queryIntentNode), workflow.StringRoute(RoutePostbackDraft, emailDraftNode), ) // Ingest Pipeline: Vision OCR -> Parallel Fan-Out -> JoinNode Fan-In -> Preview eb.AddFanOut(visionOCRNode, cleanAndDedupNode, companyEnrichNode, contextMatchNode) eb.AddFanIn(joinPreviewNode, cleanAndDedupNode, companyEnrichNode, contextMatchNode) eb.Add(joinPreviewNode, buildPreviewNode) eb.Add(buildPreviewNode, hitlConfirmNode) // Conditional branches after HITL pause: Confirm Save vs. Modify Field Loop eb.AddRoute(hitlConfirmNode, workflow.StringRoute(RouteConfirm, saveCardNode), workflow.StringRoute(RouteEdit, applyEditNode), ) // ★ Key Cycle: After modifying fields, route back to buildPreviewNode to regenerate Flex preview card and enter HITL again! eb.AddRoute(applyEditNode, workflow.StringRoute(RoutePreview, buildPreviewNode), ) // Search Pipeline: Query Rewrite -> Hybrid Search -> (Found / Empty Fallback) -> Carousel eb.Add(queryIntentNode, hybridSearchNode) eb.AddRoute(hybridSearchNode, workflow.StringRoute(RouteFound, formatCarouselNode), workflow.StringRoute(RouteEmpty, broadSearchNode), ) eb.Add(broadSearchNode, formatCarouselNode) return eb.Build() } This code perfectly embodies the Static Graph advantage mentioned in the Google Cloud Blog: you don't need to flip through thousands of lines of Prompts or track down Tool Calls scattered everywhere. Just by looking at the 30 lines of code in BuildEdges, you can 100% grasp all possible paths and loop boundaries of the entire Agent system. - * * 2. Deterministic Router: Precise Shunting with 0 Token Consumption When the LINE Webhook receives an event, we don't need to waste a Gemini API call to ask the model "Is the user sending a picture or text?". We use workflow.NewEmittingFunctionNode to create a pure code deterministic routing node RouterNode: const ( RouteImage = "route_image" RouteText = "route_text" RoutePostbackDraft = "route_postback_draft" ) func NewRouterNode() (workflow.Node, error) { return workflow.NewEmittingFunctionNode("RouterNode", func(ctx workflow.Context, _ struct{}) (string, error) { inputType, err := getStateString(ctx, StateKeyInputType) if err != nil || inputType == "" { inputType = "text" } var route string switch inputType { case "image": route = RouteImage case "postback_draft": route = RoutePostbackDraft default: route = RouteText } // Emit an event with Routes tags to let EdgeBuilder's AddRoute decide the next node ev := workflow.NewEvent(ctx, "route_selected") ev.Routes = []string{route} if err := ctx.Emit(ev); err != nil { return "", err } return route, nil }, workflow.WithDeclaredRoutes(RouteImage, RouteText, RoutePostbackDraft)) } As long as you set ev.Routes = []string{route} when emitting a workflow.Event, and use workflow.WithDeclaredRoutes(...) to validate route legitimacy during construction, ADK will automatically check for misspelled Route names during the startup phase! - * * 3. Fan-Out / Fan-In Parallel Processing: Running DB Dup Check, Google Search, and Context Inference Simultaneously Once VisionOCRNode extracts a structured models.BusinessCard from the business card photo via gemini-2.5-flash's multimodal capabilities, we have three completely independent tasks to perform: 1. CleanAndDedupNode (Pure Code + DB Query): Normalizes Taiwan mobile and landline numbers to +886-... format and queries the SQLite database to see if a business card for the same person already exists (if the job title is different, it can even mark a "Promotion/Transfer Reminder"!). 2. CompanyEnrichNode (Embedded llmagent with Google Search tool): Automatically searches the web for the latest background of the company based on the company name on the business card, condensing it into a 2-sentence summary and 3~5 industry tags. 3. ContextMatchNode (LLM Context Inference): Reads the user's most recent LINE conversation records before and after sending the photo (e.g., the user just sent "_I'm attending Cloud Summit today_") to automatically infer the meeting occasion for this business card (MetContext). In internal/workflow/nodes_ingest.go, we can even use llmagent.RunLLMAgentAsNode inside CompanyEnrichNode to call a sub-agent with geminitool.GoogleSearch{}: // Combine LLMAgent with Google Search Grounding capability within a Graph node searchAgent, err := llmagent.New(llmagent.Config{ Name: "CompanySearchSubAgent", Description: "Searches the web for company background and industry tags.", Model: deps.SearchModel, Instruction: "Given a company name, search Google and return JSON with summary (2 sentences in Traditional Chinese) and tags (3 to 5 industry tags).", Tools: []tool.Tool{geminitool.GoogleSearch{}}, }) if err == nil { _ = llmagent.RunLLMAgentAsNode(searchAgent, llmagent.NodeConfig{Name: "CompanySearchSubAgentNode"}) } 💡 Practical Details of Go Parallel Nodes: How to Avoid Race Conditions? When running eb.AddFanOut in Go, the three branch nodes execute in parallel in different Goroutines. If three nodes simultaneously read and write to the same BusinessCard JSON object and write back to the same State Key, a Lost Update (last write wins) race condition will occur! To solve this problem, we adopted a very clean technique in the implementation: - Write to independent StateDelta Keys: - CleanAndDedupNode only writes to StateKeyDedupCard, StateKeyIsDuplicate, StateKeyOldTitle. - CompanyEnrichNode only writes to StateKeyCompanySummary and StateKeyCompanyTags. - ContextMatchNode only writes to StateKeyMetContext. - ADK 2.0's Runner will sequentially Commit the ev.Actions.StateDelta emitted by each node in the main Goroutine. - Once JoinPreviewNode (eb.AddFanIn) confirms that the three parallel branches are complete, the single-threaded BuildPreviewNode uses loadMergedCardFromState(ctx) to safely merge the results of the three branches into a complete BusinessCard and renders a beautiful LINE Flex Message preview card! - * * 4. Human-in-the-Loop (HITL): NewRequestInputEvent Pause and Re-entry Loop Correction The most exciting part of this entire system is the Human-in-the-Loop pause and resume mechanism. In traditional LINE Bot development, if you want to achieve "send preview card → wait for user to click confirm or type modification → send new preview card after modification → save after confirmation," you usually have to maintain a complex user_state_machine table in the database yourself. But in ADK 2.0 Graph Workflow, this is built directly into HITLConfirmNode! Let's see how NewHITLConfirmNode in internal/workflow/nodes_ingest.go is written: func NewHITLConfirmNode(_ *Dependencies) (workflow.Node, error) { rerun := true return workflow.NewEmittingFunctionNode("HITLConfirmNode", func(ctx workflow.Context, _ struct{}) (string, error) { activeID, _ := getStateString(ctx, StateKeyActiveInterruptID) // [Phase 1: First entry to node] Not yet paused, emit NewRequestInputEvent to pause Graph! if activeID == "" { interruptID := fmt.Sprintf("hitl-card-%d", time.Now().UnixNano()) if err := emitStateDelta(ctx, map[string]any{ StateKeyActiveInterruptID: interruptID, }); err != nil { return "", err } reqEv := workflow.NewRequestInputEvent(ctx, session.RequestInput{ InterruptID: interruptID, Payload: map[string]any{ "prompt": "Please confirm if the business card info is correct. Click 'Confirm Save' or enter modifications directly (e.g., change title to Director).", }, }) if err := ctx.Emit(reqEv); err != nil { return "", err } return "waiting_for_user", nil } // [Phase 2: Wake up and rerun after receiving user response (RerunOnResume: true)] resumedVal, err := ctx.ResumedInput(activeID) var decision HITLResumePayload if err == nil && resumedVal != nil { _ = decodeViaJSON(resumedVal, &decision) } // Clear the consumed InterruptID so that if the Edit loop comes back later, it can trigger a pause again! _ = emitStateDelta(ctx, map[string]any{ StateKeyActiveInterruptID: "", StateKeyHITLAction: decision.Action, StateKeyHITLEditText: decision.EditText, }) route := RouteConfirm if strings.EqualFold(decision.Action, "edit") || (decision.Action != "confirm" && strings.TrimSpace(decision.EditText) != "") { route = RouteEdit } ev := workflow.NewEvent(ctx, "hitl_decision") ev.Routes = []string{route} if err := ctx.Emit(ev); err != nil { return "", err } return route, nil }, workflow.WithNodeConfig(workflow.NodeConfig{ RerunOnResume: &rerun, // ★ Key setting: Rerun this node function when external input resumes }), workflow.WithDeclaredRoutes(RouteConfirm, RouteEdit)) } How does the LINE Webhook feed user button clicks or text back to the paused Graph? When HITLConfirmNode emits a NewRequestInputEvent, our Engine (located in internal/workflow/engine.go) captures this InterruptID and pushes the LINE Flex Message preview card generated by BuildPreviewNode to the user. Then: - If the user clicks the "✅ Confirm Save" button (Postback Event) on the Flex Message, the LINE Webhook calls Engine.ResumeHITL(ctx, userID, HITLResumePayload{Action: "confirm"}). - If the user types directly in the chat room "Change title to Technical Director, met at AI conference" (Text Event), the LINE Webhook detects that the user has a PendingHITL waiting, automatically intercepts this text, and calls Engine.ResumeHITL(ctx, userID, HITLResumePayload{Action: "edit", EditText: text})! Inside Engine.ResumeHITL, you only need to send a FunctionResponse with workflow.WorkflowInputFunctionCallName to the ADK Runner, and the frozen Graph will seamlessly continue execution: msg := &genai.Content{ Role: string(genai.RoleUser), Parts: []*genai.Part{ { FunctionResponse: &genai.FunctionResponse{ ID: pending.InterruptID, Name: workflow.WorkflowInputFunctionCallName, Response: map[string]any{ "action": payload.Action, "edit_text": payload.EditText, }, }, }, }, } Once it enters RouteEdit, ApplyEditNode corrects the business card data, then loops back to BuildPreviewNode via eb.AddRoute(applyEditNode, workflow.StringRoute(RoutePreview, buildPreviewNode)), generating an updated Flex preview card and stopping again at HITLConfirmNode to wait for user confirmation. This is where Graph Workflow is most powerful for handling multi-turn interactive corrections! - * * 5. Natural Language Hybrid Retrieval Pipeline (Query Rewrite + Hybrid Search + Flex Carousel) When the user is not waiting for business card confirmation but directly enters query text (e.g., "_Help me find people doing AI I met at Cloud Summit last week_"), the RouterNode directs the request to the RouteText branch (defined in internal/workflow/nodes_search.go): 1. QueryIntentNode: Uses Gemini to decompose natural language into a structured SearchFilter (including Keywords, Tags, Company, Title, MetContext). 2. HybridSearchNode: Combines SQL condition filtering with tag/keyword matching and simultaneously acts as a Deterministic Router: - If business cards are found → Emit RouteFound to go directly to FormatCarouselNode. - If precise conditions yield no results → Emit RouteEmpty to automatically enter BroadSearchNode for fuzzy/relaxed retrieval (Fallback), and finally FormatCarouselNode renders up to 10 LINE Flex Carousel business cards, each with a one-click "✉️ Generate Icebreaker" Postback button! - * * IV. Deployment: Cloud Run + Self-Identity Security Authentication After completing development and passing all unit tests (go test ./...), we deployed the service to Google Cloud Run (asia-east1). In the cloud deployment architecture, we can use Cloud Run's Service Account Self-Identity (Application Default Credentials, ADC) mechanism: - By binding a Service Account with roles/aiplatform.user permissions to Cloud Run and setting environment variables GOOGLE_GENAI_USE_VERTEXAI=true and GOOGLE_CLOUD_PROJECT, the application can directly call Vertex AI / Gemini models using the Identity provided by the Cloud Run execution environment, eliminating the risk of managing static API Key leaks. - If developing locally or using the Gemini Developer API, you can easily switch via environment variables: gcloud run deploy line-card-assistant \ --source . \ --region asia-east1 \ --allow-unauthenticated \ --set-env-vars="LINE_CHANNEL_SECRET=your_secret,LINE_CHANNEL_ACCESS_TOKEN=your_token,GEMINI_MODEL=gemini-2.5-flash" - * * V. Summary of Pitfalls and Practical Experience After deeply integrating Google ADK 2.0 Graph Workflow (google.golang.org/adk/v2/workflow) with LINE Bot, there are several practical insights worth sharing with Go developers: 1. Keep Prompts Small, Leave the Flow to the Graph: Previously, a single super Agent required hundreds of lines of Prompts and often missed steps; now, the Prompt for each node (such as VisionOCRNode, CompanyEnrichNode, ContextMatchNode) is only a few lines long, with a single responsibility and fixed output format, resulting in a quantum leap in stability. 2. AddFanOut Significantly Reduces Perceived Latency: In a LINE chat room, users hate waiting after sending a business card photo. By using AddFanOut to simultaneously perform DB duplicate checks, Google Search company background investigations, and context inference, the total time taken depends only on the slowest branch, rather than the sum of all three steps. 3. Use StateDelta to Isolate Parallel Writes: When using AddFanOut in Go, remember to have each parallel branch write to its own dedicated Session State Key, and then merge them uniformly after passing through the JoinNode (AddFanIn) to elegantly avoid concurrent overwrite issues. 4. NewRequestInputEvent + RerunOnResume is a Secret Weapon for Chatbots: Whether it's a LINE Postback button or free text input, they can all be uniformly encapsulated into a FunctionResponse for WorkflowInputFunctionCallName to wake up the same Graph Session, completely bidding farewell to hand-coded spaghetti-style state machines! The complete project source code, architecture diagrams, and deployment scripts are all open-sourced on GitHub. You are welcome to clone it and play around, or directly transform it into your own enterprise-grade LINE Agent workflow! - * * References and Further Reading - Open Source Code for this Project: kkdai/line-card-assistant (GitHub) - Google Cloud Official Blog: Graph Workflows in ADK: Everything You Need to Know - ADK Official Documentation: Graph-based agent workflows - Agent Development Kit (ADK)

[AI in Practice] Upgrading from Gemini 3.5/3.6 to Gemini 4 Argon: Migration Notes and Pitfalls for LINE Bot Developers !Stylized promotional blogger visual with modern editorial branding and the text "Gemini 4 Argon". Background I have a LINE Bot that I use every day, linebot-helper-python. I use it daily to summarize URLs, ask follow-up questions about YouTube video content, search for map locations, and manage bookmarks. In terms of model configuration, to balance response speed and budget, many workflows were previously running on Gemini 3.5 (such as gemini-3.5-flash-lite for high-throughput, low-latency classification and data extraction) and Gemini 3.6 (such as gemini-3.6-flash for multi-step tool calling and multimodal reasoning). In mid-October, Google officially pushed Gemini 4 Argon (gemini-4-argon) to the GA stable version (paired with Python google-genai >= 2.30.1). After reading the official announcement and migration documents, the most attractive features on the surface are the big numbers: a 1M unified token window shared by input, thinking, and output; Long Decode that automatically relays a single call up to 1 million output tokens; Agentic Video Understanding which is now officially GA and saves 88% of tokens; and the Interactions API (/v1beta/interactions), which has become the default mainstream. But if your goal is the same as mine—to upgrade (or partially switch) the existing Gemini 3.5 / 3.6 in your LINE Bot to Gemini 4 Argon—don't assume it's as simple as changing model="gemini-3.6-flash" to model="gemini-4-argon". This generation has several changes in parameter specifications, token budget calculation methods, caching mechanisms, and even assumptions about waiting times that hit the pain points of LINE Bots exactly. This article organizes all the details you need to pay attention to when migrating from 3.5 / 3.6 to Gemini 4. - * * Specification Comparison: 3.5 Flash-Lite / 3.6 Flash vs. Gemini 4 Argon Before diving into settings, let's look at the positioning differences of these three models in one table: | Item | gemini-3.5-flash-lite | gemini-3.6-flash | gemini-4-argon | | --- | --- | --- | --- | | Positioning | Ultra-fast, low-cost, high-throughput classification & structured extraction | Balance of speed, multi-step Agent, and multimodality | Flagship deep reasoning, complex coding, long output, and autonomous agents | | Context Window | 1,048,576 | 1,048,576 | 1,048,576 (Shared Input + Thinking + Output) | | Max Single Output | 65,536 | 65,536 | generateContent: Non-streaming 32,768 / Streaming 65,536 <br>Interactions API: Synchronous 65,536 <br>_(Long Decode can continue up to 1M for both)_ | | Default thinking_level | "minimal" | "medium" | "medium" (Supports only low / medium / high) | | Context Caching | Supported | Supported | Implicit Caching only (Calling Explicit will error) | | temperature / top_p / top_k | Deprecated (Ignored) | Deprecated (Ignored) | Deprecated (Ignored, official requirement to remove) | | Pricing (Input / Output) | $0.30 / $2.50 (per 1M) | $1.50 / $7.50 (per 1M) | Early bird until 2026/11/30: $2.00 / $10.00 <br>_(Reverts to $4.00 / $20.00 from 12/1; 95% discount for cache hits)_ | | Free Tier | Available | Available | None (Must be linked to Paid Tier billing) | Just by looking at this table, you can identify several pitfalls you'll encounter if you only change the model name. Let's break them down one by one. - * * Pitfall 1: LINE Bots usually set max_output_tokens very small, which returns an empty string in Gemini 4 This is the first hidden trap that LINE Bot developers are most likely to fall into. Because a single LINE text message has a 5,000-character limit, and mobile screens are not suitable for long-winded content, many people (including myself in the past) are used to setting max_output_tokens to 2048 or 4096 in GenerateContentConfig as a safeguard to save money and prevent text overflow. If you set it this way on Gemini 4 Argon, when encountering a slightly complex problem, you will find the model returns finish_reason: "MAX_TOKENS", but response.text is empty, or it cuts off after barely two sentences. The reason is that Gemini 4 Argon adopts a Unified 1M Input-Output Window, and has two hard rules: 1. Thinking tokens are counted directly towards the max_output_tokens budget: Every thinking token the model spends before answering is deducted from your max_output_tokens. When a question requires deep reasoning, the model can easily burn through your 4096 limit just by thinking, hitting a wall and stopping before it even starts outputting the formal answer. 2. generateContent non-streaming single request limit decreased from 64K to 32K: In Gemini 3.5 / 3.6, the single non-streaming response limit was 65,536; in gemini-4-argon, if you continue using traditional generateContent, the non-streaming single limit has been lowered to 32,768 tokens (streaming or using the new Interactions API synchronous call remains at 65,536). Cause and Solution: The official migration guide states clearly—if your old project set max_output_tokens below 32K, please increase it to at least 32768 (or just keep the default 65536) when upgrading to gemini-4-argon. In Gemini 4, max_output_tokens is used as a "total emergency brake to prevent burning through money in extreme situations," not to control the length of the answer. - To spend fewer tokens and speed up response: Lower the thinking_level="low". - To make the output fit LINE message length: Write it in the system_instruction (e.g., "Please keep it within 400 words and use bullet points"). # ❌ Old safeguard setting often used in LINE Bots; thinking process will consume all 2048 tokens on gemini-4-argon config = types.GenerateContentConfig( max_output_tokens=2048, ) # ✅ Correct way for Gemini 4 Argon: Relax max_output_tokens, use thinking_level to control cost and latency config = types.GenerateContentConfig( max_output_tokens=32768, thinking_config=types.ThinkingConfig(thinking_level="low"), system_instruction="You are a LINE assistant. Keep answers concise and within 500 words.", ) - * * Pitfall 2: Differences in thinking_level settings—3.5-flash-lite's "minimal" cannot be moved directly If you have already changed the old-school thinking_budget (passing an integer token count) to thinking_level in Gemini 3.5 / 3.6, there is another small trap here: - On gemini-3.5-flash-lite, the default thinking level is called "minimal". - But on gemini-4-argon, only three levels are supported: "low", "medium" (default), and "high". If your config shares the same generator and you bring "minimal" into gemini-4-argon as is, or if you still have the old thinking_budget (although the current API will map it to the closest level, it will return a deprecation warning), it will make your settings messy. For LINE Bots, the practical division of these three levels in gemini-4-argon is very clear: - "low": The first choice for general LINE conversations, URL summaries, and real-time Q&A. Lowest token consumption and lowest Time to First Token (TTFT). - "medium" (default): Suitable for scenarios requiring multiple tool calls (Function Calling), reading long videos, or writing code. - "high": Deep math, complex itinerary optimization, long-form analysis. Note: Setting this to "high" makes it almost impossible to finish within the valid time of a LINE replyToken. You must use an asynchronous push architecture. - * * Pitfall 3: LINE Webhook's few-second lifespan vs. official recommendation of "20-minute Timeout" When reading the Gemini 4 Argon migration document, one item made me laugh out loud: > Increase client timeouts: Set HTTP client timeouts to at least 15–20 minutes per call. The official documentation even added a note in the Long Decode section: when facing difficult tasks, gemini-4-argon might spend an entire 32K or 64K hop entirely on thinking, only outputting the real answer in the next hop (under generateContent, the new SDK google-genai >= 2.30.1 has automatic_continuation=True enabled by default, which uses continuation_token to automatically loop for you on the client side; under the Interactions API, to exceed the single sync limit of 65536, you must enable background=True to let the server side run multiple hops). This is perfectly fine when running a CLI script on your own computer; but in a LINE Bot + Cloud Run architecture, waiting synchronously on the client side is a disaster: 1. LINE's replyToken has an extremely short lifespan and won't wait two minutes for you to think. 2. LINE users staring at a screen with no response for more than 10 seconds will start pressing the button a second or third time. 3. If you wait synchronously for automatic_continuation to finish several hops within the same Webhook request, Cloud Run connections and workers will quickly become deadlocked. Cause and Solution: To put Gemini 4 into a LINE Bot, you must do two things architecturally: 1. Don't phase out 3.5 / 3.6; instead, create a "Fast/Slow Model Router" The official documentation emphasizes that gemini-3.6-flash and gemini-3.5-flash-lite are still the main models for stable online supply. The healthiest LINE Bot architecture is not "switching everything to gemini-4-argon," but rather: - First-line instant replies / Intent classification / Simple structured extraction: Continue to hand these to gemini-3.5-flash-lite ($0.30/1M, extremely fast) or gemini-3.6-flash ($1.50/1M, concise, saves steps in tool calling). - Heavy reasoning / Long video follow-up / Complex tasks: Only then route to gemini-4-argon. 2. Use the new Interactions API with background=True + webhook_config for heavy tasks In this generation, Google has labeled our familiar generateContent as (Legacy) and is pushing the Gemini Interactions API (/v1beta/interactions) across its documentation. The greatest value of the Interactions API for LINE Bots is its native support for background=True and Webhooks (supporting project-level Static Webhooks or Dynamic webhook_config included in a single request): 1. A user asks a question on LINE that requires gemini-4-argon with "high" thinking or long output. 2. The LINE Bot first calls the LINE Loading Animation API and uses replyToken to reply instantly: "_This is a deep question. I am performing deep reasoning in the background and will push the result to you once finished!_" 3. Simultaneously call client.interactions.create(..., background=True, webhook_config=...), handing the task to Google's server to run multiple hops slowly. You can even put the line_user_id directly into user_metadata so Google returns it as-is during the callback. 4. When Google finishes and sends an interaction.completed event to your /webhooks/gemini endpoint, use client.interactions.get(interaction_id) to retrieve the result (if multiple Long Decode hops over 64K were run, all segments will be fully recorded in the model_output of interaction.steps). Then, calmly call LINE's push_message to push the result into the chat room. from google import genai client = genai.Client() def dispatch_heavy_line_task(line_user_id: str, prompt: str): # Hand over to Google server for background execution (even multi-round 64K hops for Long Decode will be automatically completed on the server side) interaction = client.interactions.create( model="gemini-4-argon", input=prompt, background=True, generation_config={ "max_output_tokens": 131072, "thinking_level": "high", }, webhook_config={ "uris": ["https://your-cloud-run.a.run.app/webhooks/gemini"], "user_metadata": {"line_user_id": line_user_id}, }, ) return interaction.id - * * Pitfall 4: Explicit Caching Errors—Prompt order determines if you save 95% If your LINE Bot contains a full product FAQ, long System Instructions, or large documents uploaded by users, you might have previously used client.caches.create() to create an Explicit Cache and then put cached_content into GenerateContentConfig to save money. After upgrading to gemini-4-argon, this code must be completely removed. gemini-4-argon does not support Explicit Caching. If you call caches.create or include cached_content in the config, the API will return an Error. Instead, it uses Implicit Caching, which is enabled by default: - As long as a single Prompt length reaches 4,096 tokens or more, and the preceding content (Prefix) matches a previous request, the system will automatically hit the cache. - Cache-hit Input Tokens receive a 95% discount (priced at 0.05x)—at early bird prices, this drops from $2.00 / 1M to $0.10 / 1M, which is cheaper than gemini-3.5-flash-lite! (Even when Long Decode automatically continues to the next hop, the repeated Context from before will automatically apply the 95% cache discount.) Key impact on LINE Bots: Since this has become automatic Prefix Matching, the order in which you assemble your Prompt determines the thickness of your bill: - Always put at the beginning (Immutable): Long system_instruction, shared background knowledge base, Few-shot examples, fixed background documents for that group. - Always put at the end (Changes every time): current_time timestamp, LINE user name/status, the current message. If you accidentally write "Current time: 2026-10-15 21:40:02" on the first line of your System Prompt, the first token of every request will be different, and the 95% cache discount will immediately drop to zero. - * * Pitfall 5: Strict rules tightening since 3.5 / 3.6—Sampling parameters, Prefill, and Thought Signatures These three rules have actually been in effect since gemini-3.5-flash-lite and gemini-3.6-flash, but if you are organizing your codebase across versions, be sure to clean them up together: 1. Remove temperature, top_p, top_k, candidate_count These four sampling parameters are all deprecated and ignored on Gemini 3.5 / 3.6 and Gemini 4 Argon (official notice says future generations will return HTTP 400 if these parameters are included). If you used to set temperature=0.0 to ensure stable output format, delete them all now and switch to Structured Outputs (response_schema) and clear system_instruction. 2. Prohibit Prefilled Model Turns (The last turn cannot be role: "model") A common Prompt technique used to be: to prevent the model from saying "Sure! Here is the summary..." or to force it to start outputting JSON from {, you would put {"role": "model", "parts": [{"text": "{"}]} as the last entry in the contents array. Now, if the last turn of non-empty content in the payload is the model role, the API will return an HTTP 400 Error. To suppress the opening remarks, please write it properly into the system_instruction. 3. Multi-turn conversations and Function Calling must preserve thought_signature (or just use previous_interaction_id) When a LINE Bot runs multi-step tool calls (e.g., first check bookmark DB $\rightarrow$ then fetch web content $\rightarrow$ finally summarize), the Part returned by the model will contain a thought_signature. If you manually maintain the conversation history array but only extract and reassemble the text and function_call, missing the thought_signature, the quality of multi-turn tool calls will degrade significantly or even error out. For those who don't want to manage thought_signature serialization themselves, the easiest way to upgrade to Gemini 4 is to use the Interactions API's previous_interaction_id: # Only remember the last interaction.id for that LINE user in Redis; let Google's server handle history state and thought_signature interaction = client.interactions.create( model="gemini-4-argon", input=user_message, previous_interaction_id=last_interaction_id, ) - * * The real weapon worth upgrading your LINE Bot for: Agentic Video Understanding GA After discussing all the pitfalls, let's talk about a positive upgrade for gemini-4-argon (and 3.8-flash) that is most noticeable for LINE Bots: Agentic Video Understanding is officially GA. One core feature of my LINE Bot is "drop a YouTube link and ask follow-up questions about the video content." Previously, traditional video understanding (static mode) involved taking a screenshot of the entire video every second (1 FPS) and stuffing it into the Context Window. For a 30-60 minute long lecture or Podcast video, even if the Context could fit it, the token consumption was staggering. After switching to Agentic mode (Note: it still defaults to static on all models including gemini-4-argon, so it must be explicitly enabled), the model's behavior changes to: 1. First read the audio transcript and timeline structure. 2. Based on the user's question, decide which timestamps to "scrub" to. 3. Dynamically load high-resolution and high-frame-rate frames only for key scenes to look closely. Whether you use the new Interactions API or the original generate_content, remember to add the corresponding parameters (parameter names and casing differ slightly between the two): # Method A: Using the new Interactions API (parameter is processing="agentic") interaction = client.interactions.create( model="gemini-4-argon", input=[ { "type": "video", "uri": youtube_url, "mime_type": "video/mp4", "processing": "agentic", # Remember to enable explicitly; default is still static }, { "type": "text", "text": "On the page where the speaker shows the architecture diagram, what are the three bottlenecks mentioned? Please include [MM:SS] timestamps.", }, ], generation_config={ "max_output_tokens": 32768, "thinking_level": "medium", }, ) # Method B: Using traditional generate_content (parameter is media_processing="AGENTIC") response = client.models.generate_content( model="gemini-4-argon", contents=[ types.Part.from_uri( file_uri=youtube_url, mime_type="video/mp4", media_processing="AGENTIC", # Remember to enable explicitly; default is still STATIC ), "On the page where the speaker shows the architecture diagram, what are the three bottlenecks mentioned? Please include [MM:SS] timestamps.", ], config=types.GenerateContentConfig( max_output_tokens=32768, thinking_config=types.ThinkingConfig(thinking_level="medium"), ), ) Official tests show that for long videos over 5-10 minutes and "needle in a haystack" detail questions, token usage is saved by up to 88%, and accuracy is improved by 7%. This point alone completely makes up for the higher unit price of gemini-4-argon. - * * Upgrade Checklist Finally, here is a PR self-check list for LINE Bot projects before upgrading from Gemini 3.5 / 3.6 to gemini-4-argon: 1. SDK and Billing Confirmation: Upgrade google-genai to >= 2.30.1; confirm the project associated with the API Key is linked to a Paid Tier (Free Tier and Google AI Pro cannot call gemini-4-argon). 2. Remove Sampling Parameters and Prefill: Search the entire project and remove temperature, top_p, top_k, candidate_count, and any prefilled prompts ending with role="model". 3. Remove Explicit Caching: Remove caches.create and cached_content; check Prompt assembly logic to ensure fixed, long System Instructions are placed at the beginning to trigger the 95% discount of Implicit Caching. 4. Increase max_output_tokens: Increase any limit smaller than 32768 to at least 32768 (to avoid empty responses caused by thinking tokens consuming the budget); move word count limits to system_instruction. 5. Check thinking_level Enum Values: Replace 3.5-flash-lite specific "minimal" or old thinking_budget with "low", "medium", or "high". 6. Fast/Slow Routing and Async Push: Tasks that need an instant reply within a few seconds for LINE replyToken should be set to thinking_level="low" or continue to be handled by 3.5-flash-lite / 3.6-flash; deep reasoning tasks should use the Interactions API (background=True + webhook_config) paired with LINE push_message. 7. Enable Agentic Mode for Video Understanding: When processing YouTube or long videos, add media_processing="AGENTIC" to Part.from_uri (or "processing": "agentic" for the Interactions API). If you use a Coding Agent that supports Skills (like Antigravity or Claude Code), Google also provides a migration Skill (/gemini-api-dev migrate my app to Gemini 4 Argon) that can help automatically scan for these parameter changes. However, architectural levels like "LINE replyToken timeout routing" and "Prompt prefix cache order" still need to be designed manually.