linebot-helper-python

[AI in Action] Rewriting a Daily LINE Bot from Python to Go 1.26 on Cloud Run: 1/4 Resource Usage, Google ADK v1.9.0, and Zero-D !LINE Bot Helper: Python to Go 1.26 & Cloud Run Architecture (ADK v1.9.0) Background I have a primary LINE Bot that I use heavily every day: kkdai/linebot-helper-python. Initially, it was just a small tool where you could "drop a URL and get a Traditional Chinese summary," but over the past year and a half, I've gradually integrated various new Gemini capabilities into it: - Multi-source Content Loading and Document Conversion: A three-layer web crawler Fallback Chain (HTTP $\rightarrow$ Firecrawl $\rightarrow$ SingleFile Headless Chromium), YouTube video summaries, automatic conversion of PDF / Word / PPT / Excel / EPUB to Markdown, and one-click generation of FB / LinkedIn / Threads viral social media copy Carousels. - Bookmarks and Grounding Deep Research Reports: /save, /list, and /search bookmark management. Clicking "📄 Detailed Research Report" on a card combines Google Search Grounding to write a complete Markdown report, rendered as an independent HTML page (/reports/{id}) and permanently saved in Cloud Firestore. - Google Maps Grounding and Gemini Batch API Gourmet Big Data Analysis: Send LINE location coordinates to search for nearby gas stations, parking lots, and restaurants. It can also pull 50+ latest reviews for specific restaurants and feed them into the Gemini Batch API (enjoying a 50% cost discount) for deep analysis of signature dishes and pitfalls, pushed back to LINE via Webhook or background polling. - Agentic Vision Visual Annotation: After uploading an image, gemini-3-flash-preview writes Python code to draw bounding boxes, which are then saved as images and pushed back to the chat room. - LINE Voice Message TTS and LIFF Real-time Two-way Voice Assistant: Voice messages are automatically transcribed and paired with gemini-3.1-flash-tts-preview + ffmpeg to generate M4A voice readings. Opening LIFF allows two-way streaming of gemini-3.1-flash-live-preview via WebSocket, supporting hands-free VAD, Barge-in interruption, and Live Tool Calling. The features are great, but the cost is obvious: the resident memory for Python + FastAPI + a large number of document parsing and AI packages kept increasing. To ensure background tasks and multi-concurrency didn't lag, this Cloud Run service was unknowingly scaled up to 4 vCPU / 4 GiB Memory (with --no-cpu-throttling enabled). As someone who has been writing Go for a long time, seeing a personal daily assistant consume 4 vCPU / 4 GiB was becoming hard to stomach. So, I decided to do one thing: rewrite the entire linebot-helper-python 100% into the latest Go 1.26 version (kkdai/linebot-helper-go), deploy it to Cloud Run in the same GCP project, and ensure it shares Firestore data with the old version for zero-downtime switching at any time. - * * 1. Setting Rules Before Rewriting: Breaking Down 11 Milestones with GitHub Issues Moving a nearly 90KB main.py plus a dozen modules from Python to Go can easily result in an unmaintainable maze if boundaries aren't defined and testing isn't done along the way. Therefore, after creating kkdai/linebot-helper-go, the first task wasn't writing code, but mapping out all features from the Python version. I opened Tracking Issue #12 and 11 sub-task Issues (#1 ~ #11) on GitHub, with each Issue corresponding to an independent package and complete unit tests: | Milestone Issue | Corresponding Go Package | Core Responsibilities and Compatibility Highlights | | --- | --- | --- | | #1 Project Skeleton and Dockerfile | main.go, internal/config | Go 1.26 net/http, 100% environment variable compatibility with Python version, Multi-stage Dockerfile (integrating Chromium / single-file / ffmpeg) | | #2 Gemini Dual-channel Client | internal/gemini | Supports both Vertex AI (ADC) and Gemini Developer API (GOOGLE_AI_API_KEY), covering Grounding, Code Execution, and TTS | | #3 Firestore Persistence Layer | internal/firestore, internal/services | Fully compatible with Python version's chat_sessions, bookmarks, reports, and batch_jobs collection schemas; auto-downgrades to in-memory storage without credentials | | #4 LINE Messaging API and Flex | internal/line | HMAC-SHA256 signature verification, Reply / Push / Loading Animation, Social copy and Bookmark Flex Carousel assembly | | #5 URL Crawler and Multi-format Doc Conversion | internal/loader | HTTP (including PTT over18=1) $\rightarrow$ Firecrawl $\rightarrow$ SingleFile three-layer time-budget Fallback, and PDF / DOCX / PPTX / XLSX / EPUB / ODF to Markdown | | #6 AI Toolset | internal/ai | Summaries, viral social copy, Grounding research reports, YouTube analysis, Agentic Vision, GitHub Issues aggregation, Maps search | | #7 ADK Orchestrator and Specialized Agents | internal/agents | Integrates official google.golang.org/adk v1.9.0, establishing a Root Orchestrator and 5 specialized Agents: Chat / Content / Location / Vision / GitHub | | #8 Asynchronous Webhook and Postback | internal/handlers | Webhook returns 200 OK instantly, background Goroutines handle events, 9 types of Postback dispatching, and /reports/{id}, /images/{id}, /audio/{id} endpoints | | #9 Gemini Batch API Gourmet Analysis | internal/services/batch_service.go | JSONL batch job upload, Static HMAC Webhook, Dynamic RS256 JWKS Webhook verification, and 60s background polling | | #10 LIFF Real-time Voice Assistant | internal/voice, static/liff | /ws/voice/{session_id} two-way WebSocket Relay, Push-to-Talk / hands-free VAD, Barge-in, Session Resumption, and Live Tool Calling | | #11 Cloud Run Deployment and Verification | Cloud Run (us-central1) | Deployed using existing Python version environment variables, stress tested, and container resource specs downsized | The overall architecture is shown in the diagram below: flowchart TB subgraph Client["LINE Client and LIFF"] LINE["LINE App Chatroom\n(Text / URL / Image / Voice / Location / Postback)"] LIFF["LIFF Real-time Voice Assistant\n(/liff/ + WebSocket PCM 16kHz/24kHz)"] end subgraph CloudRun["Google Cloud Run: linebot-helper-go (Go 1.26, 1 vCPU / 1 GiB)"] Mux["Go 1.26 net/http Server\n(HMAC Verification + Immediate 200 OK)"] VoiceRelay["internal/voice\nGemini Live BidiGenerateContent Relay"] Orch["internal/agents (google.golang.org/adk v1.9.0)\nRoot Orchestrator + 5 Sub-Agents"] Loader["internal/loader\nHTTP -> Firecrawl -> SingleFile\n+ PDF/Office/EPUB to Markdown"] BatchSvc["internal/services/batch_service\nGemini Batch API + JWKS/HMAC Webhook + 60s Poller"] end subgraph GCP["Google Cloud & AI Backend"] Vertex["Vertex AI / Gemini Developer API\n(Search Grounding / Maps / Vision / TTS / Live)"] FS[("Cloud Firestore\nchat_sessions | bookmarks\nreports | batch_jobs")] end LINE -->|"POST /"| Mux LIFF <-->|"GET /ws/voice/{id}"| VoiceRelay Mux --> Orch Mux --> Loader Mux --> BatchSvc VoiceRelay <-->|"WebSocket"| Vertex VoiceRelay -.->|"delegate_to_orchestrator"| Orch Orch --> Vertex Orch --> Loader Orch <--> FS BatchSvc <--> Vertex BatchSvc <--> FS - * * 2. Five Engineering Designs Particularly Noteworthy During the Rewriting Process 1. Dual-track Gemini Client for Vertex AI and Developer API In this LINE Bot, not all features use the same Gemini endpoint: - General Chat, Grounding Search, Google Maps Grounding, Image Analysis: Uses Vertex AI (using Application Default Credentials of the Cloud Run service account, no API Key required). - Native TTS Voice Synthesis (gemini-3.1-flash-tts-preview), Gemini Batch API, and Gemini Live API WebSocket: Uses Gemini Developer API (using GOOGLE_AI_API_KEY). - Agentic Vision (gemini-3-flash-preview + Code Execution): Prioritizes Developer API, with automatic Fallback to Vertex AI if no Key is set. In the Go version's internal/gemini/client.go, we converged these two paths under the same gemini.Client. Even converting the Raw PCM (24kHz, 16-bit, mono) returned by TTS into LINE-supported audio/mp4 (.m4a) via ffmpeg and calculating audio milliseconds is handled seamlessly within the same module. 2. 100% Firestore Schema Compatibility with the Python Version (Key to Zero-Downtime Switching) The biggest fear when rewriting a service is that "after switching, user chat history, half-saved bookmarks, or previously generated research report URLs will all become invalid." To prevent this, we strictly aligned the 4 Firestore Collection names and field structures in internal/firestore and internal/services with the Python version: - chat_sessions: Stores multi-turn chat history and the last GPS coordinates sent by the user (last_location). - bookmarks: Stores /save bookmarks, viral social copy candidate bookmarks (status: candidate $\rightarrow$ saved), and 7-day cached report_id and report_generated_at. - reports: Stores the fully rendered HTML of Grounding deep research reports (supporting permanent retrieval via /reports/{report_id}). - batch_jobs: Stores the mapping between Gemini Batch API task IDs and LINE user_id. This means the Python version and Go version can simultaneously read and write to the same Firestore online. The moment the LINE Webhook is pointed to the new version, all old bookmarks and report links remain completely unaffected; if I ever want to switch back to the Python version, I only need to change one line of the Webhook URL. 3. Native Go Implementation for Multi-format Document to Markdown Conversion (Saying Goodbye to Heavy Python Packages) The Python version used packages like anydoc to convert PDF, DOCX, PPTX, XLSX, ODT, EPUB, CSV, and RTF to Markdown before feeding them to Gemini for summarization. These packages in Python often pull in a long list of C extensions and dependencies. In the Go version internal/loader/doc_converter.go, since .docx, .pptx, .xlsx, .odt, and .epub are essentially ZIP-packaged XML files, we used the Go standard library's archive/zip + encoding/xml along with a lightweight PDF text extractor. With less than a few hundred lines of pure Go code, we can cleanly convert all mainstream office documents and e-book formats into structured Markdown. It compiles fast and has almost zero extra memory overhead at runtime! - * * 3. Facing the Architecture Honestly: Python Version's "Conceptual ADK" vs. Go Version's Actual Implementation of google.golang.org/adk v1.9.0 While checking the code, we found an interesting historical trace: If you open the beginning of the original Python version's agents/orchestrator.py, you'll see a comment saying: > Orchestrator Agent - Main control Agent based on Google ADK But if you look at the Python version's requirements.txt and the entire source code, you'll find that google-adk isn't actually installed, and there isn't a single line of from google.adk import ...! The Python version at that time merely "borrowed the hierarchical concept of ADK," manually creating Orchestrator, ChatAgent, ContentAgent, etc., using pure Python classes, with the underlying layer calling google-genai directly. For this Go rewrite, we decided to do it for real—directly introducing the latest official Google google.golang.org/adk v1.9.0 in go.mod (paired with google.golang.org/genai v1.57.0): | Comparison Item | Original Python Version (linebot-helper-python) | New Go Version (linebot-helper-go) | | --- | --- | --- | | ADK Package Dependency | ❌ None (not installed) | ✅ google.golang.org/adk v1.9.0 | | Agent Definition | Custom Python Class | Uses llmagent.New(llmagent.Config{...}) to create agent.Agent | | Sub-Agents Hierarchy | Manual object attributes | Root orchestrator mounts 5 specialized sub-agents via SubAgents: []agent.Agent{...} | | Tools Binding | Manual method calls | Uses functiontool.New() to register type-safe Go function tools (load_url_content, summarize_youtube_video, search_nearby_places) | | Model Interface | Direct call to google.genai.Client | Implements google.golang.org/adk/model.LLM and adkgemini.NewModel (Vertex AI Backend) | Next Step: From "Building an ADK Agent Tree" to "Fully Driven by ADK Runner" (Issue #13) To ensure the first migration version passes all existing test cases 100%, ProcessText currently retains the original fast intent routing. However, we have already planned the following 5 exciting ADK upgrades in Issue #13 ([RFC] Google ADK (v1.9.0) Deep Integration and Architecture Evolution Proposal): 1. Replace Hardcoded Regex Routing with runner.Runner + SubAgents / agenttool: When a user enters a composite command in LINE (e.g., "Help me find the restaurant mentioned in this article and save it to bookmarks"), ADK will automatically transfer or call multiple Sub-Agents in parallel. 2. Add BookmarkAgent and Research Report functiontool: Users won't need to remember /save or /search slash commands; they can simply say "Find the bookmark about MCP I saved last time and make a research report" to trigger it. 3. Implement adk/session.Service for Cloud Firestore: Wrap the existing firestore.KVStore into a standard session.Service, allowing ADK Runner to natively manage multi-turn conversation events and states. 4. Introduce workflowagents (sequentialagent / parallelagent): Deconstruct deep research reports into a sequential workflow of "Extraction $\rightarrow$ Google Search Verification $\rightarrow$ Report Writing"; change viral copy for FB / LinkedIn / Threads to parallel Sub-Agents for simultaneous output. 5. Connect LIFF Real-time Voice Assistant's delegate_to_orchestrator directly to ADK Runner. - * * 4. Cloud Run Deployment and Slimming Down: Machine Specs Cut Directly to 1/4! After completing all 10 core packages and unit tests (go vet ./... && go test ./... all passed), we deployed linebot-helper-go directly to Cloud Run. Let's look at the container resource configuration comparison between the Python and Go versions on the same GCP project: | Cloud Run Service | Runtime Environment | CPU | Memory | Concurrency | CPU Allocation Mode | Relative Resource Cost | | --- | --- | --- | --- | --- | --- | --- | | linebot-helper-python | Python 3.11 + FastAPI | 4 vCPU | 4 GiB | 80 | Always allocated (--no-cpu-throttling) | 100% (Baseline) | | linebot-helper-go | Go 1.26 Native Compilation | 1 vCPU | 1 GiB | 160 | Always allocated (--no-cpu-throttling) | 25% (75% Savings) | You might ask: The resident memory of the Go compiled executable is only 20–30 MB, why not just use 512 MiB instead of 1 vCPU / 1 GiB? Both numbers have practical engineering considerations behind them: 1. Why 1 vCPU? Because the LINE Bot needs to return 200 OK instantly upon receiving a Webhook and continue running Gemini generation and research report rendering in background Goroutines. Plus, there's a background Ticker polling the Gemini Batch API every 60 seconds. Therefore, --no-cpu-throttling (CPU always allocated) must be enabled. In Cloud Run specifications, the minimum CPU threshold for enabling --no-cpu-throttling is 1 vCPU. 2. Why 1 GiB Memory? The Go program itself is indeed under 30 MB, but don't forget our 3rd layer Fallback for the URL crawler has single-file + Headless Chromium installed in the container (used to deal with dynamic rendering sites that even Firecrawl blocks), plus voice message synthesis requires calling ffmpeg. When Headless Chromium is occasionally awakened, it instantly consumes 300–400 MB of memory. Keeping 1 GiB ensures Chromium never OOMs (Out of Memory) upon startup while cutting machine costs to 1/4 of the original Python version! The deployment command is very clean: gcloud run deploy linebot-helper-go \ --source . \ --project <YOUR_GCP_PROJECT_ID> \ --region us-central1 \ --cpu 1 \ --memory 1Gi \ --allow-unauthenticated \ --no-cpu-throttling - * * 5. Seven Devilish Details Most Likely to be Tripped Over or Missed in Cloud Deployment Moving a LINE Bot from local testing to Cloud Run, or switching from an old Python service to a new Go service, "successful compilation" often only accounts for 60% of the work. The real traps are usually hidden in Cloud Run's runtime lifecycle, network proxies, container sandboxing, and external endpoint settings. Here are the 7 key blind spots most easily overlooked during this deployment process: Omission 1: Forgot to add --no-cpu-throttling, or passed r.Context() into a background Goroutine LINE Webhook specifications require the server to return 200 OK quickly after receiving an event. Our heavy tasks (such as Grounding deep research reports taking 1–2 minutes, Agentic Vision writing code to draw, voice transcription plus TTS, and the Gemini Batch API Poller polling every 60 seconds) all return 200 OK first, then run go func() in the background and use PushMessage to push the result. There are two traps here that both beginners and experts are extremely likely to fall into: 1. Cloud Run "freezes the CPU" by default the moment the HTTP Response is sent: If you don't add --no-cpu-throttling, as soon as handleWebhookCallback returns 200 OK, Cloud Run will drop the container CPU to near 0 within tens of milliseconds. Your background Goroutine will be stuck in mid-air like it was paused, only moving a few steps when the next HTTP request comes in minutes later. The user will never receive the push! 2. Background Goroutines must never use r.Context(): In Go's net/http, r.Context() is triggered as Canceled the moment the HTTP Handler ends (sends 200 OK)! If you casually write go s.handleMessageEvent(r.Context(), event), background requests calling Vertex AI or Firestore will die directly with a context canceled error. // ❌ Incorrect: r.Context() will be canceled the moment 200 OK is returned; background tasks will fail. go s.handleMessageEvent(r.Context(), eventCopy) // ✅ Correct: Detach from the HTTP Request lifecycle, use an independent Background Context (set Timeout internally). go s.handleMessageEvent(context.Background(), eventCopy) Omission 2: Vertex AI GOOGLE_CLOUD_LOCATION=global Domain Concatenation Trap When reusing old environment variables, we saw GOOGLE_CLOUD_LOCATION=global (because many Gemini 3.x Preview models in Vertex AI are under the global location). If you write a Vertex AI REST Client in Go and follow the habit for general regions by using fmt.Sprintf("https://%s-aiplatform.googleapis.com", location), when location is "global", the resulting URL becomes: > ❌ https://global-aiplatform.googleapis.com (This domain doesn't exist in GCP; DNS lookup will fail!) In Vertex AI specifications, only regional endpoints (like us-central1, asia-east1) have a prefix; when the Location is global, you must switch to the prefix-less https://aiplatform.googleapis.com: func (c *Client) vertexEndpointBase() string { if c.VertexBaseURL != "" { return strings.TrimRight(c.VertexBaseURL, "/") } if c.Location == "" || c.Location == "global" { return "https://aiplatform.googleapis.com" } return fmt.Sprintf("https://%s-aiplatform.googleapis.com", c.Location) } Omission 3: Moving Environment Variables Using --set-env-vars Leading to Character Truncation or Secret Leakage When moving over a dozen environment variables from linebot-helper-python to linebot-helper-go, never manually string together --set-env-vars="KEY1=VAL1,KEY2=VAL2...": 1. If any Token or Prompt string contains a comma , or equals sign =, gcloud's parameter parsing will split at the wrong position or even error out. 2. Putting environment variables directly in the command line parameters leaves the entire Secret string in the terminal history or conversation logs. The safest way: Directly export the temporary /tmp/env.yaml locally using gcloud run services describe --format=json, feed it in during deployment using --env-vars-file=/tmp/env.yaml, and rm -f /tmp/env.yaml immediately after deployment. This way, no Secrets are printed, and all special characters are 100% preserved. Omission 4: Ignoring Cloud Run Reverse Proxy's X-Forwarded-Proto, Leading to LINE Rejecting Images and Audio This LINE Bot needs to dynamically assemble "public URLs pointing to its own server" in four places: 1. Research report page: https://<host>/reports/{report_id} 2. Agentic Vision annotated image: https://<host>/images/{image_id} 3. TTS voice synthesis audio file: https://<host>/audio/{audio_id} 4. Gemini Batch API Dynamic Webhook: https://<host>/api/gemini-callback/dynamic Note: LINE Messaging API's ImageMessage and AudioMessage strictly require URLs to be https://. In the Cloud Run architecture, HTTPS TLS is terminated at the outer Google Front End (GFE), so r.TLS is nil when it reaches the Go http.Request inside the container. If you don't read the X-Forwarded-Proto and X-Forwarded-Host headers, the assembled http:// URL will be rejected by the LINE Push API! Omission 5: Pursuing Minimalist Go Images (scratch / distroless) but Forgetting External Executables and Chromium Sandbox Restrictions When writing Go services, we often use FROM gcr.io/distroless/static to compress the Docker Image to under 20MB. But doing this for this multi-functional LINE Bot will immediately hit a wall: 1. Missing ffmpeg: After receiving PCM from Gemini for TTS synthesis, calling exec.Command("ffmpeg", ...) to convert to .m4a will report executable file not found. 2. Missing Headless Chromium, Chinese Fonts, and Container Sandbox Flags: The 3rd layer Fallback of the crawler uses single-file to drive Headless Chromium. In Cloud Run's container sandbox environment, if you don't add --no-sandbox, --disable-gpu, and --disable-dev-shm-usage (because the container's default /dev/shm shared memory is small) to Chromium, or if the base image doesn't have fonts-noto-cjk installed (resulting in garbled text on captured Chinese web pages) and ca-certificates (HTTPS certificate verification failure), the crawler will silently crash in the cloud. Therefore, our Dockerfile uses golang:1.26-bookworm for compilation, and the Runtime stage uses debian:bookworm-slim with explicit installation of ca-certificates, ffmpeg, chromium, fonts-noto-cjk, and @single-file/cli. Omission 6: State Splitting Between In-Memory Caching and Cloud Run Auto-scaling (Multi-Instance) Although we have persisted the four core states—chat history, bookmarks, research reports, and Batch Job mappings—to Cloud Firestore, some binary data with a "lifecycle of only 5 minutes" is stored in a Go in-memory map: - Images just uploaded by users, waiting for the "Identify Image / Agentic Vision" Quick Reply selection (imageTempStore). - PNGs just drawn with bounding boxes, waiting for the LINE server to pull them (/images/{image_id}). - M4A audio files just synthesized, waiting for the LINE server to pull them (/audio/{audio_id}). This works perfectly under a single instance, but once Cloud Run scales to 2 or more instances due to a burst in traffic, state splitting occurs: - User sends an image to Instance A (image stored in A's memory). - 3 seconds later, the user clicks the "Agentic Vision" button, and the Webhook hits Instance B $\rightarrow$ B can't find the image and replies "⚠️ Image expired, please resend!" - Or Instance A generates /audio/{id} and calls the LINE Reply API, but when the LINE media server comes to download the audio, it hits Instance B, gets a 404, and the user only sees an unplayable voice bubble in the chat room. Solution: Since a single Go instance can easily handle high concurrency (we set containerConcurrency to 160), for personal or small-to-medium assistants, it's recommended to limit --max-instances to 1 (if multi-instance high availability is needed in the future, the caching for /images and /audio must be rewritten to Cloud Storage or Firestore). Omission 7: Only Changing the Messaging API Webhook During Switching, but Missing Firebase Hosting WebSocket Restrictions and LIFF Endpoint When preparing to switch from the old service to the new one, the following three peripheral settings are most commonly missed: | Checkpoint | Why is it easily missed? | Actual Impact and Correct Practice | | --- | --- | --- | | 1. Is the existing Webhook hidden behind Firebase Hosting? | When querying existing settings via LINE API (GET /v2/bot/channel/webhook/endpoint), we found the original Webhook pointed to *.firebaseapp.com (rewritten to Cloud Run by firebase.json). | Firebase Hosting does not support WebSocket proxying! If the LIFF voice assistant (/ws/voice/{id}) goes through the Firebase Hosting domain, the WebSocket handshake will fail. Therefore, when switching, point directly to the Cloud Run URL or ensure LIFF uses a direct connection. | | 2. LIFF Endpoint URL of the LINE Login Channel | LIFF App settings are not in the Messaging API Channel but in the LINE Login Channel tab under the same Provider. | If you only change the Webhook and not the LIFF Endpoint, the chat room text will be handled by the new Go version, but opening the voice assistant will still connect back to the old Python version. | | 3. Gemini Batch API Static Webhook / Eventarc | If Eventarc or a project-level static Webhook is set in GCP to point to /api/gemini-callback/static. | Remember to update the target URL as well (while the Dynamic Webhook /api/gemini-callback/dynamic will automatically capture the Host of the incoming request, requiring no manual modification). | - * * 6. Inventory Before Switching: Identifying Three "Zombie Endpoints" from GCP Logs and Planning to Incorporate n8n Schedules! While writing the "Switching from Python Version" steps in the README, I originally listed: > "Change external schedules calling /hn, /hf, and /urls to point to the new service." But then I thought: Wait, are these three endpoints actually still being called by my n8n? We immediately used gcloud logging read to pull execution records for the linebot-helper-python and n8n services in the GCP project, and found a big surprise: - /hn, /hf, and /urls were endpoints written in the first version in November 2024, intended for external schedules to POST URL arrays for the LINE Bot to summarize and push. - But the n8n service currently in the cloud (running four active workflows: "RSS to Sheet", "Daily Tasks (Fetch Papers/Share Videos)", "Daily LINE Sender", and "AI Trends Automated Newsletter") had already been changed to run Firecrawl internally in n8n, call Gemini for summaries, assemble Flex JSON, and directly hit https://api.line.me/v2/bot/message/push! - In other words, the number of calls to /hn, /hf, and /urls on linebot-helper-python from n8n over the past few months was 0! Even more interestingly, while checking n8n's Debug Logs, we accidentally caught n8n failing to push YouTube new video summaries at 6 AM because of word count or thumbnail format issues in the Flex Carousel assembled by the Javascript node, with the LINE API returning: Workflow execution finished with error: A message (messages[0]) in the request body is invalid This discovery led us to make two immediate decisions: Decision 1: Immediately remove /hn, /hf, and /urls from linebot-helper-go Since no external services are calling them, keeping them on the public Cloud Run only adds three attack surfaces without signature protection. After removing them from server.go and redeploying, hitting /hn, /hf, or /urls online returns 404 Not Found. When switching, I only need to change the Webhook URL and LIFF Endpoint URL in the LINE Developers Console! Decision 2: Open Issue #14 ~ Issue #19 to incorporate n8n workflows into linebot-helper-go! Since linebot-helper-go is already a high-speed Go server always resident at 1 vCPU / 1 GiB (--no-cpu-throttling), and it has a built-in three-layer crawler loader.Loader, strongly-typed LINE Flex Builder, Gemini Batch API, Firestore BookmarkService, and ReportStore, why maintain a separate n8n + Postgres database? More importantly—if these daily schedules are all moved to linebot-helper-go and pushed through the same LINE Official Account, a perfect closed loop of "Push $\rightarrow$ Interaction" can be achieved: - Issue #14 (Batch Task 1/5): Establish a protected POST /api/tasks/{task_name} schedule endpoint and a Firestore feed_items deduplication queue (replacing the old method of comparing rows in Google Sheets), unifying pushes using the main official account. - Issue #15 (Batch Task 2/5): Fetch Hugging Face Daily Papers every day at 10:00. When pushing paper summary cards, directly attach "🔖 Save Bookmark" and "📄 Detailed Research Report" buttons, allowing one-click generation of Grounding deep web reports for good papers. - Issue #16 (Batch Task 3/5): Inspect YouTube subscription channel new videos every day at 06:00. Use Go's strongly-typed Flex Builder to completely solve n8n's 400 Invalid Request Body issue, and attach a "📝 Detailed Summary" Postback button directly to the card. - Issue #17 (Batch Task 4/5): Integrate multi-source RSS inspection from HN, Hatena IT, Google Dev Blog, LY Blog, Steam, etc., with daily 06:00 featured pushes. - Issue #18 (Batch Task 5/5): Aggregate weekly featured content and bookmarks every Friday at 17:00, rendering them directly into a beautiful /reports/{id} web version AI Trends Weekly pushed to LINE. - * * 7. One Last Security Reminder: Don't Leave Real Cloud Run URLs in Public Git History After writing the deployment records and switching tutorial, I did one very important cleanup task: replaced all occurrences of .run.app real service URLs, GCP Project Numbers, and LIFF IDs in README.md and GitHub Issue comments with <YOUR_CLOUD_RUN_SERVICE_URL> placeholders, and rewrote the Git Commit history to eliminate traces. The reason is simple: The LINE Bot's Cloud Run service must be set to --allow-unauthenticated to receive Webhooks from the LINE platform. Although our POST / has strict X-Line-Signature verification (requests without a Channel Secret signature are blocked with a 400), writing the real .run.app URL directly in a public GitHub Repo's README still easily attracts unnecessary automated crawler scans and traffic noise. Sharing architecture and implementation with public code while keeping real endpoints only in your own configuration backend is the most reassuring way to maintain a personal AI assistant in the long run. - * * 8. Cloud Run Deployment and Migration Self-Checklist If you also plan to rewrite an existing Python LINE Bot into Go and move it to Cloud Run, you can use this checklist to tick off items before switching traffic: - <input type="checkbox" disabled> 1. CPU Throttling Setting: As long as there are background Goroutines (Webhook returns 200 OK followed by background push, or background periodic Ticker), be sure to add --no-cpu-throttling (and set CPU to at least 1). - <input type="checkbox" disabled> 2. Context Lifecycle Decoupling: Background Goroutines are prohibited from passing in r.Context(); always use context.Background() with an independent context.WithTimeout. - <input type="checkbox" disabled> 3. Vertex AI global Domain Judgment: If GOOGLE_CLOUD_LOCATION=global, confirm the Endpoint Base URL is https://aiplatform.googleapis.com instead of global-aiplatform.googleapis.com. - <input type="checkbox" disabled> 4. External Binary Tools and Fonts: If ffmpeg, single-file, or Headless Chromium are called inside the container, confirm the final stage Docker Image includes the corresponding executables, ca-certificates, fonts-noto-cjk, and Chromium's --no-sandbox --disable-dev-shm-usage parameters. - <input type="checkbox" disabled> 5. Reverse Proxy HTTPS Headers: When dynamically generating /images/{id}, /audio/{id}, and /reports/{id} URLs, prioritize reading X-Forwarded-Proto and X-Forwarded-Host to ensure media URLs sent to LINE are always https://. - <input type="checkbox" disabled> 6. In-Memory Caching and Instance Count: If there are short-lived memory maps (like cached images/audio), it's recommended to set --max-instances=1 and increase concurrency, or switch to GCS / Firestore to avoid state splitting across multiple instances. - <input type="checkbox" disabled> 7. Dual-channel LINE Setting Synchronization: In addition to updating the Messaging API Webhook URL, don't forget to update the LIFF Endpoint URL of the LINE Login Channel under the same Provider, and confirm the WebSocket path is not intercepted by a CDN / Firebase Hosting that doesn't support WS. - <input type="checkbox" disabled> 8. Clean Up Obsolete Endpoints and Public Information: Confirm useless old endpoints (like /hn, /hf, /urls) from Cloud Logging and remove them, and ensure the public Repo and Git history do not contain real .run.app URLs and keys. - * * Summary and Open Source Project Links From this rewriting journey from kkdai/linebot-helper-python to kkdai/linebot-helper-go, here are the most valuable takeaways: 1. Open Issues Before Writing Code: 11 milestone Issues made the large-scale language rewrite highly controllable, with each module independently passing unit tests before being layered. 2. Shared Data Layer is the Foundation for Zero-Downtime Migration: As long as the Firestore Collection Schemas are aligned, services in the two languages can be switched painlessly at any time. 3. Go's Resource Efficiency on Cloud Run is Extremely High: Even with Headless Chromium and ffmpeg mounted, switching from Python's 4 vCPU / 4 GiB to Go's 1 vCPU / 1 GiB cut the bill to 25%, with even faster response times. 4. Check Cloud Logging Before Switching: Real traffic logs don't lie; you can see at a glance which endpoints are already retired and which external n8n workflows are actually better suited to be incorporated into the same service. The complete source code and subsequent ADK v1.9.0 / Batch Tasks evolution plans are all open-sourced on GitHub: kkdai/linebot-helper-go. Feel free to refer to it and exchange ideas!

[AI in Practice] Upgrading from Gemini 3.5/3.6 to Gemini 4 Argon: Migration Notes and Pitfalls for LINE Bot Developers !Stylized promotional blogger visual with modern editorial branding and the text "Gemini 4 Argon". Background I have a LINE Bot that I use every day, linebot-helper-python. I use it daily to summarize URLs, ask follow-up questions about YouTube video content, search for map locations, and manage bookmarks. In terms of model configuration, to balance response speed and budget, many workflows were previously running on Gemini 3.5 (such as gemini-3.5-flash-lite for high-throughput, low-latency classification and data extraction) and Gemini 3.6 (such as gemini-3.6-flash for multi-step tool calling and multimodal reasoning). In mid-October, Google officially pushed Gemini 4 Argon (gemini-4-argon) to the GA stable version (paired with Python google-genai >= 2.30.1). After reading the official announcement and migration documents, the most attractive features on the surface are the big numbers: a 1M unified token window shared by input, thinking, and output; Long Decode that automatically relays a single call up to 1 million output tokens; Agentic Video Understanding which is now officially GA and saves 88% of tokens; and the Interactions API (/v1beta/interactions), which has become the default mainstream. But if your goal is the same as mine—to upgrade (or partially switch) the existing Gemini 3.5 / 3.6 in your LINE Bot to Gemini 4 Argon—don't assume it's as simple as changing model="gemini-3.6-flash" to model="gemini-4-argon". This generation has several changes in parameter specifications, token budget calculation methods, caching mechanisms, and even assumptions about waiting times that hit the pain points of LINE Bots exactly. This article organizes all the details you need to pay attention to when migrating from 3.5 / 3.6 to Gemini 4. - * * Specification Comparison: 3.5 Flash-Lite / 3.6 Flash vs. Gemini 4 Argon Before diving into settings, let's look at the positioning differences of these three models in one table: | Item | gemini-3.5-flash-lite | gemini-3.6-flash | gemini-4-argon | | --- | --- | --- | --- | | Positioning | Ultra-fast, low-cost, high-throughput classification & structured extraction | Balance of speed, multi-step Agent, and multimodality | Flagship deep reasoning, complex coding, long output, and autonomous agents | | Context Window | 1,048,576 | 1,048,576 | 1,048,576 (Shared Input + Thinking + Output) | | Max Single Output | 65,536 | 65,536 | generateContent: Non-streaming 32,768 / Streaming 65,536 <br>Interactions API: Synchronous 65,536 <br>_(Long Decode can continue up to 1M for both)_ | | Default thinking_level | "minimal" | "medium" | "medium" (Supports only low / medium / high) | | Context Caching | Supported | Supported | Implicit Caching only (Calling Explicit will error) | | temperature / top_p / top_k | Deprecated (Ignored) | Deprecated (Ignored) | Deprecated (Ignored, official requirement to remove) | | Pricing (Input / Output) | $0.30 / $2.50 (per 1M) | $1.50 / $7.50 (per 1M) | Early bird until 2026/11/30: $2.00 / $10.00 <br>_(Reverts to $4.00 / $20.00 from 12/1; 95% discount for cache hits)_ | | Free Tier | Available | Available | None (Must be linked to Paid Tier billing) | Just by looking at this table, you can identify several pitfalls you'll encounter if you only change the model name. Let's break them down one by one. - * * Pitfall 1: LINE Bots usually set max_output_tokens very small, which returns an empty string in Gemini 4 This is the first hidden trap that LINE Bot developers are most likely to fall into. Because a single LINE text message has a 5,000-character limit, and mobile screens are not suitable for long-winded content, many people (including myself in the past) are used to setting max_output_tokens to 2048 or 4096 in GenerateContentConfig as a safeguard to save money and prevent text overflow. If you set it this way on Gemini 4 Argon, when encountering a slightly complex problem, you will find the model returns finish_reason: "MAX_TOKENS", but response.text is empty, or it cuts off after barely two sentences. The reason is that Gemini 4 Argon adopts a Unified 1M Input-Output Window, and has two hard rules: 1. Thinking tokens are counted directly towards the max_output_tokens budget: Every thinking token the model spends before answering is deducted from your max_output_tokens. When a question requires deep reasoning, the model can easily burn through your 4096 limit just by thinking, hitting a wall and stopping before it even starts outputting the formal answer. 2. generateContent non-streaming single request limit decreased from 64K to 32K: In Gemini 3.5 / 3.6, the single non-streaming response limit was 65,536; in gemini-4-argon, if you continue using traditional generateContent, the non-streaming single limit has been lowered to 32,768 tokens (streaming or using the new Interactions API synchronous call remains at 65,536). Cause and Solution: The official migration guide states clearly—if your old project set max_output_tokens below 32K, please increase it to at least 32768 (or just keep the default 65536) when upgrading to gemini-4-argon. In Gemini 4, max_output_tokens is used as a "total emergency brake to prevent burning through money in extreme situations," not to control the length of the answer. - To spend fewer tokens and speed up response: Lower the thinking_level="low". - To make the output fit LINE message length: Write it in the system_instruction (e.g., "Please keep it within 400 words and use bullet points"). # ❌ Old safeguard setting often used in LINE Bots; thinking process will consume all 2048 tokens on gemini-4-argon config = types.GenerateContentConfig( max_output_tokens=2048, ) # ✅ Correct way for Gemini 4 Argon: Relax max_output_tokens, use thinking_level to control cost and latency config = types.GenerateContentConfig( max_output_tokens=32768, thinking_config=types.ThinkingConfig(thinking_level="low"), system_instruction="You are a LINE assistant. Keep answers concise and within 500 words.", ) - * * Pitfall 2: Differences in thinking_level settings—3.5-flash-lite's "minimal" cannot be moved directly If you have already changed the old-school thinking_budget (passing an integer token count) to thinking_level in Gemini 3.5 / 3.6, there is another small trap here: - On gemini-3.5-flash-lite, the default thinking level is called "minimal". - But on gemini-4-argon, only three levels are supported: "low", "medium" (default), and "high". If your config shares the same generator and you bring "minimal" into gemini-4-argon as is, or if you still have the old thinking_budget (although the current API will map it to the closest level, it will return a deprecation warning), it will make your settings messy. For LINE Bots, the practical division of these three levels in gemini-4-argon is very clear: - "low": The first choice for general LINE conversations, URL summaries, and real-time Q&A. Lowest token consumption and lowest Time to First Token (TTFT). - "medium" (default): Suitable for scenarios requiring multiple tool calls (Function Calling), reading long videos, or writing code. - "high": Deep math, complex itinerary optimization, long-form analysis. Note: Setting this to "high" makes it almost impossible to finish within the valid time of a LINE replyToken. You must use an asynchronous push architecture. - * * Pitfall 3: LINE Webhook's few-second lifespan vs. official recommendation of "20-minute Timeout" When reading the Gemini 4 Argon migration document, one item made me laugh out loud: > Increase client timeouts: Set HTTP client timeouts to at least 15–20 minutes per call. The official documentation even added a note in the Long Decode section: when facing difficult tasks, gemini-4-argon might spend an entire 32K or 64K hop entirely on thinking, only outputting the real answer in the next hop (under generateContent, the new SDK google-genai >= 2.30.1 has automatic_continuation=True enabled by default, which uses continuation_token to automatically loop for you on the client side; under the Interactions API, to exceed the single sync limit of 65536, you must enable background=True to let the server side run multiple hops). This is perfectly fine when running a CLI script on your own computer; but in a LINE Bot + Cloud Run architecture, waiting synchronously on the client side is a disaster: 1. LINE's replyToken has an extremely short lifespan and won't wait two minutes for you to think. 2. LINE users staring at a screen with no response for more than 10 seconds will start pressing the button a second or third time. 3. If you wait synchronously for automatic_continuation to finish several hops within the same Webhook request, Cloud Run connections and workers will quickly become deadlocked. Cause and Solution: To put Gemini 4 into a LINE Bot, you must do two things architecturally: 1. Don't phase out 3.5 / 3.6; instead, create a "Fast/Slow Model Router" The official documentation emphasizes that gemini-3.6-flash and gemini-3.5-flash-lite are still the main models for stable online supply. The healthiest LINE Bot architecture is not "switching everything to gemini-4-argon," but rather: - First-line instant replies / Intent classification / Simple structured extraction: Continue to hand these to gemini-3.5-flash-lite ($0.30/1M, extremely fast) or gemini-3.6-flash ($1.50/1M, concise, saves steps in tool calling). - Heavy reasoning / Long video follow-up / Complex tasks: Only then route to gemini-4-argon. 2. Use the new Interactions API with background=True + webhook_config for heavy tasks In this generation, Google has labeled our familiar generateContent as (Legacy) and is pushing the Gemini Interactions API (/v1beta/interactions) across its documentation. The greatest value of the Interactions API for LINE Bots is its native support for background=True and Webhooks (supporting project-level Static Webhooks or Dynamic webhook_config included in a single request): 1. A user asks a question on LINE that requires gemini-4-argon with "high" thinking or long output. 2. The LINE Bot first calls the LINE Loading Animation API and uses replyToken to reply instantly: "_This is a deep question. I am performing deep reasoning in the background and will push the result to you once finished!_" 3. Simultaneously call client.interactions.create(..., background=True, webhook_config=...), handing the task to Google's server to run multiple hops slowly. You can even put the line_user_id directly into user_metadata so Google returns it as-is during the callback. 4. When Google finishes and sends an interaction.completed event to your /webhooks/gemini endpoint, use client.interactions.get(interaction_id) to retrieve the result (if multiple Long Decode hops over 64K were run, all segments will be fully recorded in the model_output of interaction.steps). Then, calmly call LINE's push_message to push the result into the chat room. from google import genai client = genai.Client() def dispatch_heavy_line_task(line_user_id: str, prompt: str): # Hand over to Google server for background execution (even multi-round 64K hops for Long Decode will be automatically completed on the server side) interaction = client.interactions.create( model="gemini-4-argon", input=prompt, background=True, generation_config={ "max_output_tokens": 131072, "thinking_level": "high", }, webhook_config={ "uris": ["https://your-cloud-run.a.run.app/webhooks/gemini"], "user_metadata": {"line_user_id": line_user_id}, }, ) return interaction.id - * * Pitfall 4: Explicit Caching Errors—Prompt order determines if you save 95% If your LINE Bot contains a full product FAQ, long System Instructions, or large documents uploaded by users, you might have previously used client.caches.create() to create an Explicit Cache and then put cached_content into GenerateContentConfig to save money. After upgrading to gemini-4-argon, this code must be completely removed. gemini-4-argon does not support Explicit Caching. If you call caches.create or include cached_content in the config, the API will return an Error. Instead, it uses Implicit Caching, which is enabled by default: - As long as a single Prompt length reaches 4,096 tokens or more, and the preceding content (Prefix) matches a previous request, the system will automatically hit the cache. - Cache-hit Input Tokens receive a 95% discount (priced at 0.05x)—at early bird prices, this drops from $2.00 / 1M to $0.10 / 1M, which is cheaper than gemini-3.5-flash-lite! (Even when Long Decode automatically continues to the next hop, the repeated Context from before will automatically apply the 95% cache discount.) Key impact on LINE Bots: Since this has become automatic Prefix Matching, the order in which you assemble your Prompt determines the thickness of your bill: - Always put at the beginning (Immutable): Long system_instruction, shared background knowledge base, Few-shot examples, fixed background documents for that group. - Always put at the end (Changes every time): current_time timestamp, LINE user name/status, the current message. If you accidentally write "Current time: 2026-10-15 21:40:02" on the first line of your System Prompt, the first token of every request will be different, and the 95% cache discount will immediately drop to zero. - * * Pitfall 5: Strict rules tightening since 3.5 / 3.6—Sampling parameters, Prefill, and Thought Signatures These three rules have actually been in effect since gemini-3.5-flash-lite and gemini-3.6-flash, but if you are organizing your codebase across versions, be sure to clean them up together: 1. Remove temperature, top_p, top_k, candidate_count These four sampling parameters are all deprecated and ignored on Gemini 3.5 / 3.6 and Gemini 4 Argon (official notice says future generations will return HTTP 400 if these parameters are included). If you used to set temperature=0.0 to ensure stable output format, delete them all now and switch to Structured Outputs (response_schema) and clear system_instruction. 2. Prohibit Prefilled Model Turns (The last turn cannot be role: "model") A common Prompt technique used to be: to prevent the model from saying "Sure! Here is the summary..." or to force it to start outputting JSON from {, you would put {"role": "model", "parts": [{"text": "{"}]} as the last entry in the contents array. Now, if the last turn of non-empty content in the payload is the model role, the API will return an HTTP 400 Error. To suppress the opening remarks, please write it properly into the system_instruction. 3. Multi-turn conversations and Function Calling must preserve thought_signature (or just use previous_interaction_id) When a LINE Bot runs multi-step tool calls (e.g., first check bookmark DB $\rightarrow$ then fetch web content $\rightarrow$ finally summarize), the Part returned by the model will contain a thought_signature. If you manually maintain the conversation history array but only extract and reassemble the text and function_call, missing the thought_signature, the quality of multi-turn tool calls will degrade significantly or even error out. For those who don't want to manage thought_signature serialization themselves, the easiest way to upgrade to Gemini 4 is to use the Interactions API's previous_interaction_id: # Only remember the last interaction.id for that LINE user in Redis; let Google's server handle history state and thought_signature interaction = client.interactions.create( model="gemini-4-argon", input=user_message, previous_interaction_id=last_interaction_id, ) - * * The real weapon worth upgrading your LINE Bot for: Agentic Video Understanding GA After discussing all the pitfalls, let's talk about a positive upgrade for gemini-4-argon (and 3.8-flash) that is most noticeable for LINE Bots: Agentic Video Understanding is officially GA. One core feature of my LINE Bot is "drop a YouTube link and ask follow-up questions about the video content." Previously, traditional video understanding (static mode) involved taking a screenshot of the entire video every second (1 FPS) and stuffing it into the Context Window. For a 30-60 minute long lecture or Podcast video, even if the Context could fit it, the token consumption was staggering. After switching to Agentic mode (Note: it still defaults to static on all models including gemini-4-argon, so it must be explicitly enabled), the model's behavior changes to: 1. First read the audio transcript and timeline structure. 2. Based on the user's question, decide which timestamps to "scrub" to. 3. Dynamically load high-resolution and high-frame-rate frames only for key scenes to look closely. Whether you use the new Interactions API or the original generate_content, remember to add the corresponding parameters (parameter names and casing differ slightly between the two): # Method A: Using the new Interactions API (parameter is processing="agentic") interaction = client.interactions.create( model="gemini-4-argon", input=[ { "type": "video", "uri": youtube_url, "mime_type": "video/mp4", "processing": "agentic", # Remember to enable explicitly; default is still static }, { "type": "text", "text": "On the page where the speaker shows the architecture diagram, what are the three bottlenecks mentioned? Please include [MM:SS] timestamps.", }, ], generation_config={ "max_output_tokens": 32768, "thinking_level": "medium", }, ) # Method B: Using traditional generate_content (parameter is media_processing="AGENTIC") response = client.models.generate_content( model="gemini-4-argon", contents=[ types.Part.from_uri( file_uri=youtube_url, mime_type="video/mp4", media_processing="AGENTIC", # Remember to enable explicitly; default is still STATIC ), "On the page where the speaker shows the architecture diagram, what are the three bottlenecks mentioned? Please include [MM:SS] timestamps.", ], config=types.GenerateContentConfig( max_output_tokens=32768, thinking_config=types.ThinkingConfig(thinking_level="medium"), ), ) Official tests show that for long videos over 5-10 minutes and "needle in a haystack" detail questions, token usage is saved by up to 88%, and accuracy is improved by 7%. This point alone completely makes up for the higher unit price of gemini-4-argon. - * * Upgrade Checklist Finally, here is a PR self-check list for LINE Bot projects before upgrading from Gemini 3.5 / 3.6 to gemini-4-argon: 1. SDK and Billing Confirmation: Upgrade google-genai to >= 2.30.1; confirm the project associated with the API Key is linked to a Paid Tier (Free Tier and Google AI Pro cannot call gemini-4-argon). 2. Remove Sampling Parameters and Prefill: Search the entire project and remove temperature, top_p, top_k, candidate_count, and any prefilled prompts ending with role="model". 3. Remove Explicit Caching: Remove caches.create and cached_content; check Prompt assembly logic to ensure fixed, long System Instructions are placed at the beginning to trigger the 95% discount of Implicit Caching. 4. Increase max_output_tokens: Increase any limit smaller than 32768 to at least 32768 (to avoid empty responses caused by thinking tokens consuming the budget); move word count limits to system_instruction. 5. Check thinking_level Enum Values: Replace 3.5-flash-lite specific "minimal" or old thinking_budget with "low", "medium", or "high". 6. Fast/Slow Routing and Async Push: Tasks that need an instant reply within a few seconds for LINE replyToken should be set to thinking_level="low" or continue to be handled by 3.5-flash-lite / 3.6-flash; deep reasoning tasks should use the Interactions API (background=True + webhook_config) paired with LINE push_message. 7. Enable Agentic Mode for Video Understanding: When processing YouTube or long videos, add media_processing="AGENTIC" to Part.from_uri (or "processing": "agentic" for the Interactions API). If you use a Coding Agent that supports Skills (like Antigravity or Claude Code), Google also provides a migration Skill (/gemini-api-dev migrate my app to Gemini 4 Argon) that can help automatically scan for these parameter changes. However, architectural levels like "LINE replyToken timeout routing" and "Prompt prefix cache order" still need to be designed manually.