keras-team

Keras to TFLite on Android: wrong normalization and a backwards top-5 --- title: "Keras to TFLite on Android: wrong normalization and a backwards top-5" published: true description: "One export call put a KerasHub model on Android. Then I found two bugs in my own app: the wrong pixel numbers, and a top-5 list that keeps the worst five." tags: android, keras, tflite, machinelearning canonical_url: github.com/pctablet505/keras-ondevice-android/blo… cover_image: raw.githubusercontent.com series: Keras to LiteRT on Android --- For a photo of a quail, my Android app's top-5 code picks American alligator, sea cucumber, barracouta, reel and sea urchin. The model is right; the app is not. It keeps the five worst labels, and it scales the pixels with the wrong numbers. I added LiteRT export to Keras (keras#21674) and its PyTorch path (keras#22758), so the export was the part I trusted. The scores and labels below come from a run on 22 October 2026 on a computer's CPU, with the app's preprocessing and top-5 code copied into Python. I did not run the app on a phone for this post. The export: one call LiteRT is the new name of TensorFlow Lite; the files still end in .tflite. The keras.io guide installs Keras, KerasHub and litert-torch from their main branches. On 22 October 2026 that broke this export, with 'tfl.batch_matmul' op found invalid output rank, expected 3 but got 4. The released versions worked: pip install torch \ --index-url https://download.pytorch.org/whl/cpu pip install keras==3.15.1 keras-hub==0.32.0 \ litert-torch==0.10.0 ai-edge-litert==2.3.0 Then the whole export, from my app's README: # Run with KERAS_BACKEND=torch set in the shell. import keras_hub, torch m = keras_hub.models.MobileNetImageClassifier.from_preset( "mobilenet_v3_small_100_imagenet") m.export("/tmp/mobilenetv3.tflite", format="litert", input_signature=[torch.randn(1, 224, 224, 3)]) from_preset loads a KerasHub preset: a pretrained model, fetched by name. input_signature fixes the input to one 224 by 224 colour image, because the PyTorch export path needs one fixed shape (keras#22997). The keras.io guide picks the PyTorch backend because its files need no flex ops, the TensorFlow ops that the new LiteRT Android runtime cannot run. With the pinned versions it wrote a 10,255,480-byte file. Bug 1: the pixel numbers The exported file starts after preprocessing: calling a KerasHub model directly "does not apply preprocessing" (pipeline_model.py). So the app has to resize and normalize every photo itself, with the numbers the model was trained with. My app reads them from a sidecar, a small JSON file that sits next to the model file. The sample sidecar in my app's tests uses 127.5 for every mean and standard deviation. The preset's own preprocessor.json uses ImageNet's numbers, a different pair for each colour: !Bar chart. The model expects red from -2.12 to 2.25, green from -2.04 to 2.43 and blue from -1.80 to 2.64. The app's test sidecar sends every channel from -1 to 1. Nothing crashes, and on my quail photo the answer did not even change: quail stayed first, but its raw score fell from 10.11 to 9.72. A wrong number that leaves the top answer alone is easy to ship. The fix is to copy the preset's numbers instead of guessing them: mean is -offset / scale and std is 1 / scale, on pixels from 0 to 255. As a sidecar: { "task": "image_classification", "image_width": 224, "image_height": 224, "normalization": { "mean": [123.675, 116.28, 103.53], "std": [58.395, 57.12, 57.375] }, "labels": "imagenet_labels.txt", "num_classes": 1000 } Bug 2: the top-5 list These lines pick the five labels the screen shows (ImageClassificationTask.kt, lines 64 to 72, wrapped to fit): val top5 = PriorityQueue( compareByDescending<Classification> { it.score }) probs.forEachIndexed { idx, score -> if (score.isFinite()) { val label = labelList.getOrElse(idx) { "class_$idx" } top5.add(Classification(label, score)) if (top5.size > 5) top5.poll() } } TaskResult.Success( top5.sortedByDescending { it.score }, latency) Java's PriorityQueue keeps the smallest item, by its comparator, at the head, and compareByDescending makes that the highest score. poll() removes the head. So each time the queue holds six, it throws away the best score, and the screen ends up with the five least likely of 1,000 classes. For the quail photo that list is classes 50, 329, 389, 758 and 328, while the real top five for the same input are quail, prairie chicken, partridge, black grouse and ruffed grouse. The fix is one word: compareBy instead of compareByDescending, so the head is the lowest score and poll() drops that one. The test for this screen only checks that one to five labels come back, so it could not catch this. Both bugs are still in the repository at commit b8d3e2f (3 July 2026). The fixes are small, and I have left them in for now: if you want a first pull request, they are yours. To run the app yourself, the README has the three adb push lines and the install command. The rule I took from it Never retype preprocessing numbers. Either the export script writes the sidecar from the preset's own preprocessor.json, or the normalization goes inside the exported model, so there is nothing left to keep in sync. This app keeps the sidecar, because it loads any model that has one next to it. Which do you use, a sidecar written by code or normalization inside the model? If inside, how do you export it? Next: the check that would have caught both → --- This article was written with the help of AI, from my own repository and the public docs it links.

Two Android ML tests that could never fail, and a TFLite check that can --- title: "Two Android ML tests that could never fail, and a TFLite check that can" published: true description: "One test only counted the labels; the other compared against an answer file with no answers. Here is a check from the Keras notebook to the phone." tags: android, testing, machinelearning, tflite canonical_url: github.com/pctablet505/keras-ondevice-android/blo… cover_image: raw.githubusercontent.com series: Keras to LiteRT on Android --- Two tests in my Android app could never fail. One only counts the labels; the other compares the output with an answer file that holds no answers. And both of them look at this picture: !The test image from the repo: a green square, a yellow circle and a red rectangle on a blue ground. Nothing either model was trained to recognize, so no answer to it is wrong. The app runs a Keras model exported to LiteRT, the new name of TensorFlow Lite. This post is about one rule: a check proves something only if you have seen it fail. Here is how mine could not, and the check I would build instead. Two tests that could not fail The classification test asks only this of the result (ClassificationE2ETest.kt, line 66): assertTrue("Expected top-5 results", success.output.size in 1..5) Any five labels pass, including the five least likely ones the app shows (part 1 explains why it shows those). The detection test looks better. It reads a golden file, the saved answer a test compares against, and matches each expected box by its label and its overlap. Here is the golden file (detection_golden.json, its note cut to fit): { "_note": "Golden generation skipped: ...", "model": "ssd_mobilenet_v1_detect.tflite", "image": "images/test_image.jpg" } The note in full reads "Golden generation skipped: No module named 'ai_edge_litert'". No boxes. The match loop runs once per expected box, so with none it checks nothing, and the size check becomes "at least zero" (DetectionE2ETest.kt, lines 130 to 131). The script that should write the answers wrote this placeholder and carried on. The check that can fail !The check in six steps: one photo with a known answer, the Keras model's predict(), the .tflite file in Python with ai_edge_litert on the same array, the saved answer golden.json, the phone test matching labels and boxes at IoU 0.5 or more, and a test that fails when the answer file is empty. 1. Get the notebook's answer Start from a real photo with a known answer. KerasHub's own example uses a California quail, and "quail" is class 85 in ImageNet, the label set these classifiers use (imagenet_utils.py). Its example downloads the photo with keras.utils.get_file, and on 22 October 2026 Wikimedia answered 403 -- Forbidden. That photo is CC BY-SA 3.0, too, not public domain. So I downloaded a public-domain one from the U.S. Fish and Wildlife Service, cropped it square, and loaded it from disk. With my exported preset in the example's place: image = keras.utils.load_img("california_quail.jpg") classifier = keras_hub.models.ImageClassifier.from_preset( "mobilenet_v3_small_100_imagenet", activation="softmax", ) preds = classifier.predict(np.array([image])) top_labels = keras_hub.utils.decode_imagenet_predictions(preds) predict() runs the preset's own preprocessing, so this is the answer the model was built to give. On a computer's CPU it gave quail 10.30, partridge 3.94, prairie chicken 3.77, black grouse 3.57 and ruffed grouse 3.43. One surprise: those are raw scores. Despite activation="softmax", the 1,000 scores summed to 162.5, not 1. 2. Check the .tflite file in Python first Run the .tflite file on the array the Keras model sees after preprocessing (KerasHub's hint is model(model.preprocessor(x))), with the interpreter code from the keras.io guide. Then compare the two outputs the way KerasHub's own export tests do (test_case.py): abs_diff = np.abs(keras_val - litert_val) mean_abs_diff = np.mean(abs_diff) max_abs_diff = np.max(abs_diff) On the quail photo they agreed: max_abs_diff 1.1e-05 and mean_abs_diff 1.9e-06 over the 1,000 scores, with the same five labels in the same order and quail at 10.31 in both. If the file disagrees with the notebook here, the phone will too, and the bug is in the export, not the app. 3. Hold the phone to the saved answer Save the five labels and scores as the golden file, or for detection the boxes as label, score, x1, y1, x2, y2: the keys my detection test already reads. Push the same photo to the phone and keep the test's matching rule: the same label, and a box overlap of 0.5 or more, measured as IoU (the area two boxes share divided by the area they cover together). Two guards make it a test that can fail: - An empty or missing golden file fails. It never counts as a pass. - The same pixels go in. The preset crops the photo to a square before it resizes it, and my app stretches it. Use a square photo, or crop it first. Here it is failing I copied my app's preprocessing into Python (resize to 224 by 224, then the preset's numbers) and fed the same file the same square photo. Quail still came first, at 10.11 instead of 10.31, but prairie chicken and partridge swapped places. A check that compares the ordered five labels fails right there, with the right numbers in the sidecar: what is left is the resize step. That is the rule working, on a computer's CPU; I did not run the phone for this post. What I have not solved How close is close enough? My app's resize alone moved the top score by 0.2 and swapped two labels, before any quantization (weights stored in fewer bits to make the file smaller). KerasHub's thresholds are set by hand, per model and per output, and I do not have a rule for picking one. If you have one, I would like to read it. The two tests are still in the repo as they were at commit b8d3e2f (3 July 2026). The scores and differences above come from a run on 22 October 2026 with keras 3.15.1, keras-hub 0.32.0, litert-torch 0.10.0 and ai-edge-litert 2.3.0. Next: five ways the export itself breaks, each with its public record → --- This article was written with the help of AI, from my own repository and the public docs it links.

Why a Keras model fails to convert to TFLite: 11 known cases, 5 traps --- title: "Why a Keras model fails to convert to TFLite: 11 known cases, 5 traps" published: true description: "KerasHub expects 11 of its models to fail LiteRT (TFLite) export. All 11 in one table, the five traps they fall into, and what to write instead." tags: keras, tflite, pytorch, machinelearning canonical_url: github.com/pctablet505/keras-ondevice-android/blo… cover_image: raw.githubusercontent.com series: Keras to LiteRT on Android --- KerasHub, the Keras library of pretrained models, expects 11 of its own models to fail the export to LiteRT (the new name of TensorFlow Lite), and its tests say why in one line each. Here are all 11, the five traps they fall into, and what to write instead. I contribute to Keras and KerasHub. The change that switched these export tests back on for the PyTorch backend, keras-hub#2698, is mine. The 11, in KerasHub's own words An expected failure is a test marked to fail until an upstream fix lands; when the fix lands, the test turns loud instead of silently passing. Each name links to its line in the tests: | Model | Trap | The test's own reason | |---|---|---| | D-Fine | 2. One shape | "D-FINE's multi-scale features hit a torch.export shape guard." | | Flux | 2. One shape | "torch.export guard from Flux's dynamic num_heads reshape." | | F-Net | 3. No lowering | "litert-torch has no lowering for aten.complex (from ops.fft2)." | | GPT-OSS | 3. No lowering | "litert-torch NHWC rewriter has no lowering for aten.amax." | | Mixtral | 3. No lowering | "litert-torch cannot lower aten._assert_async from MoE routing." | | Qwen3-MoE | 3. No lowering | "litert-torch cannot lower aten._assert_async from MoE routing." | | Qwen-MoE | 3. No lowering | "litert-torch cannot lower aten._assert_async from MoE routing." | | SAM3 | 3. No lowering | "torchvision::nms is not registered in the torch.export op set." | | SegFormer | 3. No lowering | "Unsupported ops during SegFormer's torch export decomposition." | | StableDiffusion3 | 3. No lowering | "AdjustablePositionEmbedding cannot be lowered by torch.export." | | VAE | 3. No lowering | "VAE's pow ops fail TFLite legalization (tfl.pow)." | At the commit that merged it, KerasHub has 75 export tests in 67 model folders: 63 carry no mark, 11 are expected failures and 1 is skipped. I counted the marks in the code; I did not run the tests. The pull request names 7 models that now pass on PyTorch: VGG, Moonshine, SAM, GPTNeoX and BASNet, plus ViTDet and Whisper within a tolerance rather than an exact match. !The export stages from the keras.io guide's own log, with the five traps: in the model code (Python if, x.shape), at Torch export (one fixed shape), at lowering and the converter passes (ops with no lowering, tfl.pow), in the model.tflite file (inputs named args_0), and under every stage (versions). 1. "symbolic tf.Tensor used as a Python bool" Export turns your Python into a graph. A Python if on a tensor, or a size read with x.shape, can break that step or freeze one case into the graph. KerasHub still skips its DeepLabV3 export test with exactly this message (test). Fix: keep sizes and branches inside the graph. One merged change, before and after (keras-hub#2466): # Before: x.shape is read once, while the graph is built t = ops.arange(x.shape[position], dtype=x.dtype) # After: "for dynamic shape compatibility with TFLite" t = ops.arange(ops.shape(x)[position], dtype=x.dtype) For a Python if, keras-hub#2467 removed the branch instead: it builds the tensor so that one token gives an empty tensor, "avoiding the need for a Python conditional". 2. A torch.export shape guard On the PyTorch backend the input shape is fixed when you export. In my words from keras#22996, the issue I filed: > The only viable workaround is to export with a fixed static shape matching the runtime maximum. D-Fine and Flux are this trap in the table. Fix: give the export one shape, the largest you will feed it: # Fails (keras#22996, shortened): the sequence length is left # open, traced at 1, and the file breaks on longer input. lm = keras_hub.models.Gemma3CausalLM.from_preset( "gemma3_270m_it") lm.export("lm.tflite", format="litert") # Works: one fixed shape at export (an image model here). m.export("m.tflite", format="litert", input_signature=[torch.randn(1, 224, 224, 3)]) The working half ran on 22 October 2026, on a computer's CPU, for my demo app's MobileNetV3. To feed a different batch size later, resize the input at run time with resize_tensor_input(), as the keras.io guide shows. 3. "no lowering for aten.complex" The export stops at the name of one operation. Lowering is the step that turns each PyTorch operation into a LiteRT one, and litert-torch, the converter, has no rule for some of them. Seven of the 11 in the table name such an operation (aten.complex, aten.amax, aten._assert_async, torchvision::nms, tfl.pow); SegFormer and StableDiffusion3 give a vaguer reason. Check, then a fallback with a cost: find your model in the table before you start. The keras.io guide's fallback is the TensorFlow backend, but its flex ops, TensorFlow operations carried into the file, "are not supported by the new LiteRT Android runtime". The app then needs the legacy runtime, org.tensorflow:tensorflow-lite, plus tensorflow-lite-select-tf-ops. 4. Inputs named args_0 Your app asks for an input by name, and the file calls it args_0. litert-torch flattened dict inputs to generic names (litert-torch#1022, closed 4 June 2026). Fix: use litert-torch newer than 0.9.1, which keeps the original keys (keras#23053), and print the names before you write app code. For my MobileNetV3 file, with litert-torch 0.10.0, it printed this: print("Signatures:", interpreter.get_signature_list()) # {'serving_default': {'inputs': ['args_0'], # 'outputs': ['output_0']}} A single image input is still called args_0; the fix keeps the keys of dict inputs. 5. tf.lite is deprecated, and the toolchain moves KerasHub switched all its export tests off in February 2026, when TensorFlow 2.20 deprecated tf.lite (keras-hub#2573), and back on in July for PyTorch only, with Keras 3.15 or newer. Keras 3 with TensorFlow 2.20 and Python 3.13 crashed during export until keras#22816. It moved again on 22 October 2026. Installed from the main branches, the way the keras.io guide does it, the export above failed in the converter with 'tfl.batch_matmul' op found invalid output rank, expected 3 but got 4. The released versions of the day before exported it: keras 3.15.1, keras-hub 0.32.0, litert-torch 0.10.0 and ai-edge-litert 2.3.0. Check, not a fix: record the versions that worked, next to the exported file: pip freeze | grep -iE "keras|litert" > export-versions.txt The one I cannot close KerasHub's DeepLabV3 export test is still skipped, while my demo app exports a DeepLabV3 preset on the same PyTorch backend. One of the two is out of date; my 22 October run covered MobileNetV3 only. If you have run that export recently, which one is it? And which trap did you hit that is not on this list? New here? Start with part 1: one model on a phone → --- This article was written with the help of AI, from my own repository and the public records it links.

Keras #22561 speedups: compiled Keras on PyTorch now generates text 11.6x faster Between master on Jul 18 and master on Oct 15, 2026, 15 of our PRs landed in Keras. Same GPU, same torch and jax versions, same models; only the Keras source changes. On the issue's test LM, compiled Keras[torch] now generates text 11.6× faster, torch.compile went from slowing Keras down to making it 6.56× faster than eager Keras, and an LLM forward makes 12–23% fewer Python calls on all four backends. Here is compiled generation at every merge. Pick a config or a workload, and hover a point to see its runs: {% codepen codepen.io default-tab=result height=640 %} Generate-32 throughput, July master vs today Tokens per second from the 2.1M-parameter test LM, 128 tokens per call (4 sequences × 32 steps); longer bars are faster. In July, compiling a Keras[torch] model made it slower: 1,458 tok/s compiled against 2,139 eager. Today the compiled model runs at 16,975 tok/s: 6.56× faster than eager Keras, within 1.53× of plain compiled PyTorch, down from 17.8×. !Generate-32 throughput in tokens per second, July versus October master. Keras on PyTorch with torch.compile: 1,458 to 16,975 (11.6 times); eager: 2,139 to 2,588; plain PyTorch 26,003 compiled and 21,870 eager. Keras on JAX under jit: 55,734 and 52,200; eager 437 to 479; plain JAX 57,642 jit and 1,438 eager. Three merges that made torch.compile work with Keras Three merges each turned one compiled workload from slow to fast in a single step. !Three cards: #23327, merged Aug 30, compiled LLM forward 2.50 to 0.326 ms, 7.68 times faster; #23201, merged Sep 11, compiled CNN forward 0.945 to 0.156 ms, 6.05 times faster; #23647, merged Sep 30, compiled generate-32 57.2 to 7.82 ms, 7.32 times faster. The three PRs: #23327 (attention), #23201 (dict ordering in the tree code) and #23647 (identity checks in the tree code). Graph breaks per forward pass, at every merge: !Step chart of torch.compile graph breaks per forward pass at every merge: the LLM drops from 11 to 0 at #23327 and the CNN from 9 to 0 at #23201; both stay at 0 through October 15. Less Python in every Keras call, on all four backends Most of the series changed shared code (Layer.call, op tracebacks, dtype handling, input checks), so the saving reaches every backend, not just torch. Python function calls per forward pass, July to today: PyTorch −23% on the LM and −33% on the CNN; JAX −12% on the LM and −17% on the CNN; TensorFlow −13% on the LM and −19% on the CNN; NumPy −16% on the LM and −18% on the CNN. Adding #23336 takes NumPy to −36% on the LM. These counts are exact and repeat call for call on any machine. {% codepen codepen.io default-tab=result height=800 %} Keras[jax] Under jax.jit the Python runs once at trace time, so Keras[jax] was already close to plain JAX and stays there: 2.30 → 2.45 ms per generate-32 against 2.22 ms for plain JAX. Eager Keras[jax] is where the saving shows in the exact counts: −12% Python calls per forward. The timing agrees: the sweep gives 293 → 267 ms (1.10×) and the back-to-back A/B rounds give 1.08×. Every config, every workload {% codepen codepen.io default-tab=result height=720 %} {% details Show all 12 configs and workloads (ms per call) %} | Keras config | Workload | Plain framework (ms) | Jul 18 (ms) | Oct 15 (ms) | Change | + #23336 (ms) | A/B check | |---|---|---|---|---|---|---|---| | Keras[torch] · eager | CNN forward | 0.064 | 0.668 | 0.477 | 1.40× | 0.466 | 1.39× | | Keras[torch] · eager | LLM forward | 0.189 | 1.88 | 1.55 | 1.22× | 1.48 | 1.20× | | Keras[torch] · eager | LLM generate-32 | 5.85 | 59.8 | 49.5 | 1.21× | 47.2 | 1.20× | | Keras[torch] · torch.compile | CNN forward | 0.076 | 1.28 | 0.160 | 8.03× | 0.161 | 8.41× | | Keras[torch] · torch.compile | LLM forward | 0.160 | 2.69 | 0.302 | 8.93× | 0.299 | 9.60× | | Keras[torch] · torch.compile | LLM generate-32 | 4.92 | 87.8 | 7.54 | 11.6× | 7.66 | 12.4× | | Keras[jax] · eager | CNN forward | 0.508 | 1.34 | 1.23 | 1.08× | 1.18 | 1.15× | | Keras[jax] · eager | LLM forward | 2.79 | 8.46 | 8.38 | 1.01× | 7.90 | 1.12× | | Keras[jax] · eager | LLM generate-32 | 89.0 | 293 | 267 | 1.10× | 269 | 1.08× | | Keras[jax] · jax.jit | CNN forward | 0.077 | 0.079 | 0.081 | 0.97× | 0.080 | 1.02× | | Keras[jax] · jax.jit | LLM forward | 0.120 | 0.131 | 0.135 | 0.97× | 0.130 | 1.05× | | Keras[jax] · jax.jit | LLM generate-32 | 2.22 | 2.30 | 2.45 | 0.94× | 2.27 | 1.05× | A/B check: July ÷ today from 3 separate back-to-back rounds (higher is a bigger speedup). {% enddetails %} Each of our PRs, measured where it landed Each PR was timed on the commit just before and just after it merged, so every row is that one PR's own effect. Δ is just after the merge minus just before it; negative is better. {% details Show the per-PR table %} | PR | Merged | What it changed | Δ calls/fwd torch | JAX | TF | NumPy | Δ eager gen-32 (ms) | Δ compiled gen-32 (ms) | Graph breaks | |---|---|---|---|---|---|---|---|---|---| | #23183 | 07-28 | Traceback details are built only when an op fails, not on every call | −598 | −586 | −622 | −546 | −2.9 | −5.2 | · | | #23327 | 08-30 | Removed the torch.compile graph break in the torch backend's dot_product_attention | +6 | 0 | 0 | 0 | +0.40 | −12.3 | LLM 11 → 0 | | #23189 | 09-03 | Fixed an always-true device check that sent every tensor through .to() | 0 | +38 | 0 | 0 | +0.28 | −1.1 | · | | #23200 | 09-05 | Layer.call stops re-setting built/_called on every call | −1,848 | −1,098 | −1,188 | −1,100 | −4.9 | −10.5 | · | | #23182 | 09-10 | O(1) dtype membership and a fast path for canonical dtype strings | −300 | −82 | −497 | −116 | −0.89 | −1.5 | · | | #23199 | 09-10 | Dropped a redundant channels_last copy on torch conv inputs | 0 | −50 | 0 | 0 | +0.40 | −0.29 | · | | #23201 | 09-11 | Torch tree code made traceable: canonical dict key order | −142 | +8 | 0 | 0 | +0.55 | −5.2 | CNN 9 → 0 | | #23296 | 09-11 | Keeps call-context kwargs on the recorded Node (fix) | −109 | −78 | −14 | −46 | −0.24 | −0.09 | · | | #23187 | 09-12 | TorchTrainer skips a full flatten walk to find the batch size | 0 | −26 | 0 | 0 | +0.14 | −0.21 | · | | #23301 | 09-12 | Layer name scope opened once per call, not twice | −330 | −568 | −638 | −330 | −0.89 | −0.21 | · | | #23190 | 09-15 | torch slice() keeps dynamic dims under torch.export | 0 | −16 | 0 | 0 | −0.23 | −0.16 | · | | #23647 | 09-30 | Torch tree code stops comparing id()s | 0 | −8 | 0 | 0 | −0.58 | −49.4 | · | | #23645 | 10-07 | torch pooling ops run in channels_last | 0 | −16 | 0 | 0 | +0.10 | −0.05 | · | | #23184 | 10-13 | Fast path for single-InputSpec input checks | −102 | −59 | −63 | −63 | +0.06 | +0.05 | · | | #23920 | 10-15 | Functional models reject a mismatched last axis (fix) | 0 | −54 | 0 | 0 | +0.10 | +0.02 | · | | #23336 | not merged yet (approved) | Memoized dtype promotion in result_type | −713 | −378 | −1,471 | −3,399 | −2.2 | +0.12 | · | {% enddetails %} Hot functions in one Keras[torch] LLM forward {% details Show the hot-functions table %} | Function | Jul 18 | Oct 15 | Change | + #23336 | |---|---|---|---|---| | isinstance | 2,600 | 1,740 | −33% | 1,680 | | traceback error wrappers | 74 | 48 | −35% | 48 | | thread-local config lookups | 624 | 513 | −18% | 513 | | standardize_dtype | 177 | 244 | +38% | 184 | | tree.flatten | 93 | 67 | −28% | 67 | | tree.map_structure | 102 | 62 | −39% | 62 | | InputSpec checks | 14 | 14 | +0% | 14 | | CallSpec objects built | 23 | 23 | +0% | 23 | | inspect.Signature.bind | 22 | 22 | +0% | 22 | | convert_to_tensor | 86 | 86 | +0% | 86 | | get_device | 138 | 138 | +0% | 138 | {% enddetails %} How this was measured - Machine: one NVIDIA RTX PRO 6000 Blackwell, kept idle during the runs. - Software: the same for every run: torch 2.13.0, jax 0.11.2, Python 3.14.7. Only the Keras source changes. - Commits: 31 snapshots of master: Jul 18 (a2f8eac8), just before and just after each of the 15 merges, Oct 15 (cc9de2b31), and Oct 15 plus #23336. - Models: the issue's two test models: a small CNN (114k parameters) and a 2-layer transformer LM (2.1M parameters, 32-token greedy generate). - Timing: the issue's benchmark script, one process at a time, warm-up first, a GPU sync after every call. Each number is a median, and the whole sweep ran three times for PyTorch and twice for JAX. - Cross-check: a separate back-to-back run of July against October gives 12.4× compiled and 1.20× eager (the sweep: 11.6× and 1.21×). - Counts: Python calls from cProfile, graph breaks from torch._dynamo.explain. These are exact and repeat on any machine. - Not covered: training, CPU, larger models, and TensorFlow or NumPy timing (call counts only). The work is tracked in issue #22561. Thank you to the Keras maintainers who reviewed and merged these PRs.