runtime

Stop Pulling a NuGet Package for zstd. .NET 11 Has ZstandardStream TL;DR: .NET 11 puts Zstandard (zstd) right next to GZipStream and BrotliStream. new ZstandardStream(output, CompressionLevel.Optimal) is the whole setup, and HttpClient can decompress zstd responses with one flag. .NET 11 is at Release Candidate 2, which ships with a go-live support license, but it is not GA yet. Everything below was run on the .NET 11 RC 1 SDK. The package nobody wanted to own zstd is everywhere now. Log pipelines, databases, and HTTP all speak it. .NET, meanwhile, made you pick a third-party wrapper, check that its native binary ships for every platform you deploy to, and hope it is still maintained next year. It is the dependency that shows up in every security review and that nobody remembers adding. What .NET 11 adds The new types live in System.IO.Compression: - ZstandardStream - a Stream, used exactly like GZipStream. - ZstandardEncoder and ZstandardDecoder - span-based, no stream needed. - ZstandardDictionary - train a dictionary for lots of small, similar payloads. - ZstandardCompressionOptions - quality, window size, checksums, long-distance matching. On the HTTP side, DecompressionMethods.Zstandard and ZstandardCompressedContent are new too. The minimal version If you have ever written a GZipStream, you already know this API: using System.IO.Compression; byte[] raw = File.ReadAllBytes("app.log"); using var output = new MemoryStream(); using (var zstd = new ZstandardStream(output, CompressionLevel.Optimal)) { zstd.Write(raw); } byte[] compressed = output.ToArray(); using var input = new MemoryStream(compressed); using var unzstd = new ZstandardStream(input, CompressionMode.Decompress); using var back = new MemoryStream(); unzstd.CopyTo(back); Swap GZipStream for ZstandardStream and the rest of your code does not notice. I ran all three built-in compressors over the same payload: 2,000 lines of fake JSON logs, 188,010 bytes, all at CompressionLevel.Optimal. | Format | Compressed size | 200 runs on my machine | |---|---|---| | GZip | 13,499 bytes | 218 ms | | Brotli | 11,936 bytes | 299 ms | | Zstandard | 11,891 bytes | 150 ms | Smallest and fastest in one row. It is a synthetic, very repetitive payload and one run on one machine, so treat the timings as a hint, not a benchmark. The .NET team's own numbers on the Canterbury Corpus are in the Preview 1 release notes linked below. Want more squeeze? ZstandardCompressionOptions takes a Quality from -131072 to 22, and the default is 3. var options = new ZstandardCompressionOptions { Quality = 19, AppendChecksum = true }; using var zstd = new ZstandardStream(output, options); At quality 19 with a checksum, the same log payload came out at 7,535 bytes. The realistic one: thousands of tiny messages Compression is bad at tiny payloads. A 150-byte JSON event has almost no repetition inside it, so there is little to squeeze. The repetition is between messages: the same keys, the same enum values, the same shape, every time. That is what zstd dictionaries are for. You train one on sample messages, then both sides use it: using System.IO.Compression; // training: byte[][] of real messages, e.g. the last 1,000 events // message: one new event as byte[] byte[] samples = [.. training.SelectMany(b => b)]; int[] lengths = [.. training.Select(b => b.Length)]; using ZstandardDictionary dictionary = ZstandardDictionary.Train(samples, lengths, maxDictionarySize: 4096); byte[] buffer = new byte[1024]; ZstandardEncoder.TryCompress( message, buffer, out int written, dictionary, ZstandardCompressionOptions.DefaultWindowLog2); byte[] restored = new byte[message.Length]; ZstandardDecoder.TryDecompress( buffer.AsSpan(0, written), restored, out int read, dictionary); I trained a 4,096-byte dictionary on 1,000 order.created events, then compressed 200 new events it had never seen: | | Average size | |---|---| | Raw JSON | 152.2 bytes | | zstd, no dictionary | 134.7 bytes | | zstd with dictionary | 38.6 bytes | Without a dictionary, zstd barely dented them. With one, the average message went from 152.2 bytes to 38.6. That is the difference between "compression is not worth it for our queue" and "why did we not do this sooner". HttpClient speaks zstd now Automatic decompression is one flag: using var client = new HttpClient(new SocketsHttpHandler { AutomaticDecompression = DecompressionMethods.Zstandard | DecompressionMethods.GZip }); string body = await client.GetStringAsync("https://example.com/api/report"); With that handler, the client sent Accept-Encoding: gzip, zstd. My local test server answered 8,000 bytes of JSON as 33 bytes of zstd, and GetStringAsync handed back all 8,000 characters. Going the other way, ZstandardCompressedContent compresses a request body while it is sent and sets Content-Encoding: zstd for you: var json = new StringContent(payload, Encoding.UTF8, "application/json"); await client.PostAsync(url, new ZstandardCompressedContent(json)); 300 copies of a small JSON object reached my test server as 36 bytes. Make sure the server actually accepts zstd request bodies before you turn this on. Can I use it? - .NET 11 only. The types ship in the shared framework; my test apps had no package references at all. - Status: .NET 11 is at RC 2 with a go-live license; GA is still ahead. Gotchas 1. The int after the dictionary is not quality. In TryCompress(source, dest, out written, dictionary, 3), that 3 is windowLog2, and the call threw ArgumentOutOfRangeException: windowLog2 ('3') must be greater than or equal to '10'. Pass ZstandardCompressionOptions.DefaultWindowLog2 (23), or set quality with ZstandardDictionary.Create(buffer, quality). 2. Both sides need the same dictionary. Decoding a dictionary-compressed message without it did not throw; TryDecompress just returned false. Version your dictionaries and ship the id with the message. 3. HttpClient does not opt in by itself. A default HttpClient sent no Accept-Encoding at all. When my test server sent zstd anyway, the default client got the 33 raw zstd bytes, not the JSON. 4. Preview samples are already stale. The Preview 1 notes use WindowLog; the RC has WindowLog2, MinWindowLog2, and MaxWindowLog2. Trust the API docs over old blog snippets, this one included once GA lands. Wrap-up Zstandard is now just another Stream in the box, with dictionaries and HTTP support on top. Grab the .NET 11 RC SDK, swap one GZipStream for ZstandardStream, and see what your payloads say. Sources - What's new in .NET 11 libraries - Microsoft Learn - .NET 11 Preview 1 libraries release notes - dotnet/core - ZstandardStream class - .NET API reference - dotnet/runtime #119575 - Zstandard compression support - dotnet/runtime #123531 - Zstandard in DecompressionMethods - .NET 11 Release Candidate 2 - .NET Blog This article was written with the help of AI and checked against the sources above.

If your game does not launch or keep crashing when trying the new Waylith Engine 1.7 Unstable version, Hardware-enforced Stack Protection (CET) might be the cause. Here's what fixed it for me. NOTE: As per the developer's note below, the solution described here is unnecessary if you have installed the latest Windows cumulative updates. However, if you're stuck on an older version of Windows 10 and cannot upgrade for any reason, the solution below should work for you. Otherwise, simply install the latest Windows cumulative update, and there should be no need to apply this workaround. For anyone having trouble launching RimWorld 1.7 with the new Waylith Engine, I recently ran into an issue where the game simply refused to start. After quite a bit of troubleshooting, I managed to find the cause and fix it. The problem turned out to be Hardware-enforced Stack Protection (CET), a Windows security feature that was interfering with the game's startup. What is Hardware-enforced Stack Protection (CET)? Hardware-enforced Stack Protection is a Windows security feature that uses CPU hardware to protect applications against certain memory-based attacks. CET stands for Control-flow Enforcement Technology. One of its features uses something called a shadow stack, which keeps a protected copy of function return addresses. This helps prevent malicious software from hijacking a program's execution flow. It's an important security feature, but certain applications and runtimes can have compatibility issues with it, particularly on older Windows builds. In my case, RimWorld 1.7 uses the new Waylith Engine and its customized .NET 10 runtime, and the game was crashing during startup whenever this protection was enabled. How to fix it (only for RimWorld) Do NOT disable this protection for your entire system. Windows allows you to configure it for individual applications. 1. Open the Start menu, search for Windows Security, and open it. 2. Go to App & browser control. 3. Scroll down to Exploit protection and click Exploit protection settings. 4. Select the Program settings tab. 5. Click Add program to customize. 6. Select Choose exact file path. 7. Navigate to your RimWorld installation folder and select RimWorld.exe. For example: C:\Program Files (x86)\Steam\steamapps\common\RimWorld\RimWorld.exe Your installation path may be different. 8. A window will appear with several security settings. Scroll down until you find Hardware-enforced Stack Protection. 9. Check Override system settings. 10. Change the setting from On to Off. Your setting should look like this. 1. Leave the other options, including Enforce for all modules and Audit only, unchecked. 2. Click Apply, close the settings, and try launching RimWorld again. That's it! In my case, the game immediately started working after making this change. My system specifications (relevant to the issue) - Operating System: Windows 10 IoT Enterprise, 64-bit - Windows Version: 20H2 - OS Build: 19042 - CPU: AMD Ryzen 9 7950X3D - RimWorld Version: 1.7.4990.1069 (Waylith Engine, unstable branch) - Game Runtime: .NET 10.0.13-waylithclr.dev (WaylithCLR) - Installed .NET Runtime: microsoft.netcore.app 10.0.3 - Faulting Module: KERNELBASE.dll (10.0.19041.1466) - Exception Code: 0x80131506 These details might be particularly relevant if you're running an older Windows 10 build. I suspect there's a compatibility issue between the newer .NET runtime, CET, and certain older Windows components. Important security note This is a workaround, not necessarily a permanent fix. Disabling Hardware-enforced Stack Protection reduces exploit protection for the application you've selected. That's why I strongly recommend disabling it only for RimWorld.exe, rather than changing system-wide security settings. Microsoft has documented similar CET-related .NET runtime compatibility problems on older Windows builds. Updating Windows to a properly serviced version may resolve the underlying issue without requiring this exception. Relevant Microsoft .NET issue: runtime#108589 If you're experiencing the same problem, give this a try and let me know whether it works for you. Hopefully this saves someone else the hours of troubleshooting I went through!

Где Runtime Async в .NET 11 экономит память, а где нет Всем привет, меня зовут Сергей Прощаев, я Tech Lead и руководитель направления Java | Kotlin разработки в FinTech & E-commerce и преподаю на курсах разработки и архитектуры в ОТУС. В этой статье расскажу про Runtime Async в .NET 11: что меняется, когда приостановкой async-методов занимается рантайм, а не код от компилятора, и где выигрыш пропадает. На собеседованиях по C# годами звучит один и тот же вопрос: «во что компилятор превращает async-метод?». Правильный ответ знают все: в структуру с конечным автоматом, методом MoveNext и билдером задачи. В .NET 11 у этого ответа появляется оговорка «если вы не включили runtime-async». Я прогнал бенчмарки команды .NET на RC2 сам: аллокации совпали с опубликованными до байта, но только при одном условии, о котором в анонсах не пишут. Сразу оговорюсь, почему про C# пишет человек с JVM. Мой основной стек — Java и Kotlin, и именно поэтому за асинхронностью в .NET интересно следить со стороны: в 2023 году Java выпустила виртуальные потоки, а команда .NET свой эксперимент с зелёными потоками отложила и пошла улучшать async/await. В ноябре ожидается релиз .NET 11, где Runtime Async уже есть, но как preview-функция: для своего кода её включают явно, а поведение проверяют на своей нагрузке. С изменениями в .NET 11 разобраться проще, если понимать, как async/await работает в классической реализации. Эту тему подробнее разберём 29 октября в 20:00 на бесплатном открытом уроке «Магические слова async/await». Три платформы, три ответа на один вопрос Задача у всех одна: поток не должен простаивать, пока сервис ждёт ответа от базы или соседнего API. Решают её по-разному. Компиляторы Kotlin и классического C# преобразуют асинхронные функции в конечные автоматы. Kotlin превращает каждую suspend-функцию в автомат с полем label и скрытым параметром Continuation. Компилятор C# делает почти то же: генерирует структуру, хранит в её полях локальные переменные, которые живут через await, и раскладывает тело метода по состояниям внутри MoveNext. JVM и CLR исполняют уже сгенерированный код и дают инфраструктуру продолжений, но сам автомат не строят. // (Kotlin) suspend-функция: компилятор добавит скрытый параметр Continuation // и превратит тело в конечный автомат с полем label suspend fun price(id: Int): Int { val base = loadBase(id) // точка приостановки, пример схематичный return base + tax(base) }sourcecraft.dev Java пошла другим путём. В JDK 21, вышедшем 19 сентября 2023 года, появились виртуальные потоки (JEP 444): код остаётся синхронным и блокирующим, а приостановкой и возобновлением занимается JVM, снимая стек виртуального потока с несущего. // (Java 21, схематично) код блокирующий, приостанавливает и возобновляет его JVM try (var executor = Executors.newVirtualThreadPerTaskExecutor()) { executor.submit(() -> callBlockingApi(id)); }sourcecraft.dev Команда .NET попробовала тот же путь. За одиннадцать дней до выхода JDK 21, 8 сентября 2023 года, она опубликовала итоги эксперимента с зелёными потоками. Вывод был сдержанным: зелёные потоки возможны, но это вторая модель программирования рядом с async/await, и они плохо уживаются друг с другом. Самая наглядная цифра из отчёта: 100 миллионов минимальных вызовов P/Invoke на зелёном потоке заняли около 1800 мс вместо 300 мс. Э ксперимент поставили на паузу и решили улучшать существующую модель. Следующим шагом стал эксперимент runtime-async в .NET 9, а итогом — Runtime Async в .NET 11. На схеме ниже три подхода сведены рядом, чтобы было видно, где живёт конечный автомат в каждом из них. Главная мысль схемы: .NET 11 занял середину. Программная модель остаётся прежней, с Task, await и окраской методов, но приостановку и возобновление организуют VM и JIT, у которых больше информации. Виртуальные потоки позволяют во многих сценариях писать блокирующий код без async-сигнатур по всему стеку вызовов. .NET от окраски методов не отказывается и выигрывает за счёт того, что ничего не надо переписывать. Что именно меняется под капотом В классической схеме компилятор C# для каждого async-метода выпускает метод-обёртку, тип конечного автомата, поля для захваченного состояния и тело MoveNext. Упрощённо это выглядит так: // (C#, упрощённо) что компилятор делает из async-метода без runtime-async public Task<int> GetPriceAsync(int id) { var sm = new GetPriceAsyncStateMachine { Id = id, State = -1 }; sm.Builder = AsyncTaskMethodBuilder<int>.Create(); sm.Builder.Start(ref sm); return sm.Builder.Task; } struct GetPriceAsyncStateMachine : IAsyncStateMachine { public int State; public int Id; public AsyncTaskMethodBuilder<int> Builder; private TaskAwaiter<int> _awaiter; public void MoveNext() { // switch по State, получение awaiter, // AwaitUnsafeOnCompleted при незавершённой задаче, SetResult в конце } public void SetStateMachine(IAsyncStateMachine stateMachine) { } }sourcecraft.dev С runtime-async компилятор не генерирует пару IAsyncStateMachine и MoveNext. Он оставляет компактный IL, ставит методу флаг async (MethodImplOptions.Async, значение 0x2000 по черновику изменений ECMA-335), а точки приостановки кодирует вызовами методов AsyncHelpers: для Task и ValueTask это перегрузки Await, для прочих awaiter-ов AwaitAwaiter и UnsafeAwaitAwaiter. Для оптимизации важнее всего пара «вызов async-метода и сразу подходящий Await». Приостановку, сохранение нужного состояния и возобновление на другом потоке организуют VM и JIT. Так что меняется не столько место, где строится прежний автомат, сколько распределение ответственности, и поэтому JIT может оптимизировать границы между async-методами, а не только код внутри каждого. Включается фича свойством в файле проекта, выключается для проекта другим свойством. В .NET 11 это по-прежнему preview: <!-- (XML, csproj) включение Runtime Async для своего кода --> <PropertyGroup> <TargetFramework>net11.0</TargetFramework> <Features>$(Features);runtime-async=on</Features> </PropertyGroup> <!-- (XML, csproj) отключение для проекта --> <PropertyGroup> <UseRuntimeAsync>false</UseRuntimeAsync> </PropertyGroup>sourcecraft.dev Есть деталь, которую легко пропустить: сами библиотеки .NET 11 уже собраны с runtime-async=on и конечных автоматов от компилятора больше не содержат. Ваш код при этом может оставаться в классическом режиме, обе модели работают вместе. Получается, что, обновившись на .NET 11, вы уже исполняете runtime-async внутри BCL, даже если флаг не трогали. Что это даёт: пять наблюдений Я бы предпочёл не принимать «быстрее и чище» на веру, поэтому собрал репозиторий dotnet-runtime-async-demo и прогнал его на .NET 11 RC2 (SDK 11.0.100-rc.2) в GitHub Actions на ubuntu-24.04. Классическую схему для отдельного метода оставляет экспериментальный атрибут [RuntimeAsyncMethodGeneration(false)], как у Тауба, поэтому оба варианта работают на одном рантайме и отличаются только тем, как их собрал компилятор. Для сверки рядом цифры из документации .NET 11 и разбора производительности Стивена Тауба. Аллокации между прогонами стабильны, время скачет, поэтому опираюсь на байты. Наблюдение 1. Живые стек-трейсы стали короче в разы. Три вложенных async-метода, внутренний создаёт new StackTrace(). В классическом режиме у меня 13 кадров, из них 3 — AsyncMethodBuilderCore.Start. С runtime-async кадров 4, и это ровно цепочка вызовов. В документации 13 и 5: там Main тоже асинхронный, а мой пробник входит в цепочку из синхронного Main через GetAwaiter().GetResult(), так что точное число кадров зависит от точки входа: (text, вывод с runtime-async=on, results/stack_runtime.txt) at Runtime.InnerAsync() at Runtime.MiddleAsync() at Runtime.OuterAsync() at Program.<Main>$(String[] args)sourcecraft.dev Важная оговорка из той же документации: стек-трейсы исключений из catch выглядят одинаково в обоих режимах, их давно чистит ExceptionDispatchInfo. Выигрывают профилировщики, отладчик и всё, что смотрит на стек во время работы. С виртуальными потоками Java сходство здесь концептуальное: при реальной приостановке await физический стек в .NET по-прежнему разматывается, Runtime Async оптимизирует существующую модель, а не заменяет её стековыми потоками. Наблюдение 2. Когда Runtime Async устраняет промежуточные задачи. Самый интересный механизм — слияние вызовов. Если A ждёт B, а B ждёт C, в классической схеме у каждого метода свой сгенерированный автомат и свой результат-задача. Автомат при этом не обязательно попадает в кучу: на синхронном пути структура живёт на стеке, а платим мы в основном за задачи-результаты. JIT с runtime-async распознаёт пару «вызов метода, возвращающего Task, и сразу await» и передаёт результат дальше обычным значением. Задача материализуется только на внешней границе, где её действительно кто-то ждёт как объект. На схеме ниже эта разница показана на цепочке из трёх методов. Главная мысль схемы: контракт Task никуда не делся, он переехал туда, где задача действительно нужна. На рисунке благоприятный случай: вся цепочка собрана в новом режиме и вызовы идут подряд. В смешанной цепочке с классическими методами или когда задачу кладут в коллекцию и вешают продолжение руками, объект по-прежнему создаётся. Бенчмарк Тауба с двумя уровнями вложенности я повторил в BenchmarkDotNet 0.16.0-preview.2 с MemoryDiagnoser, в обычном режиме, когда каждый бенчмарк идёт в отдельном процессе: Сценарий Аллокации у меня Аллокации у Тауба Цепочка завершилась синхронно, классический async 144 B 144 B Цепочка завершилась синхронно, runtime-async 0 B 0 B Цепочка с Task.Yield, классический async 248 B 248 B Цепочка с Task.Yield, runtime-async 168 B 168 B Байты совпали до единицы. По времени у меня синхронная цепочка ускорилась с 26,95 до 10,72 нс, у Тауба с 21,221 до 6,151 нс: порядок тот же, машины разные. Наблюдение 3. Глубокие цепочки с исключением дешевеют на порядок. Рекурсивная цепочка бросает исключение на дне. На глубине 30 без приостановки классический вариант аллоцирует 84,2 KB, runtime-async — 5,71 KB, у меня и у Тауба одинаково. По времени у меня 56,6 мкс против 13,0 мкс. Пример синтетический: в реальном сервисе картину изменят локальная обработка исключений, логирование, повторы и настоящие точки приостановки. Наблюдение 4. Сборка становится вдвое меньше. Приложение из десяти вложенных async-методов в классическом режиме дало SizeProbe.dll размером 10 752 байта, а с runtime-async — 5632 байта, байт в байт как в разборе. Для одного сервиса это мелочь, в крупном приложении с множеством своих async-методов эффект может накапливаться. Размер итогового NativeAOT-образа и готовых сторонних сборок я не мерил, так что количественного вывода для них не делаю. Наблюдение 5. Продолжения перестали таскать пустой контекст. Это изменение не про флаг, по документации оно работает в .NET 11 для всех. Раньше каждое продолжение снимало и восстанавливало ExecutionContext со значениями AsyncLocal, даже когда восстанавливать было нечего. Теперь рантайм этот цикл пропускает. Пункт из документации, я его не мерил: в сервисе с трассировкой и областями логирования контекст часто непустой, и эффект там, по моей гипотезе, скромный. Где выигрыш пропадает Первое, и для меня главное: результат зависит от того, как запущен замер. Сначала я гонял те же бенчмарки с флагом --inProcess, когда BenchmarkDotNet исполняет их в своём процессе, и написал свой пробник на GC.GetTotalAllocatedBytes, который тоже работает в одном процессе. Оба показали другую картину: Сценарий, в одном процессе Классический async Runtime-async Цепочка завершилась синхронно 144 B 72 B Цепочка с Task.Yield 192 B 240 B Причину расхождения я не установил, поэтому не считаю эти цифры свойством самой реализации. Методики тоже не одинаковые: мой пробник считает аллокации всего процесса, включая пул потоков, а не одного вызова. Оба режима шли на одной модели процессора, AMD EPYC 9V74, так что разница в железе маловероятна. Для выводов об аллокациях беру стандартный запуск BenchmarkDotNet в отдельном процессе: эти значения совпали с опубликованными. Абсолютное время отличается между окружениями, и я считаю его результатом конкретного прогона, а не свойством .NET 11. Внутрипроцессные цифры оставляю отдельным наблюдением, которое ещё надо проверить. Практический вывод: сравнивайте замеры, запущенные одинаково, на одной версии SDK и рантайма, иначе вы меряете методику, а не механизм. Время на коротких цепочках вообще ненадёжно. У Тауба исключение на глубине 1 с приостановкой в runtime-async медленнее в 1,3 раза, а у меня быстрее: 7,709 мкс против 6,438 мкс. Байты в обоих прогонах те же: 1,68 KB против 1,4 KB. Второе: это микробенчмарки. Типовой запрос ждёт базу миллисекунды, а экономия измеряется наносекундами. Могу себе представить нагрузочный тест I/O-bound API, где с флагом латентность не изменится, и это будет честный результат. Заметнее эффект там, где много синхронно завершающихся await (кэши, парсеры, глубокие конвейеры), и в метриках GC, а не во времени ответа. Третье: фича в preview. Документация прямо просит присылать отчёты о пропускной способности и размере библиотек в обе стороны, положительные и отрицательные. До стабилизации производительность и детали поведения могут меняться от сборки к сборке. Переменные окружения DOTNET_RuntimeAsync и UNSUPPORTED_RuntimeAsync больше не поддерживаются, включение теперь через Features, отключение через UseRuntimeAsync=false. Четвёртое: всё, что опирается на внутренности старой схемы, стоит проверить. Кадров билдера в живом стеке у меня стало 0 вместо 3, и если в коде или в инструментах кто-то разбирает стек по именам вида <Method>d__0 или по кадрам MoveNext, их больше не найдут. Я бы проверил: самописные логгеры, которые склеивают стек в строку; правила группировки ошибок в системе мониторинга; диагностику, которая ищет конечный автомат через reflection. Пятое: поддержка не сплошная. Runtime-async работает с четырьмя типами возврата: Task, ValueTask, Task<T> и ValueTask<T>. По разбору Тауба, async void, async-итераторы и собственные task-like типы со своими билдерами пока остаются на классическом преобразовании. Для пограничных случаев, вроде by-ref локальных переменных и await в обработчиках исключений, сверяться стоит с поведением текущего компилятора, а не только с черновиком ECMA-335. Что проверить у себя Мой вариант порядка, если сервис уже на .NET 10 и переезд на .NET 11 запланирован: Обновить проект на .NET 11 без флага и снять базовые метрики: аллокации в секунду, паузы GC, p99 латентности. Часть выигрыша вы получите уже здесь, потому что BCL собрана в новом режиме. В отдельной ветке включить <Features>$(Features);runtime-async=on</Features> и прогнать те же замеры на той же версии SDK и рантайма, тем же способом. Так эффект обновления BCL отделяется от эффекта флага. Сравнивать аллокации, а не только время. Найти горячий путь с глубокой цепочкой await и написать на него бенчмарк BenchmarkDotNet с MemoryDiagnoser. Там будет видно, сливаются ли вызовы. Проверить всё, что читает стек: логирование, APM-агент, группировку ошибок. Если что-то ломается, лучше узнать до GA, а не после. Если встали ограничения, отключить фичу для проекта через <UseRuntimeAsync>false</UseRuntimeAsync> и оставить отчёт команде .NET. Сроки поджимают: по официальной политике поддержки .NET 8 и .NET 9 перестают поддерживаться 10 ноября 2026 года, а .NET 11 RC2 вышел 13 октября с go-live лицензией. Окно для экспериментов как раз сейчас. Вывод Самое интересное здесь, на мой взгляд, не наносекунды, а выбор архитектуры. Java с виртуальными потоками позволила во многих сценариях писать блокирующий код без async-сигнатур, переложив приостановку на JVM, хотя асинхронные API в экосистеме никуда не делись. Kotlin до сих пор строит конечный автомат в компиляторе, ровно как C# последние двенадцать лет. .NET в 2023 году отказался от зелёных потоков и пошёл третьим путём: модель программирования оставили прежней, а реализацию перенесли туда, где у неё больше информации. Для разработчика на C# это означает простую вещь: ответ «async — это синтаксический сахар над конечным автоматом» теперь неполный. Компилятор больше не выпускает привычный MoveNext, приостановку и возобновление организуют VM и JIT, и JIT может оптимизировать границы между вложенными async-методами, потому что видит цепочку вызовов. Для разработчика на JVM это хороший повод посмотреть на корутины Kotlin иначе: компиляторная схема не единственная, и платформа с похожей моделью только что показала, сколько она стоит. Ссылка на мой репозиторий с экспериментом Начать стоит с README.md: там все таблицы этой статьи, включая разницу между запуском в одном процессе и в отдельных. shared/Chains.cs — измеряемые цепочки. src/AsyncProbe/Program.cs — стек-трейсы и аллокации. bench/AsyncBench/Program.cs — бенчмарки. ./run.sh — собирает всё с нуля и сверяет результат с инвариантами из expected.txt. ИСТОЧНИКИРепозиторий с экспериментом What's new in .NET 11 runtime, Microsoft Learn, обновлено для RC2 Toub S., Performance Improvements in .NET 11, .NET Blog, 15.09.2026 Runtime-async, черновик изменений ECMA-335, dotnet/runtime Green Thread Experiment Results, dotnet/runtimelab #2398, 08.09.2023 .NET 9 Runtime Async Experiment, dotnet/runtime #94620 .NET and .NET Core Support Policy Announcing .NET 11 Release Candidate 2, .NET Blog, 13.10.2026 JDK 21, OpenJDK Kotlin Coroutines, KEEP Чтобы расти до уровня Senior C#-разработчика и браться за более сложные задачи, важно не только уверенно писать код, но и понимать, как работает .NET под капотом. На курсе «C#-разработчик. Продвинутый уровень» можно углубить эти знания, научиться находить узкие места в приложениях и принимать более обоснованные архитектурные решения. Пройдите вступительный тест, чтобы оценить свой текущий уровень, выявить пробелы и понять, какие навыки стоит подтянуть для следующего карьерного шага.

Microsoft corrige en urgence une faille critique menaçant les applications .NET Bonjour preview.redd.it Microsoft vient de publier des correctifs de sécurité exceptionnels pour .NET. Ils corrigent une vulnérabilité dans Brotli, un algorithme de compression très répandu, qui expose certaines applications à des attaques à distance. Les correctifs se succèdent depuis plusieurs jours autour d'une même faille de sécurité. Le 21 octobre dernier, les équipes de Node.js et les développeurs de Brotli ont corrigé CVE-2026-111497, une vulnérabilité critique dans le décodeur de cet algorithme de compression conçu par Google pour réduire la taille des données échangées sur le web. Surnommée Node2Shell ou Broccoli2Shell, elle touche aussi .NET, la plateforme de Microsoft sur laquelle reposent de nombreux logiciels et services web, ce qui a contraint Redmond à publier des patchs d'urgence quatre jours plus tard. En cause, une erreur de gestion de la mémoire dont les conséquences peuvent être particulièrement graves pour les applications vulnérables. Des données compressées piégées pour corrompre la mémoire Dans le détail, un attaquant peut transmettre à une application .NET des données compressées avec Brotli qu'il a volontairement malformées, par exemple dans une requête web. Si l'application accepte de les décompresser, le décodeur intégré à .NET peut poursuivre le traitement malgré l'erreur et écrire au-delà de la zone de mémoire qui lui est réservée. Cette écriture hors limites (out-of-bounds write) peut faire planter le programme, exposer des informations sensibles ou permettre à l'attaquant d'obtenir des privilèges supplémentaires sur le système. L'attaque peut être menée à distance, sans authentification ni intervention de la victime. Le 22 octobre, les chercheurs d'Imperva ont d'ailleurs réussi à exploiter cette même faille pour exécuter du code sur une version vulnérable de Node.js. En transmettant des données Brotli piégées à leur application de test, ils sont parvenus à lui faire créer un fichier sur le serveur, alors que son code ne prévoyait aucune action de ce type. En remplaçant cette instruction par d'autres, un pirate pourrait accéder aux données sensibles et aux secrets utilisés par l'application, ou lancer d'autres commandes avec les droits dont elle dispose. Un risque qui dépend de la configuration des applications .NET L'exposition dépend de la manière dont les applications .NET utilisent Brotli. Certains serveurs asp.net Core peuvent décompresser les requêtes qu'ils reçoivent, tandis que des logiciels utilisant HttpClient peuvent traiter automatiquement les réponses compressées. D'autres applications font directement appel aux fonctions Brotli de .NET. Dans les deux premiers cas, les développeurs doivent avoir activé ces fonctions, désactivées par défaut. La vulnérabilité concerne donc les applications qui décompressent des données qu'un attaquant peut contrôler, tandis que l'ancien environnement .NET Framework n'est pas affecté. Microsoft a publié les versions 8.0.33, 9.0.22 et 10.0.14 de .NET, ainsi que la troisième Release Candidate de .NET 11, pour corriger la faille. Les équipes concernées doivent installer ces mises à jour et redémarrer leurs applications. Lorsque celles-ci embarquent leur propre environnement .NET, elles doivent également les recompiler et les redéployer pour intégrer le correctif. En attendant, les administrateurs en incapacité d’appliquer immédiatement les patchs peuvent désactiver la décompression Brotli des données provenant de sources non fiables.

Brotli CVE-2026-111497: Memory Corruption, Information Disclosure, and RCE via Crafted Compressed Streams 1. Basic Information - Article Name: Wednesday, October 21, 2026 Security Releases - Source: Node.js project - Published Date: October 21, 2026 - Original Source: Node.js project - Related Information Sources: GitHub Advisory: Python Brotli / brotlicffi, Istio Brotli Requirements, Envoy Brotli Advisory, BleepingComputer, JPCERT/CC, Microsoft .NET Advisory, Imperva, OpenAI - Related Malware, Threat Groups, CVEs, Products: CVE-2026-111497, Google Brotli, Node.js, .NET, Python Brotli, Bun, Envoy, Istio - Severity: High 2. Executive Summary The Brotli stream decoder incorrectly treats invalid block switching as a request for additional input rather than a permanent error, writing data beyond the 8-byte internal buffer. Decompressing attacker-controlled Brotli data can cause process crashes or disclose process memory, and may allow remote code execution (RCE) when additional conditions are met. Public proofs of concept and research demonstrations exist, but the reviewed public sources do not report malicious exploitation in the wild. 3. Attack Flow The following are separate paths leading to the same decoder vulnerability. Sending requests to a server and retrieving responses as a client are independent attack vectors and do not form a single mandatory sequence of steps. Server Decompression of Compressed Requests 1. An attacker sends a crafted Brotli stream as an HTTP request or API input. 2. An application or runtime decompresses the data using the vulnerable decoder. 3. Writing to the internal buffer continues even after invalid block switching, causing an out-of-bounds write. 4. This leads to process crashes, memory information disclosure, and, when memory layout and protection mechanism conditions are met, remote code execution (RCE) under the process privileges. Client Decompression of Compressed Responses 1. An attacker induces a backend client, crawler, proxy, AI agent, or similar entity to retrieve an attacker-controlled URL. 2. The client receives a response marked with Content-Encoding: br and automatically decompresses it. 3. The same decoder flaw is triggered inside the client process, potentially leading to crashes, memory information disclosure, and code execution if additional conditions are met. The following table organizes the attack vectors and investigation perspectives. RCE, subsequent child processes, and outbound communications do not occur in all environments. | Attack Stage | Attacker Action / Technique | Tools / Communications | Defender Indicators (Prerequisites / Artifacts) | | --- | --- | --- | --- | | Vector A: Server-Side Request Decompression (Initial Transmission) | Sends a crafted Brotli compressed stream to the server as an HTTP request or API input. | HTTP/HTTPS communication, POST/PUT requests, Content-Encoding: br | Operation of runtimes equipped with vulnerable decoders, and automatic request body decompression settings (such as Express middleware configurations) | | Vector A: Server-Side Request Decompression (Internal Processing) | The server application decompresses a stream containing invalid block switching, triggering an out-of-bounds write. | Internal decoder calls in Node.js, .NET, Python, Envoy, Istio, etc. | Decoder processing errors, process termination (crashes), segmentation faults (segfaults), core dumps | | Vector B: Client-Side Response Decompression (Initial Inducement) | Induces crawlers, backend integration processes, AI agents, etc., to access an attacker-managed URL. | Induced requests to external URLs, external API fetches | Automatic response decompression must be enabled in the component fetching the URL (e.g., built-in fetch()) | | Vector B: Client-Side Response Decompression (Internal Processing) | The client process decompresses the received malicious Brotli response, triggering the same vulnerability. | HTTP responses, Content-Encoding: br, client internal decoders | Abnormal termination of the client process, connection drops, service stoppages, generation of abnormal child processes | | Post-Exploitation (Remote Code Execution & Impact Spread) | Triggers code execution under process privileges or memory information disclosure when memory layout and mitigation conditions are met. | Unauthorized shell commands (e.g., /bin/sh), outbound communication, memory dumps | Suspicious child process launches from web server/client processes, unknown outbound communications | Running an HTTP server on a vulnerable Node.js runtime does not, by itself, cause received bodies to be decompressed with Brotli. A configured request parser or another component must actually decompress the input. Imperva demonstrated RCE on Node.js 24.22.0 (Linux arm64) through both the built-in fetch() response-decompression path and the request-decompression path in a configured Express/body-parser application. The same input was rejected by patched version 24.22.1. These results do not establish successful exploitation on untested builds or architectures. This vulnerability is not specific to HTTP/2. Although nicknames such as "HTTP2Shell," "Node2Shell," and "Broccoli2Shell" exist, the fundamental attack condition is a vulnerable Brotli decoder decompressing data controlled by an attacker. 4. Attacker Position and Execution Location For the server-side vector, the prerequisite is the ability to send compressed requests from the internet or an internal network. For the client-side vector, the requirement is the ability to cause the target to retrieve and decompress an attacker-controlled response. Memory corruption itself occurs inside processes such as Node.js, .NET, Python, or reverse proxies that invoke the Brotli library. 5. Visibility for Victims and Administrators - Victims: May perceive the issue as service unresponsiveness, connection drops, or 5xx errors. - Administrators: May detect decoder process segmentation faults (segfaults), core dumps, unexpected process restarts, or suspicious child process creation and external communications immediately following HTTP request processing. 6. Success and Failure Conditions Success Conditions - The vulnerable Brotli decoder binary is incorporated into the running process. - In .NET environments, this applies to Brotli-enabled request decompression, HttpClient automatic response decompression, or explicit use of BrotliStream / BrotliDecoder. Default configurations of ASP.NET Core and HttpClient do not automatically decompress, and .NET Framework is not affected. Envoy and Istio are also affected when the Brotli decompressor is explicitly enabled. - The target process actually performs decompression processing on attacker-controlled Brotli data. - Whether RCE succeeds also depends on the target version, CPU architecture, memory layout, and OS mitigations such as ASLR. Failure Conditions and Mitigations - Update to patched versions provided by respective vendors, such as Google Brotli (1.0.10, 1.1.1, 1.2.1) or Node.js (22.23.4, 24.22.1, 26.12.1). - Until patching is complete, disable or exclude Brotli request and response decompression processing enabled in existing environments. Microsoft advises against enabling previously unused decompression features as part of a mitigation. - Reject Brotli before decompression, following OpenAI's guidance. Simply removing br from Accept-Encoding or stopping outbound compression on CDNs or web servers does not protect incoming request or client-side response decompression paths. Even if decompression is stopped in Envoy or Istio, compressed data may flow to backend components, so remaining vulnerable decoders must also be protected. - After updating, restart related processes and ensure container images are rebuilt and redeployed. Updating only the operating system's Brotli library may leave vulnerable runtime-bundled copies unchanged. 7. What Happens Upon Success Successful exploitation can crash the decoder process (DoS), disclose process memory, or, when additional conditions are met, execute code with the process's privileges. In .NET environments, this may lead to privilege escalation depending on the application configuration and execution context. 8. Observable Logs The following are investigation ideas (Inference). Available logs vary by product and configuration and do not represent confirmed IOCs observed in active exploitation environments. Normal Content-Encoding: br traffic or data lengths alone cannot be used to conclusively determine an attack. - Email: No email-initiated attack activities specific to this vulnerability have been reported. - Proxy / SWG / DNS: Content-Encoding: br traffic, abnormal responses, and communication histories to backend fetch destinations. - Endpoint / EDR: Segfaults in Node.js, dotnet, Python, or proxies, core dumps, abnormal child process launches, and unexpected allocations of executable memory regions. - Identity / IdP: The reviewed sources do not report authentication IOCs specific to this vulnerability. Investigate possible service account abuse following successful RCE separately. - SaaS / Cloud: Container restarts, Pod CrashLoopBackOff states, spikes in load balancer 5xx errors, and differences in deployed image digests. - Network: Brotli streams chronologically matching decoder errors, repetitive communications from the same source, and request-response flows immediately preceding crashes. 9. Attack Success Determination If the only evidence is the arrival of crafted Brotli data or traffic matching a PoC fingerprint, classify it as an attack attempt. The occurrence of decoder errors or process crashes indicates potential memory corruption but does not immediately signify successful RCE. Confirm initial execution only when there is execution evidence, such as suspicious child processes launched by the decoder process, process telemetry matching attacker commands, or equivalent evidence. Imperva's RCE results were obtained in a research environment using Node.js 24.22.0 (Linux arm64) and specific test configurations; they do not establish reliable reproduction across all environments. 10. Investigation Playbook 1. Trigger: Crashes or segfaults in processes that decompress Brotli data, core dumps, detected PoC fingerprints, or suspicious child processes. 2. Initial Verification: Verify incorporated decoder copies, runtime and package versions, request/response decompression settings, and process restart times. 3. Endpoint Investigation: Preserve and investigate process trees before and after crashes, core dumps, memory faults, launched shells, download tools, and file modification histories. 4. Authentication & Cloud: Check service account and Workload Identity usage history, secret access logs, and cloud API operation logs. 5. Post-Exploitation Tracking: Track traces of credential access, persistence configurations, lateral movement, and data exfiltration. 6. Containment: Temporarily halt suspicious Brotli decompression processing, update to vendor-patched versions, restart processes, and rebuild container images. 7. Judgment Categories: Evaluate distinctly across stages including reachability, decoder errors, crashes, memory information disclosure, RCE, and subsequent compromises. 11. Defense and Detection Ideas - Single Event: Alerts for segfault occurrences in Brotli decompression processes coupled with preceding br encoding communications. - Chronological Correlation: Perform correlation analysis from receiving crafted compressed streams -> decoder errors or process crashes -> process restarts -> generation of suspicious child processes and outbound communications. - Proactive Threat Hunting: Cross-search SBOMs, container image layers, binary dependencies, and runtime-bundled modules to comprehensively identify vulnerable Brotli copies. - Logging Considerations: Identifying causal relationships becomes difficult if TLS-terminated plaintext payloads, client-side response headers, and process crash telemetry are not preserved. - Priority Controls: Prioritize applying vendor-patched versions, restarting processes, rebuilding and redeploying containers, and disabling unnecessary automatic Brotli decompression features. 12. Facts / Inference / Hypothesis Facts - The Node.js project explained the out-of-bounds write mechanism against the 8-byte internal buffer in Brotli and released patched versions. - JPCERT/CC provided guidance on affected versions of Google Brotli and the scope of impact on widespread embedded products. - Imperva demonstrated RCE under specific conditions in a research environment running Node.js 24.22.0 (Linux arm64). - The reviewed public sources do not report malicious exploitation in the wild. Inference - Not only web servers, but also backend clients and crawlers that retrieve data from external URLs and decompress Brotli responses serve as critical attack surfaces. - In asset management, it is necessary to track not only direct package dependencies but also container images, vendor-embedded libraries, and statically linked binaries. Hypothesis No additional hypotheses. Unconfirmed items are listed in "Uncertainties and Additional Investigation." 13. MITRE ATT&CK Mapping | ID | Technique | Confidence | Basis | | --- | --- | --- | --- | | T1190 | Exploit Public-Facing Application | medium | Inference: Mapped to the assumed path where internet-facing applications decompress Brotli requests controlled by attackers. This does not indicate observed active exploitation, and this basis does not apply to paths where clients decompress malicious responses or internal services not exposed externally. | 14. Uncertainties and Additional Investigation - The presence of active exploitation by malicious actors, patch availability status across all embedded products, and detailed RCE success conditions per product and architecture remain unconfirmed. The existence of public PoCs cannot be treated as confirmation of production environment compromise. 15. Impact on SOCs and Organizations Even for organizations that do not directly develop or operate Node.js or .NET internally, API clients, service meshes, proxies, Python packages, and container images bundling the Brotli library may be affected. Create software inventories from both public web services and batch processing tasks retrieving external URLs, and verify patch application and process restarts for each embedded copy. 16. Summary by Target Audience - For SOCs: Monitor process crashes, suspicious child process launches, and outbound communications occurring immediately after Brotli decompression processing by correlating them chronologically with request and response traffic. - For Administrators: Promptly apply patched versions provided by product vendors, and ensure process restarts and container image rebuilds/redeployments are fully completed. - For Users: Follow update guidance and restart procedures provided by service administrators.

.NET has released an out of band security update to address the vulnerability in the Google Brotli library. This is the same library NPM and others use. .NET and ASP.NET Core do not use Brotli by default, you have to opt-in. Details on what to look for in your code are in the advisory.

A rewrite of Google's C brotli in C#: runtime#136430 — overall, I'd say perf is on par. Just AI + the BlazingFastDotnet skill 🙃. Zero unsafe code in the C# impl -> full memory safety (though it does use ArrayPool), vs a fully unsafe C impl with SIMDs/etc. t.co

We can / should obsolete it here. The SINGLE use case to use GetNonZeroBytes is when you are implementing PKCS#1 padding yourself. But you shouldn't be doing that either! runtime#42763

How ReadOnlySpan<char>.IndexOf finds a substring with SIMD Leia em português. This article follows the video's sequence of ideas: the video animates the states; here we can pause at each diagram and examine the code and assembly. The walkthrough uses the same teaching example, separate from the measured inputs. The opening result comes from a selected MediumRun, but still applies to a particular machine and scenario. The KMP and BMH comparison at the end is supplementary material, outside the main video. 1. The surprising result A search written as a loop might seem cheaper than a call to a higher-level API. To test that intuition, we compared two implementations that find the first occurrence of a substring and return its index, or -1 if none exists. Here is the result used in the opening: a 16-char substring inserted at the end of a pseudorandom text containing 65,536 char elements. | Implementation | Mean per search | Standard deviation | |---|---:|---:| | Naive loop | 134.935 µs | 24.518 µs | | ReadOnlySpan<char>.IndexOf | 5.653 µs | 389 ns | For this input and host, the loop's mean was approximately 23.9× the mean for IndexOf. That is not a prediction for every string, CPU, or runtime version. As we will see, the loop and runtime also use different algorithms; this benchmark does not isolate the cost of abstraction alone. 2. The two implementations The harness uses BenchmarkDotNet. The methods, inputs, configuration, and instructions for putting together a reproduction are included in this article. Each operation receives one ReadOnlySpan<char> for the text and another for the needle. The comparison is ordinal: it compares UTF-16 char code units without culture-sensitive rules. Both implementations return the first occurrence's index, or -1 if there is no match. A .NET char is a UTF-16 code unit, not necessarily a user-perceived grapheme. The baseline implements naive, or brute-force, search: try every starting position and compare the needle character by character. The second method calls the runtime API. These two methods drive the main explanation: private static int SearchNaive( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) { if (needle.IsEmpty) return 0; int lastStart = haystack.Length - needle.Length; for (int start = 0; start <= lastStart; start++) { int offset = 0; while (offset < needle.Length && haystack[start + offset] == needle[offset]) { offset++; } if (offset == needle.Length) return start; } return -1; } private static int SearchRuntime( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) => haystack.IndexOf(needle, StringComparison.Ordinal); BenchmarkDotNet measures them through these wrappers: [Benchmark(Baseline = true)] public int Naive() => SearchNaive(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int ReadOnlySpanIndexOf() => SearchRuntime(_haystack.AsSpan(), _needle.AsSpan()); The first wrapper measures the naive loop above. The second measures ReadOnlySpan<char>.IndexOf with ordinal comparison. To explain the observed path, we use the .NET 10.0.12 runtime implementation: [SpanHelpers.IndexOf(ref char, int, ref char, int) in SpanHelpers.Char.cs](github.com/dotnet/runtime/blob/v10.0.12/src/libra…). On this host, it uses a vectorized candidate filter and checks candidates against the complete substring. This is not our own search algorithm, nor a guarantee that another runtime version or CPU will choose the same path. Before timing, GlobalSetup runs all four methods for each input and requires the expected result. For example, a scenario labeled "missing" cannot enter the benchmark if a search finds an accidental occurrence. Validation happens before measurement: int naive = SearchNaive(_haystack.AsSpan(), _needle.AsSpan()); int expected = Scenario switch { SearchCase.RandomEarlyMatch => 0, SearchCase.RandomMiddleMatch => HaystackLength / 2, SearchCase.RandomLateMatch or SearchCase.RepeatedPrefixLate => HaystackLength - _needle.Length, SearchCase.RandomMissing or SearchCase.RepeatedPrefixMissing or SearchCase.RepeatedAlternatingLateFalse => -1, _ => throw new ArgumentOutOfRangeException(nameof(Scenario)) }; VerifyResult(nameof(Naive), naive, expected); VerifyResult(nameof(KnuthMorrisPratt), SearchKnuthMorrisPratt(_haystack.AsSpan(), _needle.AsSpan()), expected); VerifyResult(nameof(BoyerMooreHorspool), SearchBoyerMooreHorspool(_haystack.AsSpan(), _needle.AsSpan()), expected); VerifyResult(nameof(ReadOnlySpanIndexOf), SearchRuntime(_haystack.AsSpan(), _needle.AsSpan()), expected); The benchmark compares a scalar, character-by-character search with the runtime's optimized implementation. It does not isolate API-call overhead. The question is what work the algorithms perform, not whether a particular syntax or abstraction is inherently faster. KMP and Boyer–Moore–Horspool appear at the end as a complementary comparison, not a replacement for the main explanation. 3. Inputs: what did we test? The harness combines seven scenarios with texts of 4,096 and 65,536 char elements. Inputs are built once during setup, outside the timed region. | Scenario | Text and needle | Expected result | |---|---|---| | RandomEarlyMatch | Pseudorandom text over a–h; needle abcdefghabcdefgh inserted at index 0. | index 0 | | RandomMiddleMatch | Same generator and needle, inserted halfway through the text. | index HaystackLength / 2 | | RandomLateMatch | Same generator and needle, inserted into the final 16 elements. | index HaystackLength - 16 | | RandomMissing | Same generator and alphabet, without inserting the needle. | -1, checked in setup | | RepeatedPrefixMissing | Text consisting only of a; needle aaaaaaab. | -1 | | RepeatedPrefixLate | Text consisting only of a, with aaaaaaab replacing the final eight elements. | index HaystackLength - 8 | | RepeatedAlternatingLateFalse | Alternating ab text; 64-element needle alternating for the first 63 elements and ending in a. | -1; false candidates differ at the final element | The pseudorandom generator is deterministic. It starts at 0xC0FFEE and updates a 32-bit state for each position using state = state * 1664525 + 1013904223. The top three bits select one of the eight characters a–h: private static string CreateOrdinaryHaystack(int length) { const string alphabet = "abcdefgh"; var chars = new char[length]; uint state = 0xC0FFEE; for (int i = 0; i < chars.Length; i++) { state = (state * 1664525) + 1013904223; chars[i] = alphabet[(int)(state >> 29)]; } return new string(chars); } In matching scenarios, the corresponding text segment is replaced with the needle. For the late match used in the opening, this is equivalent to: const string needle = "abcdefghabcdefgh"; string haystack = CreateOrdinaryHaystack(length); int matchIndex = length - needle.Length; // RandomLateMatch haystack = haystack[..matchIndex] + needle + haystack[(matchIndex + needle.Length)..]; The missing scenario keeps the same alphabet, avoiding a trivial miss caused by impossible characters. Setup verifies that the complete needle is absent. The adversarial benchmark scenario This measured case differs from both the opening and the teaching example in sections 6–7. The text alternates a and b. The needle also alternates, but its final char is changed to a: private static string CreateAlternatingNeedleWithLateMismatch() { const int length = 64; return string.Create(length, 0, static (chars, _) => { for (int i = 0; i < chars.Length; i++) chars[i] = (i & 1) == 0 ? 'a' : 'b'; chars[^1] = 'a'; }); } The runtime uses the first char, a, and searches backward for the last needle element that differs from it. Positions 63 and 62 contain a, so the selected position is 61, containing b. Both filters pass at every even start: haystack[i] == 'a' and haystack[i + 61] == 'b'. The complete substring, however, differs at its final element: the text has b at relative position 63, while the needle has a. | Text length | Expected false candidates | Result | |---:|---:|---:| | 4,096 | 2,017 | -1 | | 65,536 | 32,737 | -1 | Setup checks a conceptual one-bit-per-lane summary, 0x5555, the expected one-bit-per-byte extracted mask, 0x33333333, and the total candidate count. The first summary is neither the comparison vector nor an intermediate runtime variable: each approved lane contains FFFF. This is a scalar validation of the fixture, not a reading of runtime registers. 4. Measurement and results We used BenchmarkDotNet 0.15.8, Release builds, .NET 10.0.12 (SDK 10.0.401), Ubuntu 24.04.4 Linux x64, and an AMD EPYC 7763 at 2.45 GHz. BenchmarkDotNet reported one CPU available to the process. The host exposed AVX2 and BMI1; the assembly capture discussed here comes from this environment. Two lengths, seven scenarios, and four methods yield 56 method/case combinations in the broad ShortRun matrix. These attributes configure it: [ShortRunJob] [MemoryDiagnoser] [DisassemblyDiagnoser(maxDepth: 3)] public class SubstringSearchBenchmarks { [Params(4096, 65536)] public int HaystackLength { get; set; } [Params( SearchCase.RandomEarlyMatch, SearchCase.RandomMiddleMatch, SearchCase.RandomLateMatch, SearchCase.RandomMissing, SearchCase.RepeatedPrefixMissing, SearchCase.RepeatedPrefixLate, SearchCase.RepeatedAlternatingLateFalse)] public SearchCase Scenario { get; set; } } The broad matrix used ShortRun: three warmup iterations and three measurement iterations. For the late-match and adversarial scenarios, we also ran MediumRun: ten warmups, fifteen measurement iterations, and two processes per benchmark. This table reports MediumRun means and standard deviations; units have been converted for readability: | Input length | Scenario | Naive: mean ± standard deviation | IndexOf: mean ± standard deviation | Ratio of means | |---:|---|---:|---:|---:| | 4,096 | Late match, pseudorandom text | 3.443 ± 0.238 µs | 323 ± 15 ns | 10.7× | | 65,536 | Late match, pseudorandom text | 134.935 ± 24.518 µs | 5.653 µs ± 389 ns | 23.9× | | 4,096 | Many candidates, final-element mismatch | 149.369 ± 57.317 µs | 7.645 µs ± 390 ns | 19.5× | | 65,536 | Many candidates, final-element mismatch | 1.925 ± 0.202 ms | 129.186 ± 7.864 µs | 14.9× | The adversarial scenario has substantial dispersion, particularly in the naive loop. We therefore report dispersion alongside means rather than treat it as a typical case. Nor do we attribute the entire gain to SIMD: the C# baseline examines positions with a different algorithm from IndexOf, which also uses vector instructions. The supported conclusion is narrower: on this host, for these inputs, the runtime implementation was faster. In the original run, validation compared IndexOf, KMP, and BMH against the naive baseline across 34,193 deterministic cases: all binary strings up to eight char elements and needles up to five, plus 2,000 generated inputs containing diverse characters, including NUL and surrogate code units. Coverage included empty needles, needles longer than the text, injected matches, and missing substrings. In the original capture, adding --job Medium to a class annotated with [ShortRunJob] ran both jobs. The reported values come from Job=MediumRun rows. The reproduction at the end uses only [MediumRunJob] to run the two selected scenarios without repeating the ShortRun matrix. The project pins RuntimeFrameworkVersion to 10.0.12 and disables roll-forward. Its configuration and commands are included below, without depending on files outside this article. 5. The idea behind SIMD SIMD applies the same operation to multiple data elements. In the observed path, those elements are 16-bit UTF-16 code units. A 256-bit vector holds 16 char lanes: lane: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 char: [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] That does not mean the entire search takes one instruction. The vector selects several possible starts for verification. The path still iterates over blocks, processes masks, and checks the complete substring. 6. Finding candidates A small input for following the calculations We use the same teaching fixture as the video: a 64-code-unit needle inserted at index 6 of a 79-code-unit text, filled with x outside that segment. The needle alternates a and b, except for its final element, a. This reproduces the input; it is not a timed benchmark scenario: char[] value = new char[64]; for (int i = 0; i < value.Length; i++) value[i] = (i & 1) == 0 ? 'a' : 'b'; value[63] = 'a'; char[] text = new char[79]; Array.Fill(text, 'x'); value.AsSpan().CopyTo(text.AsSpan(6)); int result = text.AsSpan().IndexOf(value.AsSpan(), StringComparison.Ordinal); // result == 6 There are 79 - 64 + 1 = 16 valid starts, from 0 through 15, fitting one 16-lane window. The goal is to explain the path to a match, not depict the data responsible for the benchmark timings. Choosing two needle positions The first position is value[0] == 'a'. For the second, the runtime searches backward for an element different from the first: needle position: 0 1 ... 61 62 63 value: a b ... b a a ^ ^ ch1 ch2 Positions 63 and 62 contain a; position 61 contains b. Therefore, ch1ch2Distance = 61. ch2 is not necessarily the final character: it is the second position chosen for filtering. These values are broadcast into vectors: ch1 contains 16 copies of a; ch2, 16 copies of b. For start i, we ask whether text[i] == 'a' and text[i + 61] == 'b'. If either fails, that start cannot be a complete match. Offset, lane, and address are different concepts offset is the start of the candidate window in the text. Lane j represents start offset + j. Here, offset = 0: | Lane | Candidate start | First position | Second position | |---:|---:|---|---| | 6 | 0 + 6 = 6 | text[6] == 'a' | text[0 + 6 + 61], or text[67] == 'b' | | 8 | 0 + 8 = 8 | text[8] == 'a' | text[0 + 8 + 61], or text[69] == 'b' | The runtime excerpt compares the second position first, then the first, then combines them with AND. Variable types are omitted and line breaks adjusted: cmpCh2 = Vector256.Equals( ch2, Vector256.LoadUnsafe( ref searchSpace, (nuint)(offset + ch1ch2Distance))); cmpCh1 = Vector256.Equals( ch1, Vector256.LoadUnsafe(ref searchSpace, (nuint)offset)); cmpAnd = (cmpCh1 & cmpCh2).AsByte(); In this branch, searchSpace references ushort elements. Load offsets are in 16-bit UTF-16 units, not bytes. The loads read text[61..77] and text[0..16], with exclusive upper bounds. Adding the distance to the displaced load keeps each candidate's two positions in the same lane: lane 6 relates text[6] to text[67], not elements belonging to different starts. Comparing and combining all 16 lanes Each equality produces FFFF when true and 0000 when false. In this table, 1 and 0 abbreviate those 16-bit values; they are not the integer values stored in the vector. The operation remains == in every lane, even when its result is false. lane: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 text[i+61]: b a b a b a b a a x x x x x x x ch2: b b b b b b b b b b b b b b b b cmpCh2: 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 text[i]: x x x x x x a b a b a b a b a b ch1: a a a a a a a a a a a a a a a a cmpCh1: 0 0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 cmpCh1: 0 0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 cmpCh2: 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 AND: 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 Lane 6 passes both equalities: a == a and b == b. Lane 8 passes the first, text[8] == 'a', but fails the second: text[69] contains a, so a == b is false. AND retains only lane 6. SIMD comparisons process lanes in parallel. Reading columns 6 and 8 merely explains their results. AsByte() reinterprets the same bits as bytes; it does not convert FFFF to the number 1. cmpAnd != Vector256<byte>.Zero produces a scalar decision: a candidate exists. It neither identifies the index nor confirms the substring. In the captured benchmark assembly, vpcmpeqw, vpand, and vptest correspond to the comparisons, combination, and test. This evidence belongs to the documented host; it is not a guarantee of the same translation on every machine. 7. Locating and confirming the candidate From one 16-bit lane to two mask bits The runtime extracts the most significant bit of each byte: uint mask = cmpAnd.ExtractMostSignificantBits(); There are 16 elements of 16 bits, reinterpreted as 32 bytes. In lane 6, the true result contains two FF bytes; every other element contains two 00 bytes: lane 6: FFFF bytes in vector: byte 12 = FF byte 13 = FF bits in each byte: 11111111 11111111 most significant bit: 1 1 mask position: 12 13 Lane 0 corresponds to bytes and bits 0–1; lane 1 to 2–3; lane 6 to 12–13. The actual extracted mask is therefore 0x00003000: index: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 bit: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 index: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 bit: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Indices increase from left to right, as in the video. This is not the usual numeric binary notation: the least significant bit is on the left. Each bit represents a byte of the result vector, not a character directly. In the observed assembly, extraction corresponds to vpmovmskb. Counting, converting, and adding the offset The next calculation is: nint charPos = (nint)(uint.TrailingZeroCount(mask) / sizeof(ushort)); Counting starts at bit 0 and finds 12 zeros before the first 1. Thus, TrailingZeroCount(0x00003000) = 12. Dividing by sizeof(ushort), which is 2, converts the byte position represented by that bit into an element position: first set bit: 12 charPos: 12 / 2 = 6 index in text: offset + charPos = 0 + 6 = 6 charPos is relative to the window; the text index includes offset. They coincide here because offset is zero. In the captured assembly, the count corresponds to tzcnt, and division by two to a bit shift. Checking the entire sequence Only now do we check complete equality. Here is the runtime excerpt: if (valueLength == 2 || SequenceEqual( ref Unsafe.As<char, byte>(ref Unsafe.Add(ref searchSpace, offset + charPos)), ref Unsafe.As<char, byte>(ref value), (nuint)(uint)valueLength * 2)) { return (int)(offset + charPos); } valueLength == 2 is false because our needle contains 64 elements. SequenceEqual compares its 128 bytes with those of the segment starting at text[6]. These ranges have exclusive upper bounds: | Needle segment | Corresponding text segment | Equal contents | |---|---|---| | value[0..16] | text[6..22] | abababababababab | | value[16..32] | text[22..38] | abababababababab | | value[32..48] | text[38..54] | abababababababab | | value[48..64] | text[54..70] | abababababababaa | All four pairs are equal. The call returns true, and the method returns offset + charPos = 6. The table organizes data, not instruction execution order. Confirmation also uses SIMD in the captured assembly for the benchmark's needle of this size. We do not need to walk through its implementation to understand its role in the search. If confirmation failed, the runtime would clear the candidate's two bits and continue with the next marked pair, if any remained. That does not happen in this fixture, which finds a match. We can check the arithmetic without reading internal registers. Add this to the C# example that constructs text and value in section 6: uint expectedMask = 0; for (int lane = 0; lane < 16; lane++) { if (text[lane] == value[0] && text[lane + 61] == value[61]) expectedMask |= 3u << (2 * lane); } if (expectedMask != 0x00003000 || uint.TrailingZeroCount(expectedMask) / sizeof(ushort) != 6 || !text.AsSpan(6, 64).SequenceEqual(value.AsSpan()) || result != 6) { throw new InvalidOperationException("Inconsistent teaching fixture."); } expectedMask is a scalar verification calculation, not an extraction from the vector used internally by IndexOf. 8. Verify, do not assume The teaching example has an occurrence at index 6. The adversarial scenario measured in section 3 keeps the same needle but uses an entirely alternating text, without inserting an occurrence. Every even start passes both filter positions, but the complete comparison fails at the final element: the text has b, while the needle ends in a. | Input | Candidates in the first window | Extracted one-bit-per-byte mask | Search result | |---|---|---|---:| | Teaching example: 79 elements, inserted occurrence | Only lane 6 | 0x00003000 | 6 | | Adversarial: alternating ab text | Lanes 0, 2, 4, 6, 8, 10, 12, 14 | 0x33333333 | -1 | For the adversarial input, we could summarize approved lanes as 0x5555, using one bit per lane. That is only a conceptual representation. Vector results are FFFF/0000, and the extracted mask variable contains two set bits per candidate. After rejecting start 0, clearing bits 0–1 changes 0x33333333 to 0x33333330. The next trailing-zero count is 4; 4 / 2 = 2 points to the next candidate. In the observed assembly, two blsr instructions clear the two bits. For a text containing 65,536 elements, there are ((65,536 - 64) / 2) + 1 = 32,737 even candidate starts. All are false and fail at the final element. This counts distinct positions passing the filter, not measured internal SequenceEqual calls: the final window can overlap positions examined earlier. During setup, the harness recomputes the expected first-block mask and total candidate count with scalar code, and checks that all four methods return -1. That validates the input and arithmetic; it does not read internal runtime registers. The connection to instructions comes from the versioned source and captured assembly. The first expected steps are: | Mask before | Trailing zeros | Position / 2 | Confirmation | |---|---:|---:|---| | 0x33333333 | 0 | 0 | fails at relative element 63 | | 0x33333330 | 4 | 2 | fails at relative element 63 | | 0x33333300 | 8 | 4 | fails at relative element 63 | | 0x33333000 | 12 | 6 | fails at relative element 63 | Finally, the approximately 24× opening result belongs to a third input: a pseudorandom late match, with a 16-element needle and 65,536-element text. It is neither the adversarial result nor a measurement of the teaching fixture. The example explains the algorithm; each measurement belongs to its own input. 9. Advancing, finishing, and returning to the result After checking a block, the algorithm advances by a vector width. If the next advance would pass the last full candidate window, the loop clamps its offset to that final window, overlapping the previous block to cover the remaining starts without reading beyond the input. Inputs with fewer than eight possible starts cannot use this host's 128-bit vector path and take the scalar path. The exact bounds and branches come from the runtime source, not from the API's name alone. In this capture, the filter uses AVX2: word comparisons, AND, vector testing, and mask extraction. tzcnt and blsr belong to BMI1, and confirmation also uses vectorized byte comparisons. This observation applies to the documented runtime, JIT, and CPU; another machine may take a different path. The opening number shows that, in this experiment, the runtime finds candidates in parallel and outperforms the scalar, character-by-character search. It does not prove that less abstraction always costs more, or that SIMD always wins. Interpreting a benchmark requires knowing which data and algorithms were measured. Mapping the observed instructions This table summarizes the capture on Linux x64 / EPYC 7763, .NET 10.0.12, AVX2, and BMI1. It is neither a promise of one-to-one translation on other CPUs or versions nor a new capture of the teaching fixture: | Operation | Observed instructions | Role | |---|---|---| | Broadcast ch1 and ch2 | vmovd, vpbroadcastw | Repeat the value in 16 lanes of 16 bits. | | Vector256.Equals | vpcmpeqw | Produce FFFF or 0000 per lane. | | AND the results | vpand | Retain positions passing both comparisons. | | Test the vector against zero | vptest and a branch | Skip mask extraction when there is no candidate. | | ExtractMostSignificantBits | vpmovmskb | Extract 32 bits, one per byte. | | TrailingZeroCount | tzcnt | Locate the first set bit. | | Divide by 2 | shr ..., 1 | Convert a byte position to a UTF-16 element position. | | Clear a rejected candidate's two bits | blsr, twice | Continue to another candidate. | For the adversarial case's 128-byte confirmation, the observed assembly compares AVX2 chunks of 32 bytes. The first three chunks agree; the fourth contains the final-element mismatch. This describes that capture, not a requirement that SequenceEqual always execute four steps. As a side note: what about KMP and Boyer–Moore? After the main conclusion, a natural question is how the comparison changes with algorithms that avoid restarting from scratch. We added two implementations to the same harness: - Knuth–Morris–Pratt (KMP): builds a needle prefix table and uses it to resume from the longest prefix that could still match. Search and preprocessing together cost O(n + m), where n is the text length and m the needle length. - Boyer–Moore–Horspool (BMH): compares right to left and, after a mismatch, advances based on the window's final character and its position in the needle. Preprocessing costs O(m); search often skips positions, but its worst case remains O(nm). For this one search per needle comparison, timed work includes building each algorithm's tables on every call. KMP allocates and fills its prefix table; BMH builds a sparse shift table, Dictionary<char, int>, before scanning. The runtime receives the same needle and text without reused preprocessed state. All four methods return the first index or -1; GlobalSetup checks the expected result before measurement. The additional benchmark code is: [Benchmark] public int KnuthMorrisPratt() => SearchKnuthMorrisPratt(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int BoyerMooreHorspool() => SearchBoyerMooreHorspool(_haystack.AsSpan(), _needle.AsSpan()); private static int SearchKnuthMorrisPratt( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) { if (needle.IsEmpty) return 0; if (haystack.Length < needle.Length) return -1; var prefixLengths = new int[needle.Length]; for (int i = 1, prefixLength = 0; i < needle.Length; i++) { while (prefixLength > 0 && needle[i] != needle[prefixLength]) prefixLength = prefixLengths[prefixLength - 1]; if (needle[i] == needle[prefixLength]) prefixLength++; prefixLengths[i] = prefixLength; } for (int i = 0, prefixLength = 0; i < haystack.Length; i++) { while (prefixLength > 0 && haystack[i] != needle[prefixLength]) prefixLength = prefixLengths[prefixLength - 1]; if (haystack[i] == needle[prefixLength]) prefixLength++; if (prefixLength == needle.Length) return i - needle.Length + 1; } return -1; } private static int SearchBoyerMooreHorspool( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) { if (needle.IsEmpty) return 0; if (haystack.Length < needle.Length) return -1; var shifts = new Dictionary<char, int>(needle.Length); for (int i = 0; i < needle.Length - 1; i++) shifts[needle[i]] = needle.Length - 1 - i; int lastStart = haystack.Length - needle.Length; int start = 0; while (start <= lastStart) { int offset = needle.Length - 1; while (offset >= 0 && haystack[start + offset] == needle[offset]) offset--; if (offset < 0) return start; int shift = shifts.TryGetValue( haystack[start + needle.Length - 1], out int distance) ? distance : needle.Length; start += shift; } return -1; } Here is the final comparison for the two selected scenarios: MediumRun means and standard deviations, with fifteen measurement iterations, ten warmups, and two processes per benchmark. All four implementations received the same text and needle. KMP and BMH include table construction on every call. This comparison is an independent run from the historical measurement used in the opening. Compare values within this table, not directly with the opening's numbers. | Text length | Scenario | Naive loop | IndexOf | KMP | Boyer–Moore–Horspool | |---:|---|---:|---:|---:|---:| | 4,096 | Pseudorandom late match | 3.291 ± 0.093 µs | 0.320 ± 0.016 µs | 5.026 ± 0.189 µs | 10.303 ± 0.223 µs | | 65,536 | Pseudorandom late match | 120.256 ± 7.345 µs | 5.145 ± 0.267 µs | 132.655 ± 6.025 µs | 175.052 ± 8.051 µs | | 4,096 | Alternating candidates, late mismatch | 113.756 ± 10.486 µs | 7.339 ± 0.273 µs | 7.382 ± 0.476 µs | 23.617 ± 0.225 µs | | 65,536 | Alternating candidates, late mismatch | 1,728.590 ± 66.428 µs | 122.848 ± 5.355 µs | 106.727 ± 6.830 µs | 378.960 ± 8.429 µs | For the pseudorandom late match, IndexOf had the lowest mean among the four methods. KMP and BMH were slower than the simple loop with table preparation included. We did not measure searches with reused tables, so we have not isolated how much of the difference comes from preprocessing versus searching. For the 65,536-element adversarial fixture, KMP's mean was approximately 13% lower than IndexOf's: 106.727 versus 122.848 µs. At length 4,096, their means were effectively similar relative to the dispersion. BMH was slower than IndexOf in all four measured cases. This illustrates why the answer depends on the algorithm, input, and treatment of preprocessing. It is not a universal ranking: these are teaching implementations, and BMH uses Dictionary<char, int>, not a specialized low-level table. MemoryDiagnoser also recorded per-call allocations: KMP allocated 88 bytes for the 16-element needle and 280 bytes for the 64-element needle; BMH allocated 472 and 1,552 bytes, respectively. Reusing a preprocessed table would change the comparison and was not measured here. Reproducing the two selected scenarios To reproduce the experiment without accessing the article's repository, create a console project using the .NET 10 SDK. Runtime version 10.0.12 must be installed because roll-forward will be disabled: dotnet new console -n SubstringBenchmark --framework net10.0 cd SubstringBenchmark Replace SubstringBenchmark.csproj with: <Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <OutputType>Exe</OutputType> <TargetFramework>net10.0</TargetFramework> <RuntimeFrameworkVersion>10.0.12</RuntimeFrameworkVersion> <RollForward>Disable</RollForward> <ImplicitUsings>enable</ImplicitUsings> <Nullable>enable</Nullable> </PropertyGroup> <ItemGroup> <PackageReference Include="BenchmarkDotNet" Version="0.15.8" /> </ItemGroup> </Project> Use the following structure in Program.cs. Inside the class, at the marked location, copy the six methods already shown in this article: SearchNaive, SearchRuntime, CreateOrdinaryHaystack, CreateAlternatingNeedleWithLateMismatch, SearchKnuthMorrisPratt, and SearchBoyerMooreHorspool. The four [Benchmark] wrappers are already included below; do not copy them again. This avoids duplicating the implementations. using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Running; BenchmarkRunner.Run<SubstringSearchBenchmarks>(args: args); [MediumRunJob] [MemoryDiagnoser] [DisassemblyDiagnoser(maxDepth: 3)] public class SubstringSearchBenchmarks { private string _haystack = ""; private string _needle = ""; [Params(4096, 65536)] public int HaystackLength { get; set; } [Params(SearchCase.RandomLateMatch, SearchCase.RepeatedAlternatingLateFalse)] public SearchCase Scenario { get; set; } [GlobalSetup] public void Setup() { int expected; if (Scenario == SearchCase.RandomLateMatch) { _haystack = CreateOrdinaryHaystack(HaystackLength); _needle = "abcdefghabcdefgh"; expected = HaystackLength - _needle.Length; _haystack = _haystack[..expected] + _needle; } else { _haystack = string.Create(HaystackLength, 0, static (chars, _) => { for (int i = 0; i < chars.Length; i++) chars[i] = (i & 1) == 0 ? 'a' : 'b'; }); _needle = CreateAlternatingNeedleWithLateMismatch(); expected = -1; } if (Naive() != expected || ReadOnlySpanIndexOf() != expected || KnuthMorrisPratt() != expected || BoyerMooreHorspool() != expected) { throw new InvalidOperationException("Unexpected setup result."); } if (Scenario == SearchCase.RepeatedAlternatingLateFalse) { uint mask = 0; int candidates = 0; for (int i = 0; i <= _haystack.Length - _needle.Length; i++) { if (_haystack[i] != _needle[0] || _haystack[i + 61] != _needle[61]) continue; candidates++; if (i < 16) mask |= 3u << (2 * i); } if (mask != 0x33333333 || candidates != ((HaystackLength - _needle.Length) / 2) + 1) { throw new InvalidOperationException("Inconsistent adversarial fixture."); } } } [Benchmark(Baseline = true)] public int Naive() => SearchNaive(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int ReadOnlySpanIndexOf() => SearchRuntime(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int KnuthMorrisPratt() => SearchKnuthMorrisPratt(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int BoyerMooreHorspool() => SearchBoyerMooreHorspool(_haystack.AsSpan(), _needle.AsSpan()); // Paste the six methods listed above here. public enum SearchCase { RandomLateMatch, RepeatedAlternatingLateFalse } } Run: dotnet run -c Release -- --filter '*' --artifacts artifacts This reproduction selects two scenarios × two lengths × four methods, for 16 MediumRun benchmarks. Setup checks the results and adversarial arithmetic; it does not replace the historical 34,193-case exhaustive and generated-input validation described earlier. Tables and disassembly are written to artifacts/results/. Assembly capture depends on the diagnoser's support for your system; another CPU or runtime may generate different instructions. Timings will also vary: reproducing the procedure does not guarantee reproducing the historical numbers. References - .NET 10.0.12 — SpanHelpers.Char.cs - Intel Intrinsics Guide, for the observed instructions: _mm256_cmpeq_epi16, _mm256_testz_si256, _mm256_movemask_epi8, _tzcnt_u32, and _blsr_u32. These describe equivalent semantics, not C/C++ calls made by the C# code. - Knuth, Morris, and Pratt, “Fast Pattern Matching in Strings”, SIAM Journal on Computing 6(2), 1977. - Boyer and Moore, “A Fast String Searching Algorithm”, Communications of the ACM 20(10), 1977. - Horspool, “Practical fast searching in strings”, Software: Practice and Experience 10(6), 1980. The compared implementation is Boyer–Moore–Horspool, not the full Boyer–Moore algorithm. - SIMD and hardware intrinsics in .NET - MemoryExtensions.IndexOf API

Como ReadOnlySpan<char>.IndexOf encontra uma substring com SIMD Read in English. Este artigo acompanha a sequência de ideias do vídeo: o vídeo anima os estados; aqui podemos pausar em cada diagrama e consultar o código e o assembly. O walkthrough usa o mesmo exemplo didático, separado das entradas medidas. O número do gancho vem de um MediumRun selecionado, mas ainda se refere a uma máquina e a um cenário específicos. A comparação com KMP e BMH ao final é material complementar, fora da montagem principal do vídeo. 1. O resultado que intriga Uma busca escrita com um loop parece ter menos custo que uma chamada a uma API de nível mais alto. Para testar essa intuição, comparamos duas implementações que procuram a primeira ocorrência de uma substring e retornam seu índice, ou -1 quando não há match. Este é o resultado do caso usado como gancho: uma substring de 16 chars inserida no fim de um texto pseudoaleatório de 65.536 chars. | Implementação | Média por busca | Desvio padrão | |---|---:|---:| | Loop ingênuo | 134,935 µs | 24,518 µs | | ReadOnlySpan<char>.IndexOf | 5,653 µs | 389 ns | Neste cenário e neste host, a média do loop foi cerca de 23,9× a média de IndexOf. Isso não é uma previsão para qualquer string, CPU ou versão do runtime. Mais adiante veremos que o loop e o runtime também usam algoritmos diferentes; o benchmark não isola somente o custo da abstração. 2. Os dois códigos comparados O harness usa BenchmarkDotNet; os métodos, entradas, configuração e instruções para montar uma reprodução estão neste artigo. Cada operação recebe um ReadOnlySpan<char> para o texto e outro para a needle. A comparação é ordinal: compara unidades char UTF-16 sem regras culturais. As duas implementações têm o mesmo contrato: retornar o índice da primeira ocorrência, ou -1 se não houver match. Em .NET, cada char representa uma unidade de código UTF-16, e não necessariamente um grafema percebido como um caractere. O baseline implementa busca ingênua (naive/brute-force): tenta cada posição e compara a needle caractere a caractere. O segundo método é a chamada à API do runtime. Estes são os dois métodos que conduzem a explicação principal: private static int SearchNaive( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) { if (needle.IsEmpty) return 0; int lastStart = haystack.Length - needle.Length; for (int start = 0; start <= lastStart; start++) { int offset = 0; while (offset < needle.Length && haystack[start + offset] == needle[offset]) { offset++; } if (offset == needle.Length) return start; } return -1; } private static int SearchRuntime( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) => haystack.IndexOf(needle, StringComparison.Ordinal); E o BenchmarkDotNet mede os resultados desses métodos por meio destes wrappers: [Benchmark(Baseline = true)] public int Naive() => SearchNaive(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int ReadOnlySpanIndexOf() => SearchRuntime(_haystack.AsSpan(), _needle.AsSpan()); No primeiro wrapper, portanto, medimos o loop ingênuo mostrado acima. No segundo, medimos ReadOnlySpan<char>.IndexOf com comparação ordinal. Para explicar o caminho observado no segundo método, usamos a implementação do runtime .NET 10.0.12: [SpanHelpers.IndexOf(ref char, int, ref char, int) em SpanHelpers.Char.cs](github.com/dotnet/runtime/blob/v10.0.12/src/libra…). Nesse host, o método usa filtro de candidatos vetorizado e confirma os candidatos comparando a substring completa; não é um algoritmo implementado por nós nem uma garantia de que outras versões ou CPUs escolherão o mesmo caminho. Antes de cronometrar, o GlobalSetup executa os quatro métodos para cada entrada e exige que todos retornem o resultado esperado. Assim, por exemplo, um cenário chamado “miss” não entra no benchmark se uma das buscas encontrar uma ocorrência acidental. A validação é feita antes das medições: int naive = SearchNaive(_haystack.AsSpan(), _needle.AsSpan()); int expected = Scenario switch { SearchCase.RandomEarlyMatch => 0, SearchCase.RandomMiddleMatch => HaystackLength / 2, SearchCase.RandomLateMatch or SearchCase.RepeatedPrefixLate => HaystackLength - _needle.Length, SearchCase.RandomMissing or SearchCase.RepeatedPrefixMissing or SearchCase.RepeatedAlternatingLateFalse => -1, _ => throw new ArgumentOutOfRangeException(nameof(Scenario)) }; VerifyResult(nameof(Naive), naive, expected); VerifyResult(nameof(KnuthMorrisPratt), SearchKnuthMorrisPratt(_haystack.AsSpan(), _needle.AsSpan()), expected); VerifyResult(nameof(BoyerMooreHorspool), SearchBoyerMooreHorspool(_haystack.AsSpan(), _needle.AsSpan()), expected); VerifyResult(nameof(ReadOnlySpanIndexOf), SearchRuntime(_haystack.AsSpan(), _needle.AsSpan()), expected); O benchmark compara uma busca escalar caractere a caractere com a implementação otimizada do runtime; não isola somente o custo de chamar uma API. A pergunta, portanto, é qual trabalho os dois algoritmos fazem, não se uma sintaxe ou uma abstração é intrinsecamente mais rápida. Ao final, incluímos KMP e Boyer–Moore–Horspool como uma resposta complementar à pergunta sobre outros algoritmos; eles não substituem o fio principal do vídeo. 3. Entradas: o que foi testado? O harness combina sete cenários com textos de 4.096 e 65.536 chars. As entradas são construídas uma vez no setup, fora da região cronometrada. | Cenário | Texto e needle | Match esperado | |---|---|---| | RandomEarlyMatch | Texto pseudoaleatório no alfabeto a–h; needle abcdefghabcdefgh inserida no índice 0. | índice 0 | | RandomMiddleMatch | Mesmo gerador e needle, inserida no meio do texto. | índice HaystackLength / 2 | | RandomLateMatch | Mesmo gerador e needle, inserida nos últimos 16 chars. | índice HaystackLength - 16 | | RandomMissing | Texto do mesmo gerador/alfabeto, sem inserir a needle. | -1, conferido no setup | | RepeatedPrefixMissing | Texto formado só por a; needle aaaaaaab. | -1 | | RepeatedPrefixLate | Texto formado só por a, com aaaaaaab substituindo os oito chars finais. | índice HaystackLength - 8 | | RepeatedAlternatingLateFalse | Texto ab alternado; needle de 64 chars alternados nos primeiros 63 e terminando em a. | -1; falsos candidatos falham no último char | O gerador pseudoaleatório é determinístico: começa em 0xC0FFEE e, para cada posição, atualiza um estado de 32 bits com state = state * 1664525 + 1013904223; os três bits mais altos selecionam um dos oito caracteres a–h. Este é o código usado para gerar o texto: private static string CreateOrdinaryHaystack(int length) { const string alphabet = "abcdefgh"; var chars = new char[length]; uint state = 0xC0FFEE; for (int i = 0; i < chars.Length; i++) { state = (state * 1664525) + 1013904223; chars[i] = alphabet[(int)(state >> 29)]; } return new string(chars); } Nos cenários de match, o trecho correspondente é substituído pela needle. Para o caso tardio usado no gancho, o código equivale a: const string needle = "abcdefghabcdefgh"; string haystack = CreateOrdinaryHaystack(length); int matchIndex = length - needle.Length; // RandomLateMatch haystack = haystack[..matchIndex] + needle + haystack[(matchIndex + needle.Length)..]; No cenário ausente, o mesmo alfabeto é mantido, evitando um miss trivial por caracteres impossíveis: o setup confirma que a needle completa realmente não aparece. O cenário adversarial do benchmark Este caso medido é diferente tanto do gancho quanto do exemplo didático das seções 6–7. O texto alterna a e b; a needle alterna também, mas seu último char é alterado para a. A função do benchmark constrói essa needle assim: private static string CreateAlternatingNeedleWithLateMismatch() { const int length = 64; return string.Create(length, 0, static (chars, _) => { for (int i = 0; i < chars.Length; i++) chars[i] = (i & 1) == 0 ? 'a' : 'b'; chars[^1] = 'a'; }); } O runtime compara o primeiro char (a) e procura, de trás para a frente, o último char da needle que difere dele. Como os chars 63 e 62 são a, esse ponto é o índice 61, que contém b. No texto alternado, ambos os filtros passam para todo início par: haystack[i] == 'a' e haystack[i + 61] == 'b'. A substring inteira, porém, falha só no último char, porque o texto contém b no índice relativo 63 e a needle contém a. | Tamanho do texto | Candidatos falsos esperados | Resultado | |---:|---:|---:| | 4.096 | 2.017 | -1 | | 65.536 | 32.737 | -1 | O setup calcula e confere a compactação conceitual de um bit por lane (0x5555), a máscara esperada após extrair um bit por byte (0x33333333) e a contagem total de candidatos. A primeira compactação não é o vetor de comparação nem uma variável intermediária do runtime: cada lane aprovada contém FFFF. O cálculo é uma validação escalar do fixture, não uma leitura dos registradores do runtime. 4. Medição e resultados Usamos BenchmarkDotNet 0.15.8, build Release, .NET 10.0.12 (SDK 10.0.401), Linux Ubuntu 24.04.4 x64 e AMD EPYC 7763 a 2,45 GHz. O BenchmarkDotNet expôs 1 CPU ao processo. O host disponibilizou AVX2 e BMI1; a captura do assembly usada no walkthrough é desse ambiente. O fixture combina dois tamanhos e sete cenários. Cada combinação é executada pelos quatro métodos, totalizando 56 combinações método/caso na matriz ShortRun. Os atributos que configuram os benchmarks são: [ShortRunJob] [MemoryDiagnoser] [DisassemblyDiagnoser(maxDepth: 3)] public class SubstringSearchBenchmarks { [Params(4096, 65536)] public int HaystackLength { get; set; } [Params( SearchCase.RandomEarlyMatch, SearchCase.RandomMiddleMatch, SearchCase.RandomLateMatch, SearchCase.RandomMissing, SearchCase.RepeatedPrefixMissing, SearchCase.RepeatedPrefixLate, SearchCase.RepeatedAlternatingLateFalse)] public SearchCase Scenario { get; set; } } A matriz ampla usou ShortRun (3 aquecimentos, 3 medições). Para o match tardio e o caso adversarial, também rodamos MediumRun (10 aquecimentos, 15 medições, 2 processos por benchmark). A tabela mostra as médias e desvios padrão do MediumRun; as unidades foram convertidas para facilitar a leitura: | Entrada | Caso | Ingênua: média ± desvio padrão | IndexOf: média ± desvio padrão | Razão das médias | |---:|---|---:|---:|---:| | 4.096 | Match no fim, texto pseudoaleatório | 3,443 ± 0,238 µs | 323 ± 15 ns | 10,7× | | 65.536 | Match no fim, texto pseudoaleatório | 134,935 ± 24,518 µs | 5,653 ± 389 ns | 23,9× | | 4.096 | Muitos candidatos, falha no último char | 149,369 ± 57,317 µs | 7,645 ± 390 ns | 19,5× | | 65.536 | Muitos candidatos, falha no último char | 1,925 ± 0,202 ms | 129,186 ± 7,864 µs | 14,9× | O cenário adversarial tem dispersão considerável, especialmente no loop ingênuo; por isso mostramos a dispersão junto à média e evitamos tratá-lo como um caso típico. Também não atribuímos todo o ganho a SIMD: o baseline percorre posições em C#, enquanto IndexOf usa outro algoritmo, além de instruções vetoriais. O dado sustenta uma conclusão delimitada: neste host, para estas entradas, a implementação do runtime foi mais rápida. Na execução original, a validação comparou IndexOf, KMP e BMH com o baseline ingênuo em 34.193 casos determinísticos: todas as strings binárias de até 8 chars e needles de até 5, mais 2.000 entradas geradas com caracteres diversos, incluindo NUL e unidades surrogate. Ele inclui needle vazia, needle maior que o texto, matches injetados e misses. Na captura original, adicionar --job Medium à classe com [ShortRunJob] executou os dois jobs; os números reportados são das linhas Job=MediumRun. A reprodução ao final usa apenas [MediumRunJob], para executar os dois cenários selecionados sem repetir a matriz ShortRun. O projeto fixa RuntimeFrameworkVersion em 10.0.12 e desativa roll-forward. Sua configuração e os comandos estão na seção de reprodução, sem depender de arquivos externos ao artigo. 5. A ideia de SIMD SIMD aplica uma mesma operação a vários elementos de dados. No caminho observado, os elementos são chars UTF-16 de 16 bits. Um vetor de 256 bits pode conter 16 lanes char: lane: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 char: [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] Isso não significa que toda a busca acontece numa única instrução. O vetor seleciona vários inícios possíveis para verificação; o caminho ainda percorre blocos, processa máscaras e confirma a substring completa. 6. Encontrar candidatos Uma entrada pequena para acompanhar as contas Usamos a mesma fixture didática do vídeo: uma needle de 64 unidades UTF-16, inserida no índice 6 de um texto de 79 unidades, preenchido com x fora desse trecho. A needle alterna a e b, exceto pelo último elemento, que é a. Este código reproduz a entrada; não é um cenário cronometrado: char[] value = new char[64]; for (int i = 0; i < value.Length; i++) value[i] = (i & 1) == 0 ? 'a' : 'b'; value[63] = 'a'; char[] text = new char[79]; Array.Fill(text, 'x'); value.AsSpan().CopyTo(text.AsSpan(6)); int result = text.AsSpan().IndexOf(value.AsSpan(), StringComparison.Ordinal); // result == 6 Há 79 - 64 + 1 = 16 inícios válidos, de 0 a 15: cabem em uma janela de 16 lanes. O objetivo é explicar o caminho até um match, não representar os dados responsáveis pelos tempos do benchmark. Escolher dois pontos da needle O primeiro ponto é value[0] == 'a'. Para o segundo, o runtime procura de trás para a frente um elemento diferente do primeiro: posição na needle: 0 1 ... 61 62 63 valor: a b ... b a a ^ ^ ch1 ch2 As posições 63 e 62 contêm a; a posição 61 contém b. Portanto, ch1ch2Distance = 61. ch2 não significa necessariamente o último caractere, mas o segundo ponto escolhido para filtrar. Os valores são replicados em vetores: ch1 contém 16 cópias de a, e ch2, 16 cópias de b. Para um início i, perguntamos se text[i] == 'a' e text[i + 61] == 'b'. Se alguma condição falhar, esse início não pode ser uma ocorrência completa. Offset, lane e endereço não são a mesma coisa offset é o início da janela de candidatos no texto. A lane j representa o início offset + j. Nesta janela, offset = 0: | Lane | Início candidato | Primeiro ponto | Segundo ponto | |---:|---:|---|---| | 6 | 0 + 6 = 6 | text[6] == 'a' | text[0 + 6 + 61], isto é, text[67] == 'b' | | 8 | 0 + 8 = 8 | text[8] == 'a' | text[0 + 8 + 61], isto é, text[69] == 'b' | O recorte do runtime executa primeiro a comparação do segundo ponto, depois a do primeiro e finalmente o AND. Abaixo, os tipos das variáveis foram omitidos e as quebras ajustadas: cmpCh2 = Vector256.Equals( ch2, Vector256.LoadUnsafe( ref searchSpace, (nuint)(offset + ch1ch2Distance))); cmpCh1 = Vector256.Equals( ch1, Vector256.LoadUnsafe(ref searchSpace, (nuint)offset)); cmpAnd = (cmpCh1 & cmpCh2).AsByte(); Neste ramo, searchSpace referencia elementos ushort: os deslocamentos das cargas são em unidades UTF-16 de 16 bits, não em bytes. As cargas leem text[61..77] e text[0..16], usando limites finais exclusivos. Somar a distância na segunda carga mantém os dois pontos de cada candidato na mesma lane: a lane 6 relaciona text[6] e text[67], não elementos de inícios diferentes. Comparar e combinar as 16 lanes Cada igualdade produz FFFF quando verdadeira e 0000 quando falsa. Na tabela, 1 e 0 são abreviações desses valores de 16 bits, não os valores inteiros gravados no vetor. A operação continua sendo == em todas as lanes, inclusive quando seu resultado é falso. lane: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 text[i+61]: b a b a b a b a a x x x x x x x ch2: b b b b b b b b b b b b b b b b cmpCh2: 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 text[i]: x x x x x x a b a b a b a b a b ch1: a a a a a a a a a a a a a a a a cmpCh1: 0 0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 cmpCh1: 0 0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 cmpCh2: 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 AND: 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 A lane 6 passa nas duas igualdades: a == a e b == b. Na lane 8, a primeira passa (text[8] == 'a'), mas a segunda falha: text[69] contém a, e a == b é falso. O AND mantém apenas a lane 6. As comparações SIMD processam as lanes em paralelo; a leitura das colunas 6 e 8 apenas explica seus resultados. AsByte() reinterpreta os mesmos bits como bytes, sem converter FFFF no número 1. O teste cmpAnd != Vector256<byte>.Zero produz uma decisão escalar: há candidato. Ele não informa o índice nem confirma a substring. No assembly capturado do benchmark, vpcmpeqw, vpand e vptest correspondem às comparações, combinação e teste. Essa evidência é do host documentado, não uma garantia de tradução para qualquer máquina. 7. Localizar e confirmar o candidato De uma lane de 16 bits a dois bits na máscara O runtime extrai o bit mais significativo de cada byte: uint mask = cmpAnd.ExtractMostSignificantBits(); São 16 elementos de 16 bits, reinterpretados como 32 bytes. Na lane 6, o resultado verdadeiro contém dois bytes FF; nos demais elementos, ambos os bytes são 00: lane 6: FFFF bytes no vetor: byte 12 = FF byte 13 = FF bits de cada byte: 11111111 11111111 bit mais significativo: 1 1 posição na máscara: 12 13 A lane 0 corresponde aos bytes e bits 0–1; a lane 1, a 2–3; a lane 6, a 12–13. Logo, a máscara efetivamente extraída é 0x00003000: índice: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 bit: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 índice: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 bit: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Aqui os índices crescem da esquerda para a direita, como no vídeo. Não é a escrita binária numérica usual: o bit menos significativo está à esquerda. Cada bit representa um byte do vetor de resultados, não diretamente um caractere. No assembly observado, a extração corresponde a vpmovmskb. Contar, converter e somar o offset O próximo cálculo no código é: nint charPos = (nint)(uint.TrailingZeroCount(mask) / sizeof(ushort)); A contagem começa no bit 0 e encontra 12 zeros antes do primeiro 1. Portanto, TrailingZeroCount(0x00003000) = 12. Dividir por sizeof(ushort), que vale 2, converte a posição do byte representado pelo bit em posição de elemento: primeiro bit marcado: 12 charPos: 12 / 2 = 6 início no texto: offset + charPos = 0 + 6 = 6 charPos é relativo à janela; o índice no texto inclui offset. Os valores coincidem neste exemplo porque o offset é zero. No assembly capturado, a contagem corresponde a tzcnt e a divisão por dois a um deslocamento de bits. Confirmar a sequência inteira Só agora verificamos a igualdade completa. O recorte do runtime é: if (valueLength == 2 || SequenceEqual( ref Unsafe.As<char, byte>(ref Unsafe.Add(ref searchSpace, offset + charPos)), ref Unsafe.As<char, byte>(ref value), (nuint)(uint)valueLength * 2)) { return (int)(offset + charPos); } O teste valueLength == 2 é falso, pois nossa needle tem 64 elementos. SequenceEqual compara seus 128 bytes com os do trecho iniciado em text[6]. Os intervalos abaixo têm limite final exclusivo: | Trecho da needle | Trecho correspondente do texto | Igualdade | |---|---|---| | value[0..16] | text[6..22] | abababababababab | | value[16..32] | text[22..38] | abababababababab | | value[32..48] | text[38..54] | abababababababab | | value[48..64] | text[54..70] | abababababababaa | Os quatro pares são iguais: a chamada retorna true, e o método retorna offset + charPos = 6. A tabela organiza os dados, não a ordem de execução das instruções. A confirmação também usa SIMD no assembly capturado para a needle desse tamanho no benchmark; não precisamos abrir sua implementação para entender seu papel na busca. Se a confirmação falhasse, o runtime limparia os dois bits do candidato e continuaria com o próximo par marcado, se ainda houvesse algum. Isso não acontece nesta fixture, que encontra o match. Podemos conferir a matemática sem ler registradores internos. Acrescente este trecho ao exemplo C# que constrói text e value na seção 6: uint expectedMask = 0; for (int lane = 0; lane < 16; lane++) { if (text[lane] == value[0] && text[lane + 61] == value[61]) expectedMask |= 3u << (2 * lane); } if (expectedMask != 0x00003000 || uint.TrailingZeroCount(expectedMask) / sizeof(ushort) != 6 || !text.AsSpan(6, 64).SequenceEqual(value.AsSpan()) || result != 6) { throw new InvalidOperationException("Fixture didática inconsistente."); } expectedMask é um cálculo escalar de verificação, não a extração do vetor usado internamente por IndexOf. 8. Confirmar, não presumir O exemplo didático tem uma ocorrência no índice 6. O cenário adversarial medido na seção 3 mantém a mesma needle, mas usa um texto inteiramente alternado, sem inserir a ocorrência. Nesse texto, todo início par passa nos dois pontos do filtro, mas a comparação completa falha no último elemento: o texto contém b, enquanto a needle termina em a. | Entrada | Candidatos na primeira janela | Máscara extraída por byte | Resultado da busca | |---|---|---|---:| | Didática: 79 elementos, ocorrência inserida | Apenas lane 6 | 0x00003000 | 6 | | Adversarial: texto ab alternado | Lanes 0, 2, 4, 6, 8, 10, 12 e 14 | 0x33333333 | -1 | No adversarial, poderíamos resumir as lanes aprovadas como 0x5555, usando um bit por lane. Essa é apenas uma representação conceitual; os resultados vetoriais são FFFF/0000, e a variável mask extraída contém dois bits marcados por candidato. Por exemplo, depois de rejeitar o início 0, limpar seus bits 0–1 muda a máscara de 0x33333333 para 0x33333330. A próxima contagem dá 4; 4 / 2 = 2 aponta ao candidato seguinte. No assembly observado, dois blsr fazem a limpeza dos dois bits. No texto de 65.536 chars, há ((65.536 - 64) / 2) + 1 = 32.737 inícios pares candidatos. Todos são falsos e falham no elemento final. A contagem refere-se às posições distintas que passam no filtro, não a uma medição de chamadas internas de SequenceEqual; a última janela pode sobrepor posições já examinadas. O harness recalcula escalarmente, a partir do fixture, a máscara esperada do primeiro bloco e a contagem total de candidatos no setup, além de verificar que os quatro métodos retornam -1. Isso valida os dados e o cálculo do caso, não lê registradores internos do runtime; essa ligação vem do código versionado e do assembly capturado. Os primeiros passos esperados são: | Máscara antes | Zeros à direita | Posição / 2 | Confirmação | |---|---:|---:|---| | 0x33333333 | 0 | 0 | falha no elemento relativo 63 | | 0x33333330 | 4 | 2 | falha no elemento relativo 63 | | 0x33333300 | 8 | 4 | falha no elemento relativo 63 | | 0x33333000 | 12 | 6 | falha no elemento relativo 63 | Por fim, o resultado de aproximadamente 24× da abertura pertence a uma terceira entrada: match tardio pseudoaleatório, com needle de 16 elementos e texto de 65.536. Não é o resultado do adversarial nem da fixture didática. O exemplo explica o algoritmo; cada medição pertence à sua própria entrada. 9. Avançar, terminar e voltar ao resultado Depois de verificar um bloco, o algoritmo avança por uma largura vetorial. Se o próximo avanço ultrapassaria o último início que ainda cabe, o loop volta ao offset do último bloco completo, sobrepondo parte do bloco anterior para examinar as posições finais sem ler além da entrada. Entradas com menos de oito inícios possíveis não cabem no caminho vetorial de 128 bits deste host e seguem o caminho escalar. Os limites e branches exatos estão no código-fonte do runtime, não são inferidos apenas pelo nome da API. No assembly desta execução, o filtro usa AVX2: comparação de words, AND, teste de vetor e extração de máscara; tzcnt e blsr pertencem a BMI1, e a confirmação também usa comparações vetoriais de bytes. Essa observação vale para a versão do runtime, JIT e CPU registrados; outras máquinas podem usar outro caminho. O número inicial mostra que neste experimento o runtime encontra candidatos em paralelo e supera a busca escalar caractere a caractere. Não demonstra que menos abstração sempre custa mais ou que SIMD sempre vence. Para interpretar qualquer benchmark, precisamos saber quais dados e qual algoritmo foram medidos. Relação com as instruções observadas Esta tabela resume a desassemblagem capturada no host Linux x64 / EPYC 7763, .NET 10.0.12, AVX2 e BMI1. Não é uma promessa de tradução um-para-um para outras CPUs ou versões, nem uma nova captura da fixture didática: | Operação | Instruções observadas | Papel | |---|---|---| | Replicar ch1 e ch2 | vmovd, vpbroadcastw | Repetir o valor em 16 lanes de 16 bits. | | Vector256.Equals | vpcmpeqw | Produzir FFFF ou 0000 por lane. | | AND dos resultados | vpand | Manter quem passou nas duas comparações. | | Teste do vetor contra zero | vptest e salto | Evitar extrair máscara quando não há candidato. | | ExtractMostSignificantBits | vpmovmskb | Extrair 32 bits, um por byte. | | TrailingZeroCount | tzcnt | Localizar o primeiro bit marcado. | | Divisão por 2 | shr ..., 1 | Converter posição de byte para elemento UTF-16. | | Limpar dois bits do candidato rejeitado | blsr, duas vezes | Prosseguir para outro candidato. | Para a confirmação de 128 bytes do caso adversarial, o assembly observado compara trechos AVX2 de 32 bytes. Os três primeiros trechos coincidem; o quarto contém a divergência no elemento final. Isso descreve a captura desse caso, não uma exigência de que SequenceEqual sempre execute quatro passos. A título de curiosidade: como se saem KMP e Boyer–Moore? Depois da conclusão principal, vale a curiosidade: como a comparação se comportaria contra algoritmos que evitam recomeçar a busca do zero? Para isso, acrescentamos duas implementações ao mesmo harness: - Knuth–Morris–Pratt (KMP): constrói a tabela de prefixos da needle e a usa para retomar do maior prefixo que ainda pode coincidir. Busca e pré-processamento têm custo O(n + m) no total, onde n é o texto e m a needle. - Boyer–Moore–Horspool (BMH): compara da direita para a esquerda e, após uma falha, avança conforme a posição do último caractere da janela na needle. O pré-processamento custa O(m); a busca costuma saltar posições, mas o pior caso permanece O(nm). Para esta comparação de uma busca por needle, o tempo cronometrado inclui construir as tabelas de cada algoritmo em cada chamada. Assim, o KMP aloca e preenche a tabela de prefixos, e o BMH monta sua tabela esparsa de saltos (Dictionary<char, int>) antes de percorrer o texto. O runtime recebe a mesma needle e o mesmo texto, sem estado pré-processado reutilizado. Os quatro métodos retornam o primeiro índice ou -1; o GlobalSetup confere todos contra o resultado esperado antes da medição. O código adicional do benchmark é: [Benchmark] public int KnuthMorrisPratt() => SearchKnuthMorrisPratt(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int BoyerMooreHorspool() => SearchBoyerMooreHorspool(_haystack.AsSpan(), _needle.AsSpan()); private static int SearchKnuthMorrisPratt( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) { if (needle.IsEmpty) return 0; if (haystack.Length < needle.Length) return -1; var prefixLengths = new int[needle.Length]; for (int i = 1, prefixLength = 0; i < needle.Length; i++) { while (prefixLength > 0 && needle[i] != needle[prefixLength]) prefixLength = prefixLengths[prefixLength - 1]; if (needle[i] == needle[prefixLength]) prefixLength++; prefixLengths[i] = prefixLength; } for (int i = 0, prefixLength = 0; i < haystack.Length; i++) { while (prefixLength > 0 && haystack[i] != needle[prefixLength]) prefixLength = prefixLengths[prefixLength - 1]; if (haystack[i] == needle[prefixLength]) prefixLength++; if (prefixLength == needle.Length) return i - needle.Length + 1; } return -1; } private static int SearchBoyerMooreHorspool( ReadOnlySpan<char> haystack, ReadOnlySpan<char> needle) { if (needle.IsEmpty) return 0; if (haystack.Length < needle.Length) return -1; var shifts = new Dictionary<char, int>(needle.Length); for (int i = 0; i < needle.Length - 1; i++) shifts[needle[i]] = needle.Length - 1 - i; int lastStart = haystack.Length - needle.Length; int start = 0; while (start <= lastStart) { int offset = needle.Length - 1; while (offset >= 0 && haystack[start + offset] == needle[offset]) offset--; if (offset < 0) return start; int shift = shifts.TryGetValue( haystack[start + needle.Length - 1], out int distance) ? distance : needle.Length; start += shift; } return -1; } Esta é a comparação final para os dois cenários escolhidos. São médias e desvios padrão do MediumRun (15 medições, 10 aquecimentos, 2 processos por benchmark); todas as quatro implementações receberam o mesmo texto e needle. As colunas KMP e BMH incluem a construção das tabelas em cada chamada. Esta rodada comparativa é independente da medição histórica usada no gancho; os tempos da tabela devem ser comparados entre si, não diretamente com os números da abertura. | Texto | Cenário | Loop ingênuo | IndexOf | KMP | Boyer–Moore–Horspool | |---:|---|---:|---:|---:|---:| | 4.096 | Match tardio pseudoaleatório | 3,291 ± 0,093 µs | 0,320 ± 0,016 µs | 5,026 ± 0,189 µs | 10,303 ± 0,223 µs | | 65.536 | Match tardio pseudoaleatório | 120,256 ± 7,345 µs | 5,145 ± 0,267 µs | 132,655 ± 6,025 µs | 175,052 ± 8,051 µs | | 4.096 | Candidatos alternados, mismatch tardio | 113,756 ± 10,486 µs | 7,339 ± 0,273 µs | 7,382 ± 0,476 µs | 23,617 ± 0,225 µs | | 65.536 | Candidatos alternados, mismatch tardio | 1.728,590 ± 66,428 µs | 122,848 ± 5,355 µs | 106,727 ± 6,830 µs | 378,960 ± 8,429 µs | No match tardio pseudoaleatório, IndexOf teve a menor média entre os quatro métodos; KMP e BMH foram mais lentos que o loop simples com a preparação das tabelas incluída. Não medimos essas buscas com tabelas reutilizadas, portanto não isolamos quanto da diferença vem do pré-processamento e quanto vem da busca. No fixture adversarial de 65.536 chars, KMP ficou cerca de 13% abaixo da média de IndexOf (106,727 contra 122,848 µs); no tamanho 4.096, suas médias ficaram praticamente iguais diante da dispersão. BMH foi mais lento que IndexOf nos quatro casos medidos. Isso ilustra por que a resposta depende do algoritmo, da entrada e de como o pré-processamento é contabilizado. Não é um ranking universal: são implementações didáticas, e o BMH usa uma tabela baseada em Dictionary<char, int>, não uma tabela especializada de baixo nível. MemoryDiagnoser também registrou alocações por chamada: KMP alocou 88 bytes para a needle de 16 chars e 280 bytes para a de 64; BMH alocou 472 e 1.552 bytes, respectivamente. Reutilizar uma tabela pré-processada mudaria essa comparação e não foi medido aqui. Reprodução dos dois cenários selecionados Para montar uma reprodução sem acessar o repositório do artigo, crie um projeto de console com SDK .NET 10. A versão de runtime 10.0.12 precisa estar instalada, pois o roll-forward estará desativado: dotnet new console -n SubstringBenchmark --framework net10.0 cd SubstringBenchmark Substitua SubstringBenchmark.csproj por: <Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <OutputType>Exe</OutputType> <TargetFramework>net10.0</TargetFramework> <RuntimeFrameworkVersion>10.0.12</RuntimeFrameworkVersion> <RollForward>Disable</RollForward> <ImplicitUsings>enable</ImplicitUsings> <Nullable>enable</Nullable> </PropertyGroup> <ItemGroup> <PackageReference Include="BenchmarkDotNet" Version="0.15.8" /> </ItemGroup> </Project> Use a estrutura abaixo em Program.cs. Dentro da classe, no local indicado, copie os seis métodos já apresentados neste artigo: SearchNaive, SearchRuntime, CreateOrdinaryHaystack, CreateAlternatingNeedleWithLateMismatch, SearchKnuthMorrisPratt e SearchBoyerMooreHorspool. Os quatro wrappers [Benchmark] já estão abaixo; não os copie novamente. Assim não precisamos duplicar as implementações no texto. using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Running; BenchmarkRunner.Run<SubstringSearchBenchmarks>(args: args); [MediumRunJob] [MemoryDiagnoser] [DisassemblyDiagnoser(maxDepth: 3)] public class SubstringSearchBenchmarks { private string _haystack = ""; private string _needle = ""; [Params(4096, 65536)] public int HaystackLength { get; set; } [Params(SearchCase.RandomLateMatch, SearchCase.RepeatedAlternatingLateFalse)] public SearchCase Scenario { get; set; } [GlobalSetup] public void Setup() { int expected; if (Scenario == SearchCase.RandomLateMatch) { _haystack = CreateOrdinaryHaystack(HaystackLength); _needle = "abcdefghabcdefgh"; expected = HaystackLength - _needle.Length; _haystack = _haystack[..expected] + _needle; } else { _haystack = string.Create(HaystackLength, 0, static (chars, _) => { for (int i = 0; i < chars.Length; i++) chars[i] = (i & 1) == 0 ? 'a' : 'b'; }); _needle = CreateAlternatingNeedleWithLateMismatch(); expected = -1; } if (Naive() != expected || ReadOnlySpanIndexOf() != expected || KnuthMorrisPratt() != expected || BoyerMooreHorspool() != expected) { throw new InvalidOperationException("Resultado inesperado no setup."); } if (Scenario == SearchCase.RepeatedAlternatingLateFalse) { uint mask = 0; int candidates = 0; for (int i = 0; i <= _haystack.Length - _needle.Length; i++) { if (_haystack[i] != _needle[0] || _haystack[i + 61] != _needle[61]) continue; candidates++; if (i < 16) mask |= 3u << (2 * i); } if (mask != 0x33333333 || candidates != ((HaystackLength - _needle.Length) / 2) + 1) { throw new InvalidOperationException("Fixture adversarial inconsistente."); } } } [Benchmark(Baseline = true)] public int Naive() => SearchNaive(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int ReadOnlySpanIndexOf() => SearchRuntime(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int KnuthMorrisPratt() => SearchKnuthMorrisPratt(_haystack.AsSpan(), _needle.AsSpan()); [Benchmark] public int BoyerMooreHorspool() => SearchBoyerMooreHorspool(_haystack.AsSpan(), _needle.AsSpan()); // Cole aqui os seis métodos indicados acima. public enum SearchCase { RandomLateMatch, RepeatedAlternatingLateFalse } } Execute: dotnet run -c Release -- --filter '*' --artifacts artifacts Essa reprodução seleciona dois cenários × dois tamanhos × quatro métodos, totalizando 16 benchmarks MediumRun. O setup verifica os resultados e a matemática do adversarial; ele não substitui a validação exaustiva histórica de 34.193 casos descrita anteriormente. As tabelas e a desassemblagem são gravadas em artifacts/results/. A captura de assembly depende do suporte do diagnoser no sistema; outra CPU ou runtime pode gerar outras instruções. Os tempos também variarão: reproduzir o procedimento não garante reproduzir os números históricos. Referências - .NET 10.0.12 — SpanHelpers.Char.cs - Intel Intrinsics Guide, para as instruções observadas: _mm256_cmpeq_epi16, _mm256_testz_si256, _mm256_movemask_epi8, _tzcnt_u32 e _blsr_u32. São referências de semântica equivalente, não chamadas C/C++ feitas pelo código C#. - Knuth, Morris e Pratt, “Fast Pattern Matching in Strings”, SIAM Journal on Computing 6(2), 1977. - Boyer e Moore, “A Fast String Searching Algorithm”, Communications of the ACM 20(10), 1977. - Horspool, “Practical fast searching in strings”, Software: Practice and Experience 10(6), 1980. A implementação comparada é Boyer–Moore–Horspool, não Boyer–Moore completo. - SIMD e hardware intrinsics no .NET - API MemoryExtensions.IndexOf