reubeno

winnow-args 0.5.0 I spent a bit more time working on winnow-args. I improved it after trying to integrate it with brush, maki, em. What changed Mainly I expanded a bit relations between fields and groups to address what maki needs, improved the subcommand and multicommand support in preparation to em release and fixed more corner cases in brush improving winnow-args in the process. Now we are closer to have feature parity with usage and clap but the road is still long. The early adopters that used the derive API do not have to change anything in their code. | Feature | 0.2.0 | 0.3.0 | 0.4.0 | 0.5.0 | usage 7 | clap 4 | |---|:-:|:-:|:-:|:-:|:-:|:-:| | Errors and rules | | | | | | | | "Did you mean" tips | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | Every missing required argument reported at once | – | – | ✓ | ✓ | ✓ | ✓ | | required_if, required_unless_all | – | 0.3.1 | ✓ | ✓ | ✓ | ✓ | | required_if_all | – | – | ✓ | ✓ | ✓ | ✓ | | requires_if, exclusive | – | 0.3.1 | ✓ | ✓ | ✓ | ✓ | | requires naming a group | – | – | ✓ | ✓ | ? | ✓ | | Own message for a conflict or requirement | – | – | ✓ | ✓ | – | – | | validate, a check on the parsed struct | – | 0.3.1 | ✓ | ✓ | ✓ | ~ | | Values and defaults | | | | | | | | from_str, a value parsed by FromStr | – | ✓ | ✓ | ✓ | ✓ | ✓ | | default_from, a default made from other fields | – | 0.3.1 | ✓ | ✓ | ✓ | ~ | | A bare env, named after the field | – | – | ✓ | ✓ | ? | ✓ | | A repeated default on a Vec | – | – | ✓ | ✓ | ? | ✓ | | bool_value (--wait=false) | – | – | ✓ | ✓ | ✓ | ~ | | value_names (--point <X> <Y>) | – | – | ✓ | ✓ | ✓ | ✓ | | env_fallback, more variable names | – | – | – | ✓ | ✓ | – | | Subcommands | | | | | | | | Generated dispatch (run, run_async) | 0.2.1 | ✓ | ✓ | ✓ | ✓ | – | | A flag that selects the default subcommand | 0.2.1 | ✓ | ✓ | ✓ | ✓ | – | | Global flags in a flattened struct | 0.2.1 | ✓ | ✓ | ✓ | ✓ | ✓ | | A subcommand variant with named fields | – | ✓ | ✓ | ✓ | ✓ | ✓ | | default_subcommand_on_empty | – | 0.3.1 | ✓ | ✓ | ✓ | – | | External subcommands | – | – | ✓ | ✓ | ✓ | ✓ | | Multicall | – | – | ✓ | ✓ | ✓ | ✓ | | Help | | | | | | | | *emphasis* and **strong** painted | ✓ | ✓ | ✓ | ✓ | ? | – | | Global flags listed in a subcommand's help | – | ✓ | ✓ | ✓ | ✓ | ~ | | Default subcommand marked, its page after the root's | – | ✓ | ✓ | ✓ | ✓ | – | | Examples | – | ✓ | ✓ | ✓ | ✓ | ~ | | A paragraph under a heading | – | ✓ | ✓ | ✓ | ✓ | ~ | | A flag's warning | – | ✓ | ✓ | ✓ | ✓ | – | | flatten_help | – | ✓ | ✓ | ✓ | ✓ | ✓ | | hide_default_value, hide_env, display_order | – | 0.3.1 | ✓ | ✓ | ✓ | ✓ | | verbatim_doc_comment | – | – | ✓ | ✓ | ✓ | ✓ | | Width in terminal columns (unicode) | – | – | ✓ | ✓ | ✓ | ✓ | | visible_alias | – | – | ✓ | ✓ | ✓ | ✓ | | long_version, before_help, usage | – | – | ✓ | ✓ | ✓ | ✓ | | What each choice means | – | – | – | ✓ | ✓ | ✓ | | hide_choices | – | – | – | ✓ | ✓ | ✓ | | help = CONSTANT | – | – | – | ✓ | – | ✓ | | brief_help: --help prints the short page | – | – | – | ✓ | – | – | | Deprecation | | | | | | | | Deprecated flags, positionals and subcommands | – | – | – | ✓ | ✓ | – | | deprecated_warn_at, deprecated_remove_at | – | – | – | ✓ | ✓ | – | | deprecated_env | – | – | – | ✓ | ✓ | – | | Completion and pages | | | | | | | | Completion of a value by hint or function | ✓ | ✓ | ✓ | ✓ | ✓ | ~ | | Man and markdown pages | ✓ | ✓ | ✓ | ✓ | ✓ | ~ | | Effects, outputs, exit codes in the pages | – | ✓ | ✓ | ✓ | ✓ | – | | A page for each multicall command | – | – | ✓ | ✓ | ? | – | Here some more details Errors and rules An unknown flag, subcommand or value now comes with a tip for what was probably meant (0.2). The search runs only once there is an error, so a correct command line pays nothing for it. When several required arguments are missing, they are all reported at once, each as --help lists it (0.4). The rules gained most of what clap has: required_if, required_unless_all, requires_if and exclusive (0.3.1), and required_if_all, plus requires naming a group (0.4). conflicts, requires and groups take a message with the program's own wording. For anything the attributes can't express, validate = function runs the program's own check on the parsed struct. {% asciinema 81pPfsa7FVdxICbN %} Values and defaults We have new attributes to parse the cli values to types: - from_str parses a field with its type's FromStr, for a type from another crate (0.3). - default_from makes a default from other fields, after their own defaults (0.3.1). - bool_value takes --wait=false, and value_names gives a flag several placeholders, --size <W> <H> (0.4). A bare env reads the variable named after the field, and a Vec can repeat default (0.4). env_fallback reads more variables after env (0.5). {% asciinema m7BqhYuh4EoT93Ts %} Subcommands Most of it came with the effort to support completely all I need for em. With default_subcommand_flags, a flag the root doesn't take selects the default subcommand, so $ em -pu @world Maps automatically to: $ em emerge -pu @world # (0.2.1) default_subcommand_on_empty runs it for a line with no command at all (0.3.1). A subcommand can be a variant with named fields (0.3), and a flattened struct can hold global flags (0.2.1). #[arg(run)] on an enum of subcommands generates the dispatch to the command it holds (0.2.1). {% asciinema zX58u3EIiweaHIsG %} multicall makes one binary act as each of its subcommands when it is run under that subcommand's name, the way busybox's applets work (0.4). Help and errors use the name it was run as. #[arg(external)] on a variant takes any word that names no other subcommand, plus the rest of the line, the way cargo foo runs cargo-foo. {% asciinema 9l8opO2IjQlbyAeg %} Help winnow-args starting point was to get the best possible performance, but having nice help does not impair this need so I tried to make it as expressive as possible. Help has examples, a paragraph under a heading, and a flag's warning (0.3). A subcommand's help lists the global flags it inherits, and the default subcommand is marked, with its page after the root's. *emphasis* and **strong** spans are painted (0.2), and an example's line is painted word by word (0.3.2). 0.4 added verbatim_doc_comment, widths in terminal columns under the unicode feature, visible_alias, long_version, before_help and usage. {% asciinema ZtAsdHDCFnoDLoxi %} Help can say what each choice means (0.5). A ValueEnum variant's doc comment, or choices("fast" = "…") on a field, is listed one value a line by --help, while -h keeps the short [possible values: …]. A hand-written FromArg can answer from data only the running program has; brush uses this for the complete -A and -o choices, whose meanings live in brush-core. 0.5 also added hide_choices, help = CONSTANT, and brief_help, which makes --help print the short page. {% asciinema 11ICmBww1mcmZRP7 %} Deprecation A flag, a positional, a subcommand or an environment variable can be marked deprecated (0.5). Using one prints a warning. deprecated_warn_at keeps the warning quiet until a later version, and deprecated_remove_at says when it goes away. deprecated_env covers a variable that has been renamed. Args::parse prints the warnings; Args::warnings returns them to a program that wants to report them itself. {% asciinema UQn8DPDBsxa4VYlb %} Completion and pages A value is completed by a hint, hint = "dir", or by the program itself, complete = function (0.2). The completion script asks the program, so it always matches what the program accepts. {% asciinema 9FsUFuUiJzxWkj9q %} With WINNOW_ARGS_SPEC set, the derives write a TOML description of each type at compile time (0.2). winnow-args-man and winnow-args-markdown turn those descriptions into pages. The program is only checked, never run, so the pages can be made for a target that can't run on the build host, this addresses one of the papercut that comes from generating man and completion from the cli. Since then the pages have picked up examples, headings, effects, outputs and exit codes (0.3), a page for each multicall command, ls.1 rather than box-ls.1 (0.4), and deprecations (0.5). {% asciinema dP5e6FfvRGlxAidT %} Coming next I'll keep improving the crate aiming to reach feature-parity (see the checklist tracking the features), you are welcome to use it, find bugs and send patches as usual.

winnow-args: yet another args parser winnow-args is a command line parser for Rust, built on winnow. Why another one There are plenty of great crates already, why adding one more given that you have clap, bpaf and usage? All of them are quite good for different purposes: - clap is pretty much the standard, quite user-friendly, but bit slow and not as flexible as you want once you need to parse something more than the usual --long, -l options. - bpaf is way more flexible, but not built for speed. - usage-rs has a very interesting approach to get the best performance while having as many features as clap. Sadly the 3 do not have at the same time the flexibility and speed I need. Speed on the cli? In brush, every builtin is a small command line: set -eu, declare -x NAME=value, printf -v var '%s' x. A script runs thousands of them, so the parser runs thousands of times in one process, and a parser that builds its description of the command at each call is paid for at each call. Builtins also have to support patterns that are not so common, e.g: set +x turns off what set -x turns on. I'm using brush extensively in my portage rewrite to parse ebuilds, anything that gets me a bit of speed is welcome. How can a cli be annoyingly irregular? One of the most annoying and yet widely used cli syntax is the one used by the GNU linker and thus supported by lld, mold and wild. Since I started looking at supporting more special pattern I decided to tackle also that. Sadly the mold author prefers the current bespoke solution at least so far. Here few examples on how irregular it can be: ld -shared -o out.so --as-needed -lc a.o -z now - long options with one dash: -shared - short options with arguments: -o and -z - flags whose order matters: --as-needed applies to the libraries after it. And it can get even more complex. winnow-args Under the hood it leverages winnow zero-copy combinators over a slice of &OsStr, that maps neatly with the native BStr provided. To make it easier to use a derive is provided in a fashion similar to the other crates. - A struct is a command line. - A field in the struct is a flag or a positional, its doc comment is its help. - Enums can be used to express possible values or subcommands. Here some examples: mini-mise This example uses subcommands, aliases, dynamic custom values, examples, runtime completion support and colored help. use winnow_args::color::{Color, Paint, Palette, Theme}; use winnow_args::complete::{Candidate, Shell}; use winnow_args::{Args, Subcommand, ValueEnum, report_with}; /// Install and run development tools #[derive(Args, Debug)] #[arg( name = "tool", version, default_subcommand = "use", example("tool use -g node@22", header = "Install a tool for every project"), example = "tool ls --json" )] struct Cli { /// Show more (repeat for more) #[arg(short, long, count, global)] verbose: u8, /// When to use color #[arg(long, global, default = "auto")] color: When, #[arg(subcommand)] command: Command, } #[derive(ValueEnum, Debug, Clone, Copy)] enum When { Auto, Always, Never, } #[derive(Subcommand, Debug)] enum Command { /// Install a tool and add it to the project #[arg(alias = "u", example = "tool use node@22 python@3.13")] Use { /// Use the global config instead of the project's #[arg(short, long)] global: bool, /// How many downloads run at once #[arg(short, long, default = "4", env = "TOOL_JOBS")] jobs: u32, /// What to install, as `name@version` #[arg(positional, required, complete = tools)] tools: Vec<String>, }, /// List installed tools Ls { /// Print JSON #[arg(long)] json: bool, }, /// Remove a tool #[arg(alias = "rm")] Remove { /// The tool to remove, as `name@version` #[arg(positional, complete = tools)] tool: String, /// Remove it even if a project uses it #[arg(short, long, warning = "Projects that pin it will stop working.")] force: bool, }, /// Print a completion script Completion { #[arg(positional)] shell: Shell, }, } /// What a shell offers for a tool, given what is typed of it. fn tools(typed: &str) -> Vec<Candidate> { ["node@20", "node@22", "python@3.13", "ruby@3.4", "zig@0.15"] .into_iter() .filter(|tool| tool.starts_with(typed)) .map(|tool| Candidate { value: tool.to_owned(), help: "", }) .collect() } fn main() { let args: Vec<_> = std::env::args_os().skip(1).collect(); if let Some(answer) = Cli::completion_request(&args) { print!("{answer}"); return; } let cli = match Cli::parse_from(&args) { Ok(cli) => cli, // Help and version end up here too: `report_with` prints and says how to exit. The theme() implementation is further below. Err(error) => std::process::exit(report_with(&error, "tool", &theme())), }; if let Command::Completion { shell } = cli.command { print!("{}", Cli::completion_script(shell)); return; } println!("{cli:#?}"); } {% asciinema fX6BHOh0thPZOtfz %} Help, from the doc comments -h is the summary and --help the whole text. A subcommand's page lists the global flags it inherits, a flag can carry a warning, and the examples are declared, not pasted into a string. {% asciinema 64W4a36iU5TjZGbi %} Palettes and themes Help and errors are painted from a palette, and the terminal says how deep it goes: no color in a pipe or under NO_COLOR, the 16 basic colors, 256, or 24 bits. A theme is a palette for each depth. The default one, on a 16-color and on a 256-color terminal: {% asciinema qYYB6bMBaGckIi5q %} {% asciinema rkklH2W2GNBb2OKX %} With NO_COLOR=1, or in a pipe: {% asciinema MG0siWLqlYesGrhX %} A program brings its own with a few lines. Here one palette for 24-bit terminals, and clap's look for people who are used to it: /// The paints of help and errors: `TOOL_THEME` picks one. fn theme() -> Theme { const fn rgb(r: u8, g: u8, b: u8) -> Paint { Paint::fg(Color::Rgb(r, g, b)) } // On a 24-bit terminal only; the others keep the default palettes. const SUNSET: Palette = Palette { header: rgb(255, 158, 100).bold(), flag: rgb(224, 175, 104).bold(), command: rgb(224, 175, 104).bold(), placeholder: rgb(187, 154, 247), default: rgb(247, 118, 142), choice: rgb(247, 118, 142), ..Palette::DEFAULT_256 }; match std::env::var("TOOL_THEME").as_deref() { // clap's own look: bold and underline, color only in errors. Ok("clap") => Theme { ansi16: Palette::CLAP, ansi256: None, truecolor: None, }, Ok("sunset") => Theme { truecolor: Some(SUNSET), ..Theme::DEFAULT }, _ => Theme::DEFAULT, } } {% asciinema hAdFLlsaDPqNIwC1 %} {% asciinema oixa3Esawlmvhmki %} Extended error reporting {% asciinema xF1aaEceau970LHW %} Completion that asks the program tool completion fish prints a short script that calls the program back, so the shell always offers what this build of the program takes. Flags come with their descriptions, a value with its choices, and complete = tools above hands the question to a function of yours. {% asciinema AHYvMV3tZ9w5FImy %} bash, zsh, fish, elvish and PowerShell are supported. Exotic syntaxes {% asciinema usoXSDtvhoGQK7YW %} - -eu +x is three fields of type Option<bool>, each declared #[arg(short = 'e', plus = 'e')]. The linker's flags are an enum, and the struct holds them in the order they were given. Under the hood !Image description The derive produces the following: - the parser - the help as static data - optionally a toml description. At run time the words go through a small lexer, the generated parser binds each to its field, FromArg turns a word into a value, and a last step applies the environment, the defaults and the rules between fields. Help, the tips in errors and completion all read the same static data, and only when needed. The toml descriptions are needed by the documentation builders. They read the files the derive writes on request and they do not depend on running the code generated for the target architecture. Key features - The parser is generated at compile time. The derive writes a match over the flags of your struct. Nothing is assebled when the program starts, so a program with two hundred subcommands starts parsing as fast as one with two. - Words are borrowed. Nothing is copied or re-split, and the bytes need not be UTF-8 until a field's type asks for text. - Mistakes in the declaration are compile errors. A conflicts("nme") that names no field, two flags with the same letter, a positional after a list that takes every word: the build fails. - Help is data. The prose sits in read-only memory until --help asks for it, and a feature leaves it out of the binary altogether if it is not needed. Everything is a winnow parser, so you can build your own by combining the basic parsers together manually. Performance winnow-args is built essentially for speed first, comfort as close second. The repository has two benchmarks used to make sure adding features won't cause slow downs. It compares with the other argument parsers I normally use: | | a small CLI | mise's 211 commands | |---|---:|---:| | winnow-args | 187 ns | 327 ns | | usage 7 | 516 ns | 767 ns | | clap 4 | 15.0 µs | 757 µs | | bpaf 0.9 | 14.8 µs | 2.65 ms | !Image description The same command lines are written once per framework in the repository's benchmarks/. A program that parses once will not feel the difference with clap on a small command line. A shell does, and so does a tool with as many subcommands as mise. Implemented features A good deal of features are already implemented, but some found on more mature parsers are still missing. | | winnow-args | usage | clap | |---|:-:|:-:|:-:| | Flags, values, positionals, subcommands, global flags, flatten | ✓ | ✓ | ✓ | | Conflicts, requirements, groups | ✓ | ✓ | ✓ | | Help from doc comments | ✓ | ✓ | ✓ | | Completion scripts | ✓ | ✓ | ~ | | Parser generated at compile time, words borrowed | ✓ | ✓ | – | | Combinators, without a macro | ✓ | – | ~ | | +x options, as a shell builtin takes | ✓ | – | – | | Long options with one dash, -z keywords, as GNU ld takes | ✓ | – | – | | Flags kept in command-line order | ✓ | ~ | ~ | | Flag constraints checked at compile time | ✓ | – | – | | Help prose left out of the binary by a feature | ✓ | ~ | – | | A CLI built at runtime, external subcommands | – | ✓ | ✓ | | Conditional requirements (required_if) | – | ✓ | ✓ | | Man pages, markdown | ~ | ✓ | ~ | | "Did you mean" suggestions | ✓ | ✓ | ✓ | winnow-args-man and winnow-args-markdown render one page per command. They do not run the program to build it: - when WINNOW_ARGS_SPEC set, the derives write a description of each type while cargo check or cargo build expands them. - the pages are built from those files. So man pages can be built correctly even when cross compiling. In closing With version 0.3.0 winnow-args graduated from being a quick experiment to see how far I could push the parsing performance and now it can be used to deal with the quirks of complex CLIs. Some features that I didn't need to support brush and portage-cli em are still missing and I have plans to add a more flexible generator based on minijinja, but I welcome you to try it and possibly provide feedback and pull requests to improve it further.