cli.md

Command-line reference

This page documents the user-facing arguments parsed by the current native executables. Build paths follow the build guide.

voxtral

voxtral transcribes one WAV file and exits.

Usage

voxtral --model path.gguf --audio file.wav [options]

Show the installed build's help:

./build-vulkan/voxtral --help

The input must be a RIFF/WAVE file containing 16-bit integer PCM or 32-bit IEEE float samples. Channels are averaged to mono. The current loader does not resample or reject a mismatched sample-rate header, so the caller must provide audio sampled at 16 kHz.

Arguments

Argument	Value	Default	Description
--model	Path	Required	GGUF model to load. Missing value or model load failure is an error.
--audio	Path	Required	Input WAV file. Missing value or unsupported/unreadable data is an error.
--threads	Text parsed by strtol	0	CPU thread request. Values less than or equal to zero select the runtime default, which uses hardware concurrency capped at 16.
--seed	Text parsed by strtoul	0	Reserved for sampling. The current CLI accepts and stores the converted value but does not apply it.
--prompt	Text	Empty	Compatibility argument. The current CLI accepts the value but does not apply it.
--n-tokens	Text parsed by strtol	0	Maximum decoded tokens; alias of --max-len. Values less than or equal to zero process the whole file.
--max-len	Text parsed by strtol	0	Maximum decoded tokens. Values less than or equal to zero process the whole file.
--verbose	Flag	Off	Set debug logging. Argument order matters if followed by another --log-level.
--log-level	error, warn, info, or debug	info	Set the exact case-sensitive log threshold.
--dump-logits	Path	Empty	Write up to the first 32 step-zero logits as newline-separated text.
--dump-logits-bin	Path	Empty	Write all step-zero logits as native float32 raw bytes.
--dump-tokens	Path	Empty	Write generated token IDs on one line.
--output-text	Path	Empty	Write decoded text to a file while retaining normal standard output.
--gpu	Backend name	auto	Select auto, vulkan, or none for documented builds. The parser is case-insensitive.
--metal	Flag	Off	Compatibility alias for --gpu metal; Metal is not a validated target in this guide.
-h, --help	Flag	Off	Print usage and exit successfully.

The parser also recognizes cuda and metal as --gpu values. Recognition is not proof that the backend was compiled or is available at runtime.

Not currently validated by this project

CUDA and Metal are not documented build targets for this project. There is no CUDA build command in this documentation. Use --gpu vulkan only with a Vulkan-enabled build and --gpu none for CPU.

The integer helpers require the conversion to consume the complete value, but do not separately check an empty value, overflow, or a sign on the unsigned seed conversion. Avoid those edge cases and pass ordinary in-range decimal text. Unknown options, missing values, rejected conversions, invalid log levels, and unrecognized backend names print an error followed by usage.

Examples

CPU:

./build-cpu/voxtral --model models/voxtral/Q4_K_M.gguf --audio /path/to/audio.wav --gpu none

Vulkan:

./build-vulkan/voxtral --model models/voxtral/Q4_K_M.gguf --audio /path/to/audio.wav --gpu vulkan --threads 8

Limit the decode and save text:

./build-vulkan/voxtral --model models/voxtral/Q4_K_M.gguf --audio /path/to/audio.wav --gpu vulkan --max-len 512 --output-text transcript.txt

Capture diagnostic token and logit output:

./build-vulkan/voxtral --model models/voxtral/Q4_K_M.gguf --audio /path/to/audio.wav --gpu vulkan --dump-tokens tokens.txt --dump-logits logits.txt --dump-logits-bin logits.f32

Output formats

On successful transcription, standard output contains:

1. The decoded transcript, or [no-transcript].
2. A line beginning with [tokens] followed by token IDs, or a second [no-transcript] line when no tokens were generated.

Logs and a final runtime summary are written to standard error. The summary reports elapsed processing time, compile-time backend flags, registered runtime backends, and whether each known runtime backend has a device.

Optional files contain:

• --output-text: the displayed transcript plus a newline.
• --dump-tokens: space-separated decimal token IDs plus a newline.
• --dump-logits: up to 32 decimal float values, one per line.
• --dump-logits-bin: the complete first-step logit vector as native float32 bytes without a header. Consumers must account for host endianness.

Failure to open an optional output file is logged but does not change an otherwise successful inference exit status.

Exit behavior

Exit status	Meaning
0	Transcription completed. --help also exits 0.
1	Argument parsing or required-argument failure.
2	Model loading failed.
3	Context creation failed.
4	WAV loading or transcription failed.

The process prints its runtime summary even after model, context, or transcription failure.

voxtral-server

voxtral-server exposes the HTTP batch and WebSocket realtime APIs documented in Voxtral server. The current engine requires a realtime GGUF and a Vulkan-enabled server build.

Usage

voxtral-server --model PATH [options]

Show help or version:

./build-server/voxtral-server --help

./build-server/voxtral-server --version

Arguments

Argument	Value	Default	Description
--model	Path	VOXTRAL_SERVER_MODEL, otherwise required	Realtime GGUF model. A command-line value overrides the environment.
--listen	Address	127.0.0.1	TCP bind address. It must be non-empty and resolvable.
--port	Integer	8080	TCP port from 1 through 65535. Signs and trailing characters are rejected.
--api-key-file	Path	Unset	Read the Bearer token from a file. The token must be non-empty text, no more than 4096 bytes, without NUL; trailing CR/LF is removed.
--no-auth	Flag	Off	Disable authentication. Allowed directly only for localhost, ::1, or a 127.* bind.
--allow-insecure-no-auth	Flag	Off	Explicitly permit --no-auth on a non-loopback bind. This exposes the API without authentication.
--max-upload-mib	Positive integer	512	Maximum batch request body in MiB.
--realtime-soft-lag-ms	Positive integer	1000	Emit session.warning when realtime backlog reaches this threshold. Must not exceed the hard threshold.
--realtime-hard-lag-ms	Positive integer	5000	End a realtime session when backlog reaches this threshold. Must be at least the soft threshold.
--realtime-buffer-ms	Positive integer	1000	Capacity of the bounded server-side PCM queue, expressed as 16 kHz audio time.
--idle-timeout-sec	Positive integer	60	End a configured realtime session with no client frame activity for this many seconds.
--log-level	error, warn, info, or debug	info	Exact case-sensitive server log threshold.
--help	Flag	Off	Print usage and exit 0 immediately.
--version	Flag	Off	Print server, library, and packed API versions and exit 0 immediately.

All numeric settings except the port accept values from 1 through UINT32_MAX, subject to byte-count overflow checks for the upload limit and sample-capacity checks for the realtime buffer.

Configuration errors exit with status 2. Model/backend startup and unexpected runtime errors exit with status 1. Normal shutdown, --help, and --version exit with status 0.

Environment

Command-line options override their environment equivalents:

Environment variable	Default	Equivalent or purpose
VOXTRAL_SERVER_MODEL	Unset	--model
VOXTRAL_SERVER_LISTEN	127.0.0.1	--listen
VOXTRAL_SERVER_PORT	8080	--port
VOXTRAL_SERVER_API_KEY	Unset	Bearer token supplied directly in the process environment
VOXTRAL_SERVER_API_KEY_FILE	Unset	--api-key-file
VOXTRAL_SERVER_MAX_UPLOAD_MIB	512	--max-upload-mib
VOXTRAL_SERVER_REALTIME_SOFT_LAG_MS	1000	--realtime-soft-lag-ms
VOXTRAL_SERVER_REALTIME_HARD_LAG_MS	5000	--realtime-hard-lag-ms
VOXTRAL_SERVER_REALTIME_BUFFER_MS	1000	--realtime-buffer-ms
VOXTRAL_SERVER_IDLE_TIMEOUT_SEC	60	--idle-timeout-sec
VOXTRAL_SERVER_LOG_LEVEL	info	--log-level

If both key environment variables are set, VOXTRAL_SERVER_API_KEY takes precedence over VOXTRAL_SERVER_API_KEY_FILE. A command-line --api-key-file then replaces the environment key. --no-auth clears both. Authentication command-line options are processed left to right: --api-key-file after --no-auth re-enables authentication, while a later --no-auth disables it.

VOXTRAL_SERVER_REALTIME_RAW_PARTIALS=1 is a diagnostic-only environment override. It disables identical-partial suppression and the normal 200 ms partial-delivery interval. Do not use it as a normal server setting.

Avoid putting keys directly in shell history. A protected key file is the preferred configuration.

Examples

Authenticated loopback server:

VOXTRAL_SERVER_API_KEY_FILE="$HOME/.config/voxtral/api-key" ./build-server/voxtral-server --model models/voxtral/Q4_K_M.gguf --listen 127.0.0.1 --port 8080

Unauthenticated loopback development server:

./build-server/voxtral-server --model models/voxtral/Q4_K_M.gguf --listen 127.0.0.1 --port 8080 --no-auth

Tune bounded realtime limits:

VOXTRAL_SERVER_API_KEY_FILE="$HOME/.config/voxtral/api-key" ./build-server/voxtral-server --model models/voxtral/Q4_K_M.gguf --realtime-buffer-ms 1600 --realtime-soft-lag-ms 1200 --realtime-hard-lag-ms 6000

The server permits one active batch or realtime inference operation per process. It does not queue inference requests.

voxtral-quantize

voxtral-quantize converts eligible tensors in an existing Voxtral GGUF to a selected quantized type. It accepts both voxtral_realtime and voxtral architectures.

Usage

voxtral-quantize model-in.gguf model-out.gguf type [nthreads]

There is no dedicated --help option. Running the following prints positional usage and the supported types to standard error, then exits 1 because the argument count is invalid:

./build-cpu/voxtral-quantize --help

Arguments

Argument	Value	Default	Description
model-in.gguf	Path	Required	Readable input GGUF with architecture voxtral_realtime or voxtral.
model-out.gguf	Path	Required	New output GGUF. The path is opened for binary output and can be truncated.
type	Quantization name	Required	Case-insensitive type or accepted alias from the list below.
nthreads	Text parsed by std::stoi	Hardware concurrency, or 1 if unavailable	Optional worker count. It must begin with an integer; values less than or equal to zero become 1. The current parser does not reject trailing characters after the integer prefix.

Wrong argument count, an unknown type, thread text without an integer prefix, an unreadable input, an unsupported architecture, or a write/quantization failure exits 1. Successful quantization exits 0 and reports tensor progress, sizes, and total time.

Keep the input file until the output has passed application-specific quality and integrity checks.

Supported quantization types

Type	Accepted aliases	Notes
Q2_K	—	K-quant
Q3_K	—	K-quant
Q4_0	—	Legacy 4-bit type
Q4_1	—	Legacy 4-bit type
Q4_K	—	K-quant
Q4_K_M	—	Mixed Q4_K/Q6_K project policy
Q5_0	—	Legacy 5-bit type
Q5_1	—	Legacy 5-bit type
Q5_K	—	K-quant
Q6_K	q6	K-quant
Q8_0	q8	8-bit type

Eligible tensors are quantized. Tensors that are not eligible, already have the target type, or have an incompatible row width are copied in their source type; incompatible widths produce a warning.

Examples

Use the default worker count:

./build-cpu/voxtral-quantize model-f16.gguf model-q4-k-m.gguf Q4_K_M

Use eight workers:

./build-cpu/voxtral-quantize model-f16.gguf model-q8.gguf Q8_0 8

See Models for converted downloads and the Python conversion workflow. Documentation verification does not execute quantization.