Adding tests for Kimi-K2.6 and GLM-5.2-FP8#252
Open
amd-droy wants to merge 189 commits into
Open
Conversation
…n, lib refactor map
* feat(dtni): vllm_single PoC — typed configs + orch-driven VllmJob
Replace 4 byte-similar vllm_single wrappers with a single parametrized
suite, per-variant config + threshold dirs, a typed pydantic loader, and
a new orch-driven VllmJob whose container lifecycle is owned entirely by
ContainerOrchestrator (launch:true).
- cvs/lib/dtni/{verdict,config_loader}.py — 5 threshold kinds; pydantic v2
models with extra=forbid; 3-pass placeholder substitution; model.remote=1
raises NotImplementedError pointing at v1 resource_resolver.
- cvs/lib/inference/vllm_orch.py — standalone VllmJob driven by orch.exec.
Drops dead self.port_no distributed branch, random_range_ration typo,
globals.error_list indirection, silent-skip in verify_inference_results.
- cvs/tests/inference/vllm/{conftest,_shared,vllm_single}.py — orch fixture
owns container lifetime; test_print_results_table moved to _shared.
- cvs/input/dtni/vllm_single/{4 variants}/{config,threshold}.json — all
variants pinned to rocm/vllm-dev:nightly for the PoC; thresholds carry
MI300X-realistic floors (~1/3 of MI355X totals) for the verification node.
- cvs/input/cluster_file/mi300x_g21u37.json — single-node MI300X cluster
for verification on 10.245.135.13.
- Delete the 4 old per-model wrappers and mi355x_vllm_single.json.
Verification (offline gates): pytest --collect-only enumerates 9 parametric
cells + test_print_results_table; cvs list vllm_single discovers both test
functions; load_variant of a missing-models-dir variant resolves the
expected /models/{id} path; pydantic ValidationError fires on a
percentile_metrics typo. On-hardware verification is deferred: target node
lacks pre-fetched models, HF token, and benchmark server scripts.
Legacy cvs/lib/inference/{base,vllm,inference_max}.py untouched; other
suites (sglang, inferencemax, pytorch_xdit, megatron, jax) unaffected.
* fix(dtni): cluster file uses devbox-correct key path
The devbox /data/atnair is /data/atnair (not /home/atnair), and the node
10.245.135.13 authenticates with id_ed25519 (not id_rsa). Update the
verification cluster file so the orch fixture authenticates first try.
Confirmed via a lifecycle smoke that brought up an alpine container on
the node and tore it down at the right boundaries.
* fix(dtni): remediate vllm_single lifecycle review findings
- is_ready: grep in-container instead of cat-ing the whole server log
- thresholds: fail at load if sweep cells lack a threshold entry; hard-error
(not silent skip) on per-cell verdict miss
- HTML report: per-test timing rows with explicit units (no cross-row leak)
- client failure: treat only a nonzero failed-request count as failure
- pin HF cache to the mounted models dir; shlex-quote shell interpolation
- raise server readiness budget to 60min for remote model pulls
- remove legacy cvs.lib.inference.vllm.VllmJob (no remaining importers)
* refactor(dtni): collapse vllm_single to single W1 config + generic cluster file
- Replace 4 model variants under input/dtni/vllm_single/ with one W1 config
(Llama 3.1 70B FP8-KV, TP=8) at input/config_file/inference/vllm_single/.
- Rename cluster file to mi300x_vllm_single.json with <changeme> placeholders
so it is generic/shareable rather than node-specific.
- config_loader: add enforce_thresholds gate (record-only scaffolds),
glob threshold sibling, drop enumerate_variants, note generalization seam.
- Move pytest_generate_tests into vllm_single test module; drop aa/ab/zz
lifecycle-ordering prefixes from test names.
- Drop unused imports / apply ruff formatting across touched lib + test files.
* fix(dtni): address vllm_single PR review findings
- config_loader: drop dead BenchmarkParams class + unused benchmark_params
field (and the matching key in the w1 config)
- config_loader: raise FileNotFoundError/ValueError instead of AssertionError
in load_variant (AssertionError is stripped under python -O)
- config_loader: collapse _resolve_cluster_mapping; clarify the container
runtime docstring and the intentional model_validator ordering
- vllm_orch: build the bench client command as a shlex.quote-d arg list so a
model id or path containing a space or $ cannot break the inner bash layer
* fix(dtni): address second-round vllm_single review
- conftest: scope inf_res_dict per-module to match sibling fixtures and
avoid cross-module result-table bleed
- test_model_fetch: split the offline/pre-staged path from the download
poll loop; presence-check with retries so a slow mount that reads 0 on
the first du does not false-fail a model that is present
- start_server: shlex.quote scripts_dir/server_script/server_log, matching
the per-path quoting used elsewhere in the file
- test_teardown: set lifecycle.torn_down only after verifying the container
is gone so the orch finalizer retries an incomplete teardown
- _clone_bench_serving: document why the bench_serving URL is a hardcoded
calibration fork (not stock vLLM, unpinned HEAD), kept for legacy parity
…#227) * feat(dtni): move vllm bench client to stock vllm bench serve Drop the kimbochen/bench_serving fork clone in VllmJob.run_client and invoke the in-image stock "vllm bench serve" CLI instead. The fork was cloned at unpinned HEAD; pinning the client to the run image tag gives Spec 1 a stable artifact contract to parse. - run_client: remove _clone_bench_serving call; head tokens become vllm/bench/serve; drop the now-dead cd /app in client_cmd - remove _clone_bench_serving and its (false) calibration comment - drop Params.bench_serv_script (extra=forbid) and the matching key in the vllm_single config; rename client_log to client.log base.py / inferencemax still use the fork (separate workload, untouched). Source-only change; no metrics added (Spec 1). enforce_thresholds=false. * fix(dtni): robust client completion + launch-failure detection for stock bench Harden wait_client_complete after the move to stock vllm bench serve: - COMPLETION_RE: key off the unconditional "Serving Benchmark Result" banner instead of the "End-to-end Latency" metric header. Stock prints metric headers only when the metric is in --percentile-metrics, so a config omitting e2el would never be detected as complete and would spin to the poll cap (~90 min) on an otherwise-successful run. - add CLIENT_LAUNCH_FAIL_RE: a CLI launch failure (bad/renamed flag, missing bench subcommand, vllm not on PATH) exits before any summary and is neither a Python traceback nor a Failed-requests line, so it too would hang the poll cap. Treat it as a hard failure, like a crash. - drop dead export RESULT_FILENAME=results: consumed only by the removed fork client; stock takes the name via --result-filename.
Hnimrama/inferencemax uplift Refactors InferenceMax for the DTNI pytest layout (inferencemax_single): ContainerOrchestrator-based conftest, suite/threshold JSON loading, benchmark model selection, and tighter server/client lifecycle handling against current InferenceX upstream. Benchmarking: stop cloning third-party bench_serving; resolve benchmark_serving.py from the installed vllm package (BENCH_SCRIPT) for InferenceMax and vLLM single paths. Host-mounted server entrypoints live under cvs.lib.dtni.vllm_benchmark_scripts (vllm_serve_mi300x.sh); samples and docs use <changeme> container placeholders, legacy benchmark_script_repo called out as ignored, and volume_dict guidance avoids duplicate Docker :/workspace mounts. vLLM single (vllm_orch): align with dev/dtni completion and client-failure detection while keeping python3 "$BENCH_SCRIPT" invocation. Misc: optional run_plugin --log-file; sglang_disagg total_generated_tokens key; log redaction and small review fixes from PR feedback. Test with cvs run inferencemax_single (cluster + suite JSON, HF token) and spot-check vllm_single if configs touch shared modules.
ContainerOrchestrator.setup_sshd() ran a fixed command list ending in `/usr/sbin/sshd -p2224` and asserted every step succeeded, for every container run regardless of node count. The orch pytest fixture calls it unconditionally, so a single-node run on a minimal image with no /usr/sbin/sshd failed the whole fixture with a generic "SSH setup command failed" message and never ran the workload. The in-container sshd exists only so MPI (mpirun's plm_rsh_args -p 2224) can reach peer ranks on other nodes. A single-node run execs directly via docker exec and never distributes over MPI, so the sshd setup is dead weight there. Guard setup_sshd() to return True early when len(self.hosts) <= 1, after the container_id precondition. The host count lives on the orchestrator, so the decision belongs there; multinode runs are unchanged.
…llm result path (#233) * refactor(lib): rename dtni -> utils and split out generic config machinery Rename cvs/lib/dtni to cvs/lib/utils ("utils" says what it is: pure functions any lib can call; "dtni" was a leftover project codename). verdict.py moves unchanged. config_loader.py is trimmed to the framework-agnostic half: the paths/model/image/container schema, the 3-pass placeholder substitution, the enforce_thresholds gate on a new BaseVariantConfig, and a substitute_config() helper (file read + substitution + sibling-threshold discovery). The inference-only schema moves to a sibling module in a later commit. * refactor(inference): move vllm_parsing into cvs/lib/inference/utils The client.* metric parser (to_client_metrics + CLIENT_METRICS surface) is inference-specific and should not sit in the shared utils dir. Move it under a new cvs/lib/inference/utils package. Content unchanged. * feat(inference): inference config schema with named-combo sweep selector The inference half of the old config_loader: GoodputSlo, SeqCombo, Sweep, Params, Roles, and VariantConfig(BaseVariantConfig) with cell_key and the threshold-coverage check. load_variant() delegates the file read and placeholder substitution to the generic substitute_config(). Replaces the sequence_combinations x concurrency_levels cartesian with a named-combo + explicit runs[] selector: each run is a {combo, concurrency} pair, so the config enumerates exactly the cells to run (no NxM explosion). A model_validator rejects duplicate combo names and runs referencing an unknown combo at load time. * feat(inference): self-contained server cmd + artifact-based result parsing build_server_cmd/start_server assemble a `vllm serve` arg list in Python (mirroring run_client) instead of cloning and running an external .sh, so a run needs no hand-staged script. --max-model-len is derived per cell from isl/osl/random_range_ratio so any sweep change stays self-consistent. parse_results reads the stock extensionless `results` JSON artifact that `vllm bench serve` writes to --result-dir and delegates namespacing + derived-metric math to vllm_parsing.to_client_metrics, replacing the brittle console-log regex table. Missing/empty/unparseable artifacts hard-fail the cell rather than recording a silently-green empty row. Adds the optional --goodput SLO gate (per-cell, omitted when no SLO) and threads goodput_slo through run(). * feat(suite): per-metric result rows, Value/Unit columns, sweep selector pytest_generate_tests now drives parametrization from the named-combo + runs[] selector instead of the cartesian. test_vllm_inference only runs the benchmark and stashes results; the verdict moves into a new test_metric (one pytest test = one HTML row per metric per cell), with inline Value/Unit columns added via pytest_html hooks in conftest. test_setup_sshd gates its 2224 probe on len(orch.hosts) > 1, mirroring the single-node orchestrator guard: single-node runs skip the in-container sshd (it exists only for inter-node MPI) and must not probe for it. Import paths follow the dtni -> utils / inference.utils moves. * chore(config): rename vllm_single config pair, adopt selector, drop cluster file Rename the config/threshold pair to {model}_{precision}_{config|threshold} .json and convert the sweep to the named-combo + runs[] selector. Pin the image to rocm/vllm-dev:nightly (the previously pinned :nightly-sshd tag does not exist on Docker Hub, and single-node runs skip in-container sshd). Delete cvs/input/cluster_file/mi300x_vllm_single.json: a cluster file only needs node IP + user/key/orchestrator; the variant config supplies the container block, so the bespoke per-suite cluster file is redundant. * test(inference): unit tests for parser, sweep selector, and verdict guards Cover to_client_metrics purity + derived metrics, the named-combo/runs sweep selector (expansion, unknown-combo and duplicate-name rejection), the run_client goodput/metric-percentiles flags, table-cell rendering, and the verdict None-guards. Adds JSON fixtures for the stock results artifact. * docs: suite-authoring guide + AGENTS.md for shared and inference helpers Add a human reference guide (plans/building-a-cvs-test-suite.md) that walks the six-layer suite architecture using vllm_single as the worked example: the generic <-> framework config seam, the named-combo + runs[] sweep selector, the self-contained Python-built server cmd, lifecycle-as-tests, and a checklist for authoring a new inference or training suite. Add per-package AGENTS.md docs naming the public entry points, the seam, and the non-obvious gotchas: - cvs/lib/utils: substitute_config / BaseVariantConfig / evaluate_all, the 3-pass placeholder order, sibling-glob threshold discovery, parent-first validator ordering. - cvs/lib/inference/utils: load_variant / to_client_metrics / CLIENT_METRICS, the cell_key single-source-of-truth, the coverage check that prevents a silent green, and the validators mirrored in pytest_generate_tests. * docs: expand suite guide with lib restructure, drop redundant section rules Document the dtni -> utils rename and the shared (cvs/lib/utils) vs domain-specific (cvs/lib/inference/utils) split: what lives where, the rule for placing a new helper, and the directory map. Note training is not yet ported and this guide is the blueprint for that port. Remove the manual --- horizontal rules between sections: heading levels already render their own bottom border, so the extra rules produced a double-underline. Minor prose/format cleanups. * docs: demote headings so GitHub stops underlining sections * removing old plan * fix(config): re-key W1 threshold to the swept CONC=16 cell The threshold file carried placeholder CONC=64/128/256 entries while the sweep's only run is concurrency 16, so cell_key() matched no threshold. The mismatch was masked by enforce_thresholds=false (warned, not raised) and would have failed load the instant enforcement was flipped on. Re-key to the single CONC=16 cell the runs selector actually enumerates. * fix(inference): drop dead server-env exports from build_server_cmd MODEL/ISL/OSL/MAX_MODEL_LEN/RANDOM_RANGE_RATIO/TP/CONC/PORT were exported into /tmp/server_env_script.sh but read by nothing after the .sh->Python server command refactor -- both _server_argv and run_client pass these as explicit flags. Keep only the env the vllm process actually consumes (HF token, HF cache pin, AITER flags). Also drops the second _derive_max_model_len call that fed the dead MAX_MODEL_LEN export. * fix(inference): move --kv-cache-dtype out of the driver into config _server_argv hard-coded --kv-cache-dtype fp8, baking a per-model property into the shared orchestrator -- a non-fp8-KV model dropped into the suite would be served wrong with no config recourse (extra_serve_args can only add, so an override would pass the flag twice). Declare it in the W1 config's roles.server.extra_serve_args instead; the driver stays model-agnostic and 'new model = new config' holds. * refactor(inference): share the sweep-selector validator across load and collection pytest_generate_tests hand-reimplemented the duplicate-name and unknown-run.combo checks that Sweep._check_runs_reference_known_combos already enforces, with divergent semantics (first-failure raise vs all-at-once). Extract validate_sweep_selector() as the single home and call it from both the typed validator (load time) and the collection-time raw-JSON path so the rule can't drift. * fix(inference): key the per-cell out_dir by isl/osl/conc out_dir was fixed per job, so a multi-cell sweep would overwrite each cell's `results` and client.log, and parse_results could cat a prior cell's stale artifact if the current cell's client failed to write one. Key it by cell. Latent today (the shipped sweep has one cell). * refactor(inference): normalize goodput_slo to dict-only run_client accepted goodput_slo as either a dict (.get) or an object (getattr) via a per-key hasattr branch, but the only production caller passes a raw dict -- the object path existed solely for a unit test, and the dual path meant the typed GoodputSlo's validation never reached the command builder. Consume the dict only and drop the object-form test. * test(inference): drop unused _fake_variant parameter goodput_slo_unused was never read (goodput is threaded through _make_job). * chore(inference): placeholder personal/image refs in example config The committed vllm_single example config carried a personal hf-token path and a concrete image tag (duplicated in image.tag and container.image). Replace all three with <changeme> so the file is a template a new user must fill in -- it still loads (collection works) and only fails at run time when an unedited value is read, which is the intended signal. * refactor(inference): server serve_args as a {flag: value} map The per-model server knobs were a flat [flag, value, flag, value] list (roles.server.extra_serve_args), which reads poorly. Replace with a roles.server.serve_args {flag: value} map (flag without the leading --): a scalar renders --flag value, True a bare --flag, a list the flag once per element -- so it stays readable while still covering vllm bare/repeatable flags. _server_argv flattens the map via a new _flatten_serve_args helper; the derived flags (tp/max-model-len/port) stay code-built. Also repoints a stale unit test that asserted MAX_MODEL_LEN in the env script (it moved to the --max-model-len flag in an earlier commit) to assert against the server argv instead. * feat(verdict): add unit-agnostic max threshold kind The only ceiling kind was max_ms, whose message hard-codes ms. A count metric like client.failed needs an upper bound without the unit lie; add a plain max with the same comparison and an honest message. * feat(inference): enforce a declared gated-metric SLO contract Previously only cell-presence was validated: a cell could exist while a given metric had no spec, and test_metric (spec is None -> return) would silently report a green record-only row even under enforce_thresholds=true. A new perf metric was thus unvalidated by default. Declare GATED_METRICS beside CLIENT_METRICS -- the perf+health subset that must assert (throughput, mean+p99 latency, success_rate/failed) -- and extend _check_thresholds_cover_sweep to require a spec for every gated metric in every present cell, reusing the same enforce-vs-warn path. A new metric is record-only until added to the set; once gated, the loader forces a spec in every cell before the suite can run green. Inputs, totals, and derived diagnostics stay record-only by design. * fix(inference): single image source on container.image The image was declared twice: top-level image.tag (live -- conftest copied it onto the container block) and container.image (dead -- overwritten by that copy). The duplicate forced a top-level image block whose remote field was unused and whose tag silently shadowed container.image. Make container.image the single source: drop the top-level ImageSpec block from the generic BaseVariantConfig, drop the conftest overwrite so the merged container.image is used as-is, and remove the now-schemaless image block from the example config. * feat(inference): gate the full latency distribution Expand GATED_METRICS from the mean+p99 subset to every emitted latency quantile (mean/median/p90/p95/p99) for ttft, tpot, itl, and e2el -- itl omits p90 as CLIENT_METRICS has no producer for it. Throughput and success_rate/failed health are unchanged. Inputs, totals, secondary throughputs, and derived diagnostics stay record-only. The example threshold file gains a placeholder spec for each newly gated metric (23 total) so the loader gated-coverage check passes. * docs: reflect image collapse, serve_args rename, max kind, GATED_METRICS - utils/AGENTS.md: drop top-level image (now container.image); add max verdict kind - inference/utils/AGENTS.md: document GATED_METRICS contract + dual-axis coverage check - building-a-cvs-test-suite.md: container.image, serve_args map, max kind, GATED_METRICS - dtni-dev-guide.md: SUPERSEDED banner pointing to building guide + AGENTS.md * refactor(inference): rename vllm_orch → vllm_single throughout The module name vllm_orch.py implied disaggregated orchestration; this is a single-node suite. Align the module name with the suite file and suite name. Update all import sites, AGENTS.md prose, and the plan doc. * fix(config): drop ModelSpec.precision, make threshold_json an explicit field ModelSpec.precision was an unvalidated free-text field with no downstream use; the kv-cache-dtype flag belongs in serve_args. Removing it prevents configs silently carrying a stale or misleading label. Replace the sibling-glob threshold discovery (glob('*threshold.json') next to the config) with an explicit threshold_json field on BaseVariantConfig. The glob was fragile: ambiguous when multiple threshold files coexist, and invisible in the config spec. An explicit absolute path is transparent, repo-portable, and validated as part of the schema. Update the example config to add threshold_json: "<changeme>" and drop model.precision. * test(config): unit tests for ModelSpec, BaseVariantConfig, substitute_config Cover the contracts changed by the precision-removal and threshold_json-explicit commits: ModelSpec forbids precision and extra keys; BaseVariantConfig requires threshold_json; substitute_config reads the threshold via raw['threshold_json'] as a literal absolute path, not by globbing the sibling directory; a sibling *threshold.json must NOT be auto-discovered (regression guard for old behavior); comment keys are stripped. No hardware; pure filesystem via tempfile. * refactor(inference): O(n) duplicate detection in validate_sweep_selector Replace list.count() inside the loop (O(n²)) with Counter: one pass to build the frequency map, one comprehension to collect duplicates. Suggested in review. * test(inference): address review — setUpClass, if __name__ at end of file Convert per-test module loads to setUpClass so each heavy import runs once per class, not once per test method: TestTableCellRendering._cell() (loads _shared.py + stubs tabulate), TestKeyConsistency._producer_keys() (runs parse_results), TestMetricTests.setUp() (_load_vllm_single). Suggested in review for TestTableCellRendering; applied the same fix to the other two classes that had the identical problem. Move `if __name__ == "__main__": unittest.main()` from mid-file (line 282) to the very end. Test classes defined after the guard were still discovered by both pytest and python -m unittest (Python parses the full file first), but the placement looked like dead code. Flagged in review. * test(inference): address review + expand coverage for config loader models Review items: remove dead `vc = _variant(sw)` assignment that was never read; move `import warnings` from inline to top-level imports. New test classes covering contracts changed in this PR: TestModelSpecNoPrecision (extra field rejected), TestThresholdJsonField (required field, constructs with it), TestCellCoverageAxis (missing cell and extra threshold-key axes, warn/raise modes), TestExpectedCellsBoundaries (empty runs, unreferenced combo), TestGoodputSlo (construction, missing fields, forbid extra keys, optional on SeqCombo), TestSeqComboForbid (required fields, extra keys). All no-hardware. * feat(config): expand w1 llama31_70b_fp8kv sweep to 5 ISL/OSL cells at conc=16 Replace the single ISL=128/OSL=2048 placeholder cell with the full ISL/OSL matrix requested in review: ISL=1024 / OSL=1024 ISL=8192 / OSL=1024 ISL=1K / OSL=8192 ISL=1K / OSL=4096 ISL=5000 / OSL=1024 All five cells run at concurrency=16, TP=8, with record-only placeholder thresholds (enforce_thresholds=false). threshold.json carries a spec for every GATED_METRICS member per cell so the loader coverage check passes. * fix(config): use literal ISL/OSL values (1000/8000/4000 not 1024/8192/4096) * style(inference): collapse multi-line string concat in launch cmd Co-Authored-By: Claude <noreply@anthropic.com> * adding documentation * renaming config and threshold files --------- Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
Signed-off-by: amd-droy <droy@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cvs/lib/inference/sglang/sglang_disagg_orchestrator.py - Added this file for orchestrating the runs.
Added the threshold files for llama-70b, deepseek-ri-0528, Kimi-2.6-w4A8, GLM-5.2-FP8 and gpt-odd-120b for the concurrency and ISL/OSL sweeps.
Added the long Context accuracy tests for ISL - 96K 128K 200K
Removed the Run deck reports viewer for now. Will integrate it once its fully updated.
Attaching the reports generated for reference.
reports.zip