[doc] Update stages doc with top-level stages#597
Conversation
|
@marnovandermaas, WDYT? |
martin-velay
force-pushed
the
top_level_stages
branch
from
2d50ddb to
f094b94
Compare
marnovandermaas
left a comment
marnovandermaas
left a comment
There was a problem hiding this comment.
Thank you so much for this PR. I really enjoyed the motivation sections you added. I put some of my thoughts in here, which I hope are useful.
| These stages apply to the `top_chip` integration testbench. | ||
| The two key documents governing chip-level verification are: | ||
|
|
||
| - **Verification plan** (primary): [`hw/top_chip/dv/data/top_mocha_vplan.hjson`](../../hw/top_chip/dv/data/top_mocha_vplan.hjson) - defines the coverage metrics and their mapping to tests. |
There was a problem hiding this comment.
This doesn't exist yet right?
There was a problem hiding this comment.
Not yet, but under writing
| The two key documents governing chip-level verification are: | ||
|
|
||
| - **Verification plan** (primary): [`hw/top_chip/dv/data/top_mocha_vplan.hjson`](../../hw/top_chip/dv/data/top_mocha_vplan.hjson) - defines the coverage metrics and their mapping to tests. | ||
| - **Testplan**: [`hw/top_chip/data/chip_testplan.hjson`](../../hw/top_chip/data/chip_testplan.hjson) - captures individual testpoints and their associated tests. |
There was a problem hiding this comment.
This also doesn't exist yet right?
There was a problem hiding this comment.
Not yet indeed, I am also already working on it
| |-----------|----------|----------------| | ||
| | V0 | Initial Work | <ul> <li> Chip-level testbench being set up </li> <li> Chip-level verification plan being written </li> <li> Chip-level testplan being written </li> </ul> | | ||
| | V1 | Smoke Passing | <ul> <li> All IPs smoke-tested at chip level </li> <li> Testbench infrastructure validated </li> <li> CI smoke regression running </li> </ul> | | ||
| | V2 | Integration Complete | <ul> <li> All planned chip-level tests passing </li> <li> All chip interfaces connected to an active agent and exercised end-to-end </li> <li> End-to-end interrupt routing confirmed for all interrupt-capable IPs </li> <li> Cross-IP integration paths and reset sequences exercised </li> <li> Chip-level coverage targets met </li> </ul> | |
There was a problem hiding this comment.
Can we say that the tests are passing with at least 90% of random seeds?
There was a problem hiding this comment.
Here I wanted to distinguish tests according to their Milestone. If they are mapped as V1 in the testplan then we should have 100% of them to signoff V1. For V2 we should have all V1 and all V2.
| | **Item name** | **Description** | | ||
| |---------------|-----------------| | ||
| | TOP_DV_DOC_DRAFTED | DV document drafted covering testbench architecture, agent topology, firmware-driven stimulus model, and chip-level coverage intent. | | ||
| | TOP_VPLAN_COMPLETED | Verification plan (`top_mocha_vplan.hjson`) complete with the metric-to-test mapping for each coverage item and milestone specified. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | |
There was a problem hiding this comment.
I think completed is too ambitious here. I would like drafted here. That doesn't mean it cannot be complete but I think it might prevent us from doing the sign-off with a vplan that is good enough.
There was a problem hiding this comment.
It is maybe ambitious, but it should be among the first task we complete as the DV is then driven by that. I'd like to keep it as it is more or less. Also, a point to raise, having a vPlan written and reviewed doesn't mean that it shouldn't move at all. Of course in reality we'll have spec updates or we'll notice that we were wrong in the vPlan. Other I could reformulate slightly in that sense, but i feel that if we write "draft" people will assume that they only need to create the file with few titles, IMO it should be more than 90% done and a review with the stakeholders should have been done already. I don't want to end up driving the DV in a wrong direction or miss really major points. Does it sounds OK for you?
| |---------------|-----------------| | ||
| | TOP_DV_DOC_DRAFTED | DV document drafted covering testbench architecture, agent topology, firmware-driven stimulus model, and chip-level coverage intent. | | ||
| | TOP_VPLAN_COMPLETED | Verification plan (`top_mocha_vplan.hjson`) complete with the metric-to-test mapping for each coverage item and milestone specified. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | | ||
| | TOP_TESTPLAN_COMPLETED | Chip-level testplan (`chip_testplan.hjson`) complete with at least one testpoint per integrated IP. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | |
There was a problem hiding this comment.
Same as for vplan.
| | TOP_BOOT_INFRA_PASSING | The SW-to-DV pass/fail signalling mechanism is confirmed working before any other firmware-driven test result is trusted. | | ||
| | TOP_ALL_TESTS_PASSING_V1 | All V1 testpoints in the testplan passing. | | ||
| | TOP_VPLAN_COVERAGE_V1 | All V1 items defined in the verification plan achieved. | | ||
| | TOP_SMOKE_REGRESSION_IN_CI | V1 smoke suite runs automatically on PRs touching top-level RTL or testbench and failures block merge. | |
There was a problem hiding this comment.
Can't we do the same with out current nightly setup?
There was a problem hiding this comment.
You mean without it being part of the CI? I think it'd be good to have UVM top tests running for CI at some point (maybe in a private one). Don't you think so?
Signed-off-by: martin-velay <mvelay@lowrisc.org>
martin-velay
force-pushed
the
top_level_stages
branch
from
f094b94 to
0bf8c92
Compare
2 participants
No description provided.