BUG: refuse to run a Monte Carlo over a results file it cannot write - #1161
Open
thc1006 wants to merge 3 commits into
Open
BUG: refuse to run a Monte Carlo over a results file it cannot write#1161thc1006 wants to merge 3 commits into
thc1006 wants to merge 3 commits into
Conversation
import_outputs() accepts .csv and .json, points output_file at the file, and offers continuing a simulation from it. simulate() only writes JSONL, and __setup_files opens with w+ when append is False, so the imported file was truncated and then filled with records its own extension does not describe. Checked before any file is opened, and named per path so the message says which one has to change. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1161 +/- ##
===========================================
+ Coverage 82.18% 83.55% +1.37%
===========================================
Files 122 130 +8
Lines 16355 17107 +752
===========================================
+ Hits 13441 14294 +853
+ Misses 2914 2813 -101 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The note under import_outputs said any previously saved file could be used to continue a simulation, which is what led a .csv into output_file in the first place. Say which format that holds for. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
import_results points input_file, output_file and error_file at one path, and a run then appends input rows and output rows into it. Compared by inode once the files exist, so a symlink, a hard link, a/../run.txt and a case-insensitive filesystem are all the same file rather than three names. json.dumps kwargs reach the writer, so indent=2 wrote records across several lines while every reader here takes one line at a time. The run finished and the completeness check then called the file it had just written damaged. indent of 0 and "" do the same, as does a newline inside separators. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request type
Checklist
ruff check/ruff format --check) has passed locallypytest tests/unit tests/integration) have passed locallyCurrent behavior
import_outputs()reads.txt,.csvand.json, setsoutput_fileto the path it read, and its own docstring said the imported file could be used "to continue a simulation".simulate()only writes JSONL, and__setup_files()opens the file withw+whenappend=False. Importing a CSV and running again therefore empties it and writes JSON objects into a file still named.csv:Measured on
develop:results.csvgoes from 31 bytes to 0, then fills with JSONL. The extension, the reader that would be picked for it, and the bytes on disk all disagree, and the imported data is gone.append=Trueis the same mismatch without the truncation, appending JSONL rows after a CSV header.New behavior
simulate()checks all three log paths before anything is opened, and refuses anything that is not.txt. The message names which ofinput_file,output_fileorerror_filehas to change, and says CSV and JSON stay importable for analysis. Uppercase.TXTis accepted, since it is the same file to the filesystem.The note under
import_outputsno longer offers any saved file as something to resume from, since that is what led a.csvintooutput_filein the first place.This does not narrow what can be imported. It only stops
simulate()writing over a file it cannot read back.Breaking change
A run that pointed a log at
.csvor.jsonwas destroying or corrupting that file, so nothing was relying on it.Additional information
Six tests. Four drive the real
simulate()through themonte_carlo_calistofixture over{.csv, .json} x {append=False, append=True}and assert both that it raises and thatread_bytes()is unchanged. The others cover the per-path naming and the uppercase suffix.Removing the call from
simulate()turns only the four integration cases red, while the direct check stays green; making the check accept everything turns both red. That is what separates the wiring from the check.Local run against
developat 1d04bcc: ruff clean, pylint exit 0,pytest tests/unit tests/integration2219 passed, 51 skipped.A format-aware log backend would be the larger fix. This is the preflight that stops data being lost in the meantime.