Why do error messages not match? And aren't meaningful?

[tim-rohrer]

VS Code Version: 1.74.3
Python plugin: v2022.20.2

I'm not sure I have a full pattern yet of what is happening, but if I run tests from the beaker, the last test doesn't run and Output shows:

Run completed, parsing output
Error while running tests: test_fetch_csv_data_fails_if_bad_file
TypeError: Cannot read properties of undefined (reading 'length')

Error while running tests:
TypeError: Cannot read properties of undefined (reading 'length')

Finished running tests!

However, the test doesn't show green or red, just a plain grey circle.

If I then go to the terminal and run pytest tests, the test fails with:

        """The exception value."""
>       assert (
            self._excinfo is not None
        ), ".value can only be used after the context manager exits"
E       AssertionError: .value can only be used after the context manager exits

So yeah, there is a problem with the test but the test beaker doesn't provide meaningful data, or even explicitly fail.

Why is this?

[brettcannon]

Do you use any pytest plug-ins that change the output? Or are you explicitly printing to stdout as part of your test run? And is your config for running tests in VS Code the exact same as pytest tests?

[tim-rohrer]

No, and no. Other than running the script provided by VS Code (when clicking on the beaker the first time, I've done no configuration for either. Just pip install pytest.

If this isn't something you've seen before, I'll work on a minimal reproducible code set and post it here along with screen caps of what I'm seeing.

[brettcannon]

Yeah, we've not seen this before, so a reproducer would be great! You can follow the instructions at https://aka.ms/pvsc-bug to report an issue.

[tim-rohrer]

I've not been able to reproduce this; I had deleted a bunch of test code from my early attempts.

Tbh, the test runner seems to be working okay now. I definitely saw some flakiness early on, but that could (and probably should) be attributed to my inexperience.

I'm going to mark this as answered/resolved, and if I do come up with it again I'll make a better write up and open a bug report.