Add @FakerSource annotation and FakerSourceProvider for dynamic test data generation

[spannm]

RFC — looking for feedback, not asking for a merge yet

Right now, "generate N fake values and check a property" means @RepeatedTest plus a manual faker.x().y() call, and locale variants each get their own static Faker field. @FakerSource replaces that with one annotation:

@ParameterizedTest(name = "[{index}] {0}")
@FakerSource(code = "address#longitude", repeat = 10)
void testLongitude(String longitude) { ... }

It resolves a provider method via reflection (also handles locales, whole-provider injection, and combinatorial params). Test-only, lives under src/test — this is tooling for our own suite, not public API. I migrated AddressTest, CodeTest, FinanceTest, HttpTest as a proof of concept.

Why it's nice: the standout win is much better test feedback. Instead of one aggregate @RepeatedTest result where generated values are buried in a loop, each value gets its own named, inspectable row in the report — you can see exactly which input triggered a failure. Compare irishAddress before/after:

Before (@RepeatedTest — values hidden):

After (@FakerSource — values visible by name):

Same story for creditCardWithType — before you only see the enum branch that ran, after you see the actual generated card number per case:

Before:

After:

It also kills off a bunch of one-off static locale-Faker fields and @MethodSource generator methods.

Tradeoff to weigh: resolution is string-based reflection, so a rename like Address.longitude() isn't caught by the IDE while editing. That said, it is caught immediately at build time — an incorrectly annotated @FakerSource simply fails the test, same as any other broken test, so nothing wrong slips through CI unnoticed. There's also a fallback that re-instantiates the test class if the instance isn't ready yet during early JUnit lifecycle, and method resolution grabs the first match by name/type — worth watching if we ever get overloaded provider methods, though it hasn't been an issue in the proof-of-concept migration.

Before I migrate more tests: given the much better test feedback, is this a direction we want — with the understanding that any misconfiguration surfaces as a failing test, not a silent runtime issue?

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.48%. Comparing base (eb75d1c) to head (4762d0a).
⚠️ Report is 1 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #1869   +/-   ##
=========================================
  Coverage     92.48%   92.48%           
- Complexity     3561     3562    +1     
=========================================
  Files           346      346           
  Lines          7048     7048           
  Branches        685      685           
=========================================
  Hits           6518     6518           
- Misses          365      366    +1     
+ Partials        165      164    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[asolntsev]

I would say that the public API for end users is new Faker().address().longitude(). And this API should be called from tests.

The fact that DF uses YML and cryptic strings like address#longitude is rather an implementation details. Most users should be aware of it.

[spannm]

I would say that the public API for end users is new Faker().address().longitude(). And this API should be called from tests.

The fact that DF uses YML and cryptic strings like address#longitude is rather an implementation details. Most users should be aware of it.

@asolntsev fair point — but this is test-only tooling under src/test, not a public API change. The code string just points at the same public call (address().longitude()), the same way @MethodSource("someGenerator") already points at a method by name today. Under the hood, @FakerSource still exercises the public API — it doesn't replace or hide it.

The real payoff is the test report: every generated value now shows up individually and by name (see before/after screenshots above), instead of being buried in a @RepeatedTest loop. And if the code string is wrong, the test just fails at build — no silent runtime surprise.

Happy to rename code if it undersells that it's just referencing existing public API.

P.S.: Updated PR description for clarity and added screenshots.

cc: @bodiam

[bodiam]

Sorry, I'm closing this because of AI messages.

[spannm]

That's disappointing - I put in a considerable amount of work and genuinely believe it adds value to the test suite. The screenshots speak for themselves.

[bodiam]

The fact that you put a lot of work in it or not is no reason to add it to the code. Besides, you said you were looking for feedback, and now you have feedback.

I don't think it's very useful at this stage, @asolntsev doesn't think it's super useful, and if you want, you can always create a small side library, datafaker-junit5 or so, which has this functionality. If it's used by other people, we can always reconsider to add it to the core, and make it part of the "public API" or so, but at this stage, I'm not convinced about this.