Provide a way to control bias

[dubzzz]

💡 Idea

For some users fast-check might not be biased enough. We should maybe provide a way to have a deeper control on the bias value.

Motivation

#5964 (comment)

[mcandre]

strat by simply ensuring that all zero values are checked. this is the lowest hanging fruit with the deepest impact.

[dubzzz]

strat by simply ensuring that all zero values are checked

Unfortunately, that's not an option.

It would have deep impact on core principles of fast-check and would break some of them notably the fact to be reproducible without having to recompute the whole sequence of inputs.

It's also going against another of the leading reasons for property based testing which is that we don't have to think about counterexamples ourselves as runners will go through them one day. If we always run counterexamples X, Y and Z we might never run against 'random' values ever.

For these reasons, for now, I don't see any viable option except providing users with a way to increase the bias.

[mcandre]

libFuzzer already does this. There's a reasonable expectation that the fuzzing framework does this out of the box. About 50% of fuzzing frameworks do not do this by default, inviting accidents. It may take a rewrite, but the safety concerns are important.

Yes, I specifically use fuzzing frameworks to do two things: test all my zero states, and test uniform random states. It's frustrating to have to do either manually. In practice, too many engineering teams do neither. Because of the high barrier to entry.

[dubzzz]

In fast-check, nothing prevents users to implement there own generators of integer values being possibly backed by the shrinker of our own integer generator, so nothing blocks you to get this behavior.

[dubzzz]

Anyway, as a maintainer I don't plan to implement this feature. So up to you to propose an extra set of arbitraries that can be used in fast-check and that fulfill your needs or to fork and maintain a fork.

[mcandre]

I'll stick with formal verification. Proving software correct is easier than jumping through hoops.