Repository navigation
Allow tuning of size distribution introduced in 2.10.0 #444
Description
From my experience with other QuickCheck libraries, I was surprised by the prevalence of ("", 0) in this:
> Gen.sample 10 10 Arb.from<(string * int) list>.Generator;;
val it : (string * int) list list =
[[("", 0); ("", 4)];
[(null, 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0);
("", 0); ("", 0)]; [("lb", 1)];
[("", 0); ("", 0); (null, 0); ("", 0); ("", 4); ("", 0)];
[("", 0); ("3", 1); ("", 0); (null, 0); ("", 0); ("", 0); ("", 0); ("", 0);
("", 0)]; [("", -1); ("", 0); ("", 0); ("", 0)]; [("", 0); ("", -1)];
[(null, 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0)];
[("", 0); (null, 0); ("^", -1)]; [("QAz", 0)]]
I've briefly scanned the paper you reference in the release notes for 2.10.0, and, if I understand correctly, FSCheck is only using the fixedSize Distributor described in the paper. I think it would be useful to implement the other Distributors described, and provide some mechanism for selecting them when generating data.
Activity
kurtschelfthout commentedon Jul 18, 2018
I became convinced that the way the original quickcheck distributed size is less than ideal; what we're used to is not necessarily better.
Also, my sense is that adding distributors causes more complexity than it's worth.
The options available in FsCheck today, and likely for the forseeable future, are:
Gen.kurtschelfthout commentedon Jan 16, 2021
Just to let you know I've changed my mind about what I said above. I now think we should somehow allow either the old or the new behavior for product-like types, and that it should be configurable. I need to think a bit of how to fit it into the API though, and it will go into v3.
witoldsz commentedon Nov 29, 2021
I am having similar issue, i.e.: having such a signature:
rates: Map<ProviderId, RateObject list>andMaxTest=5the FsCheck is generating, almost always, for 3 of 5 times the map with empty list.On the other hand, if I change
ratesto:rates: RateObject listandMaxTest=5then FsCheck is generating 5 times a very long lists. Not even once the list is empty.
The
RateObjectlooks like this:What is wrong? Why the generated data is so bad? On one side the lists are empty 3/5 times, on the other I cannot even get an empty list to see if the code handles it well. Please advise.
AlexeyRaga commentedon Jul 12, 2022
I am having this problem too.
When generating a record with a few
stringfields, in a reasonable (100-200) amount of tests the strings are not getting longer than 1, rarely 2 characters and are very oftennull.This is problematic because this way the actual coverage is very small.
Has anyone found a workaround? Except to tuning
startSize, which is not a greatest idea?kurtschelfthout commentedon Jul 12, 2022
Why not? Increasing
StartSizeandEndSizeseems fine to me.