No, keep it high; "diminishing returns" just means that each 1% of fertility/virility still increases the chance of impregnation, but that increase gets smaller (though never to or below 0) as fertility/virility gets higher. As a random example, going from 100% to 200% fertility might give you a 10% increase in the chance of getting pregnant, while going from 500% to 600% might give a 5% increase, and 1,000% to 1,100% might only give 2% (for a total of +17%, higher when counting the other increases between those points). I've no idea if those values are even close to what would actually happen, but hopefully you get the idea.