And if you follow the main questline instead of exploring the Valley, you will be locked out of it forever without ever knowing.
IIRC, you can't get past the minotaur guards peacefully (which I assume most players will prefer) unless you get the Champion of Hawkthorne title that requires you to do the centaur village quest. That quest kinda points you in the direction of the marefolk village, which is close to the caves where you find the cursed armor.
I fully agree that ideally there would be at least some kind of hint dropped at some point about the existence of the cave and the armor set, though I'd prefer if its effects are kept a surprise or at least pretty vague. The proximity of the cave to the marefolk village is already a huge hint as to what any artifact contained there might do, considering that the curse that befalls all of their men and the way they deal with it is the most prominent aspect about them.
Yeah, honestly, I feel žthat Brint probably turns off more players then he gains, so having something along the lines of big red sign along the lines "No worries guys, you can turn this bull into a best girl". Because really, not having it just seems like shooting yourself in a foot. Especially considering, going by statistics, Brienne is second most popular companion, and I do wonder how much more she would gain if she came along from the start instead of being saddled to a big log.
There really are no obvious aspects of Brint that could be a deal breaker for people when it comes to simply having him in your party, now that his and Cait's collective sluttery been gated behind you sleeping with both of them (and agreeing that you indeed want what each of them offers after they ask). Even if you don't want him as your Champ's bulldaddy, he is a cool dude that has a lot of interesting things to say and a lot to do in most of the adventures he follow the Champ around for. He is second only to Cait in this regard.
Aside from not keeping the cave's existence a complete easter egg, the only thing I'd change if I could wave a wand or throw some cash and make it so is removing the need to betray your companion's trust and allow a clearly cursed artifact to forcibly rewrite his personality.