You will get your
shit wrecked unless you play on Story (ie the lowest) difficulty. The game is balanced around having a three-person party and
no accommodation will be made if you choose to handicap yourself by running anything less than that.
And yes, you will miss all kinds of incidental dialogue involving your companions and there are several quests where the presence of certain party members can make a huge difference, depending on how you've built your Champion. To pick one example, you're not going to do well at all in the early parts of Winter Wolf without companions because your success in the first part is partially determined by whether you bring 'strong' characters with you and there are all kinds of checks in the second part which the game has flags for and which are largely based on who you brought with you; there aren't any 'wrong' companions to bring but the game assumes that you brought
someone.
Also, there are various plotlines that depend on your companions, Cait in particular. There's a lot of existing/planned content based around the temple which is completely inaccessible unless you finish the Cat Call quest (all of which requires her presence) and there's stuff only indirectly connected to Cait which is (currently at least) gated behind that quest. There's likely to be a similarly large amount of stuff in the upcoming Khor'minos plotline which will be enhanced if you have Brint/Brienne with you. You can't avoid bringing Ryn along for one mandatory dungeon and there's an optional one that requires her as well.
There's also an entire storyline gated behind Kiyoko's recruitment path and while you can play through the game without it, you're literally never going to see
any of that stuff on a run where you ignore her entirely, and you're also probably going to Bad End yourself on a particular quest if you both refuse to engage with her content and insist on running a solo-Champion party. Completing
that quest is required to access most of the recurring kitsune content.