All the affinity is self-contained between characters. If say Kyle and Hanna have maxed out affinity, it doesn't change Kyle or Hanna's interactions with the other characters.

Affinity does affect the ending, but that's a more specific case for one character that you'll figure out when you play, so be on the lookout for that in the 2nd half of the game. Happy Fuga-ing