To make things even, it's supposed to sit out the strongest scores on the tribe with more members. BUT, the system instead chooses the lowest scores of players who STARTED the challenge, meaning that all those 0s the inactives got don't come into play. If you changed it to include those 0s, I think you'd find a fairer game that actually punished inactives instead of rewarding them.