arXiv:2311.06840v4 Announce Type: replace-cross
Abstract: We explore “omitted label contexts,” in which training data is limited to a subset of the possible labels. This setting is standard among specialized human experts or specific, focused studies. By studying Simpson’s paradox, we observe that “correct” adjustments sometimes require non-exchangeable treatment and control groups. A generalization of Simpson’s paradox leads us to study networks of conclusions drawn from different contexts, within which a paradox of nontransitivity arises. We prove that the space of possible nontransitive structures in these networks exactly corresponds to structures that form from aggregating ranked-choice votes.