Despite numerous studies devoted to mathematics aptitude and achievement, research on how individuals experience math has remained relatively fragmented. Here, using a combined theoretical and data-driven approach, we sought to characterize self-reported math experiences, with a particular focus on negative math experiences. An examination of existing literature led to the identification of eight potential facets of math experiences: emotional, cognitive, physiological, behavioral, testing, classroom/social performance, self-efficacy, and attitudinal. We generated survey items intended to probe experiences within each of these facets and constructed a preliminary questionnaire of 107 candidate items, comprising positively and negatively framed statements about one’s math experiences, with data from a final analytic sample of N = 803 adult participants. Focusing on negative items, four key factors emerged from the data: negative attitudes and avoidance, physiological experiences, testing and educational experiences, and cognitive and emotional experiences. These results point to opportunities for contact between literatures (e.g., between negative attitudes and avoidance behaviors), and toward relatively unexplored topics, such as the importance of negative physiological experiences when facing math. On a practical level, we also provide short subscales with sound internal metrics for each of the four factors identified above. Taken together, this work may prove useful on both a theoretical and a methodological level for those looking to develop a unifying framework of negative math experiences.