Advertise on podcast: Total Survey Design

On this episode of the Total Survey Design podcast, I put generative AI to the test by asking it to critique and improve the Microsoft Word Feedback survey: a short, everyday questionnaire that many of us have seen after using the program. I compare the suggestions from Grok, ChatGPT, and Gemini, highlight where they agree, where they differ, and share my own thoughts on what they got right and what they missed. I recap the original survey (which takes just 50 seconds to describe), dive into the strengths and weaknesses of the AI outputs, and explain why I still think human expertise in survey methodology matters most. Along the way, I discuss the pitfalls of the Net Promoter Score scale, the importance of balanced question wording, building trust with privacy statements, and more. There is still time to have your voices heard! Share your thoughts about this podcast at: https://www.tinyurl.com/totalsurveydesign Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

On this bonus episode of the Total Survey Design podcast, I follow up on Trump and Rogan’s polling critiques with a listener email highlighting nonresponse bias, the real issue behind low response rates, where responders often differ systematically from non-responders. I explain how rigorous weighting corrects for this bias, why it’s not foolproof, and how strong weighting helped make the 2024 election polls some of the most accurate in years, with top aggregators and pollsters like AtlasIntel missing final margins by less than 2–3 points. The takeaway: polling isn’t broken, it is actually getting better with time! Complete the Podcast Feedback Survey by visiting: www.tinyurl.com/TotalSurveyDesign Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

On this episode of the Total Survey Design podcast, I examine an episode of the Joe Rogan Experience podcast, #2219, which aired on October 25th, 2024, where he discusses survey sampling with President Donald Trump. They both express skepticism about the reliability of polls for making broad national claims. They also refute the ability to make predictions from small samples and criticize polling methodology. In this episode, I talk about what they got right and what they got wrong, diving into the science of sampling. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, Azdren creates a feedback survey from scratch, showing the process from idea formation, to messy first draft of the questionnaire, to final product.  To see the final version of the questionnaire and to give your valuable feedback on this podcast, please visit https://tinyurl.com/totalsurveydesign  Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

This episode of the Total Survey Design podcast demystifies the statistical technique of survey weighting (or post-stratification), explaining how it corrects sample imbalances by giving "quiet groups" more influence. The hosts define weighting, detail when it's an effective correction tool for probability samples, and provide practical advice on avoiding common pitfalls like over-adjustment that can hurt data precision. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

This episode explains the important distinction between response rate and nonresponse error in survey research. Drawing on principles from Dillman et al.(2014), it clarifies that a high response rate does not necessarily mean low nonresponse error, and vice versa. Through two practical examples, the episode illustrates how systematic differences between respondents and nonrespondents -- not the number of responses -- determine the presence of nonresponse error. The episode emphasizes that evaluating survey quality requires more than just looking at response rates; it requires understanding potential bias in who does and does not respond. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, the hosts speak with Mark Miazga, Director of University Survey and Assessment Services at the University of Minnesota, about his journey into survey research. Mark shares how his upbringing in rural Wisconsin and a background in law and nonprofit work shaped his perspective on data and public service. The conversation covers his transition from corporate law to survey methodology, his work managing complex institutional surveys, and how empathy and practical experience influence his approach to data collection and analysis. The discussion also touches on bridging gaps between institutional research and survey methodology, as well as a discussion of an institution-wide survey calendar, survey timing, and sampling. The episode wraps up with four points of advice for future surveyors.  Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, we talk with Robert Santos, Director of the U.S. Census Bureau from 2022 to 2025, about his professional journey and philosophy toward data and public service. Rob shares how his upbringing in a Mexican American community in San Antonio and how his early exposure to music and math shaped his path into statistics. The conversation explores his commitment to equity in data, the importance of cultural understanding in survey design, and his reflections on his time at the Census. He also discusses challenges and innovations in modern data collection (including the use of AI), and the trade-offs between cost, quality, and timeliness in survey methodology. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, I talk about a survey I didn’t plan to analyze—one that popped up the moment I opened Microsoft Word. Microsoft asked for feedback, and well… I had some. I break down the flaws in their opening question, including directional bias, vague wording, and unlabeled scale points. I also reflect on how these design choices impact data quality and what could have been done better. It’s a reminder that even the most common surveys we see in the wild from even the biggest tech giants can teach us a lot about what not to do. If you just enjoy a good critique, you’ll want to stick around for this one. Contact us at: [email protected]. And follow us on Instagram @totalsurveydesign. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode of Total Survey Design, Dr. Azdren Coma and Dr. Seon Yup Lee interview Dr. Jon Krosnick, a leading expert on political behavior, survey methodology, and public opinion. Dr. Krosnick is a professor at Stanford University who has advised the U.S. Census Bureau, led national studies on voting behavior and climate change, and published extensively on how people form attitudes and make decisions. Recorded at the historic St. Louis Union Station Hotel during the 2025 AAPOR conference, this conversation covers Dr. Krosnick’s career path, the role of mentorship, and his research on trust and measurement validity. The episode also addresses broader topics in academia, such as the pressures of publishing, the importance of constructive feedback, and the value of humility and replication in science.  Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode I sat down with Thom to talk about surveys. We talked about his time at the Social and Economic Sciences Research Center, working with Don Dillman on various projects including the 2000 U.S. Census, some of his favorite projects, my dissertation survey, the application of the Tailored Design Method on a wide variety of cultures, and much more. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, we do a small retrospective on the first season. We also look ahead to see what is to come. We also give a call to action, seeking your support to ensure that this project continues to exist into the future. If you would like to support us, you can do so by visiting our Ko-fi page at: https://ko-fi.com/totalsurveydesign Thank you. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, we explore essential considerations for crafting effective surveys.  We begin by emphasizing the importance of defining the purpose of your survey, which serves as a guide through each step of its creation. We discuss how understanding your main goal—whether it's measuring customer satisfaction or academic variables—shapes your questions and improves data quality. Additionally, we highlight the significance of knowing your target population, budget, and timeframe. By the episode's end, we offer practical advice on streamlining your survey to avoid unnecessary questions, reduce costs, and enhance response quality. Join us to learn how to design your survey with precision and purpose. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, we discuss the target population. We emphasize the importance of a well-defined target population, in context of the goal of your survey, and in relation to your sampling frame. We offer examples of target populations of various studies, and offer suggestions for the types of variables that you may want to consider when thinking about your target population.  Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]

In this episode, we interview James McCall, a survey methodologist at Westat. James shares what first drew him to surveys, and what drove him into a career of surveys. We talk about his role as a survey methodologist in industry. We discuss cognitive testing, data scripts, and other elements of the art and science of survey design. We also discuss our pet-peeves, such as matrix questions, check-all-that-apply questions, pre-testing, having too-short timelines, and more. James also shares advice for social scientists who are interested in pursuing a career in survey methodology. Support the show Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign Contact us at: [email protected]