If we are using a follow-up survey as a measurement tool, should we be measuring improvement in the majority of respondents or majority of total learners?

If you are aiming to satisfy the Improves Performance Commendation Criterion, you would need to measure the improvement in respondents.  The ACCME explains that “the majority of learners whose performance has improved would be calculated based on those learners who respond to the follow-up survey”.  The expectation is that the provider should be ensuring that there are adequate mechanisms in place to optimize the response rate.