I think I figured this out. The problem is that the preview's opt-out link doesn't have identifying information in it, so the unsubscribe page doesn't know who is unsubscribing. This makes sense because it is a preview email. You don't want preview emails to alter your contact's opt-in status.
I tested this theory by sending my self a distribution that was not a preview. The opt-out link looked like this:
http_s://[Redacted]/CP/Register.php?OptOut=true&RID=[lots of characters]&LID=[more characters]&DID=[even more characters]&BT=[just a few characters]&_=1
The preview's opt-out link looked like this:
http_s://[Redacted]/CP/Register.php?OptOut=true&RID&LID&DID=[lots of characters]&BT=[just a few characters]&_=1
I assume that the RID and LID parameters are the IDs that the opt-out page needs to successfully unsubscribe the recipient. Not having them results in the #Registration, RequireRecipientID# error.