Answering my own question again: A helpful chat with Qualtrics support confirmed that this feature is not supposed to work on Matrix questions and multiple question pages for the same reason it doesn't on mark all that apply questions.

This is an awkward situation when a single bipolar question is used. As this requires one mouse click events whereas one would be sufficient. I can see that it somewhat problematic for multiple questions in a matrix; but even in that case, a control on "all questions" answered (if forced) should be sufficient.