Hi Emetis!
What you could do is to save that average as an embedded data on the survey flow, and using an action(workflow) send that embedded data to the XM Directory, in that way, when the person takes a new survey you could show the last AVG of the score that the person gave.
You could use scoring instead of a math operation, and on the survey flow you would have the option to set a value for the embedded data with the scoring average.
In this link you could find a video that I made https://drive.google.com/file/d/1JWBlasU6b8bpGLsOGRE3ki0K0-3wWbM_/view?usp=sharing