How do i link my custom css file in the File Library to a survey?

If you're using the old look and feel editor, use the "External CSS" input text box and add the URL to your CSS code.

Does this work?

But that will mean the css file is hosted externally, not in my files Libraries. I need to reference it from my file library as its the central place where all assets are stored

Try taking the link in the "href" within the photo that you took above and putting that URL in the "External CSS". You can still use the External CSS for CSS in Qualtrics libraries, as long as you have the link.
Something I forgot to add-- you'll also want to be sure that you have the "Blank (for styling)" Theme selected to ensure that your CSS shows up properly and is not conflicting with the CSS on other themes.
You can also get the URL for any file in your library by clicking on the file, selecting "View", and copying the URL of the webpage it brings you to.

The workaround I use for this is to place the script and link tags in an answer choice. This is especially useful, when I want to import specific libraries for specific pages.
Take a look at this demo.
This is what the choices look like:

thanks