I think I understand what you're saying. If you are storing the ID in Qualtrics after the first time via iQ directory or a contact list, you'll need to leave those blank to be set by the panel or URL. I don't know of a way to do it without storing the recipient as a contact, though.
Once you store them as a contact, see instructions
here, I'd create a branch after a First time user check and then if they aren't a first time user, fill it with the panel. More instructions
here.
Hope I understood and that this is helpful!