API Pagination in R and httr2

Question

I am attempting to use R’s httr2 package to return a list of all mailing lists from the list mailing list endpoint. I am really struggling to leverage the httr2 iteration helpers in this context. Specifically, it seems the skipToken query parameter is encoded and I cannot simply use iterate_with_offset(“skipToken”, offset = 100)How should the skipToken value (the offset argument in the above line of code) be encoded to be recognized by the Qualtrics API?

Nam Nguyen · Accepted Answer

@brianjiterate_with_offset only works for numeric-based pagination.skipToken doesn’t just a page number,the skipToken is passed as a query parameter in each request after the first one.skipToken is a string: The value is provided by the API and you simply need to pass it in subsequent requests as a query parameter. The loop continues until skipToken becomes NULL, indicating there are no more pages of results to fetch.Hope it help you

brianj · Answer

Thanks, Nam. I ended up writing my own function for the next_req argument inhttr2::req_perform_iterative(). So overall, it looks something like this:library(tidyverse)library(httr2)# Build Requestreq <- request("https://yul1.qualtrics.com/API/v3") %>% req_url_path_append("directories/`my_directoryId`/mailinglists") %>% req_headers(`X-API-TOKEN` = Sys.getenv("QUALTRICS_API_KEY"))# Iterate through all pagesresps <- req_perform_iterative( req, next_req = function(resp, req){ url <- resp_body_json(resp)$result$nextPage if (!is.null(url)) { req_url(req, url) } }, max_reqs = Inf)

Sign up

Login to the Community

Scanning file for viruses.

This file cannot be downloaded