| multi {curl} | R Documentation |
AJAX style concurrent requests, possibly using HTTP/2 multiplexing. Results are only available via callback functions. Advanced use only!
multi_add(handle, done = NULL, fail = NULL, data = NULL, pool = NULL) multi_run(timeout = Inf, poll = FALSE, pool = NULL) multi_set(total_con = 50, host_con = 6, multiplex = TRUE, pool = NULL) multi_list(pool = NULL) multi_cancel(handle) new_pool(total_con = 100, host_con = 6, multiplex = TRUE) multi_fdset(pool = NULL)
handle |
a curl handle with preconfigured |
done |
callback function for completed request. Single argument with response data in same structure as curl_fetch_memory. |
fail |
callback function called on failed request. Argument contains error message. |
data |
callback function or open connection object for receiving data.
If |
pool |
a multi handle created by new_pool. Default uses a global pool. |
timeout |
max time in seconds to wait for results. Use |
poll |
If |
total_con |
max total concurrent connections. |
host_con |
max concurrent connections per host. |
multiplex |
enable HTTP/2 multiplexing if supported by host and client. |
Requests are created in the usual way using a curl handle and added
to the scheduler with multi_add. This function returns immediately
and does not perform the request yet. The user needs to call multi_run
which performs all scheduled requests concurrently. It returns when all
requests have completed, or case of a timeout or SIGINT (e.g.
if the user presses ESC or CTRL+C in the console). In case of
the latter, simply call multi_run again to resume pending requests.
When the request succeeded, the done callback gets triggered with
the response data. The structure if this data is identical to curl_fetch_memory.
When the request fails, the fail callback is triggered with an error
message. Note that failure here means something went wrong in performing the
request such as a connection failure, it does not check the http status code.
Just like curl_fetch_memory, the user has to implement application logic.
Raising an error within a callback function stops execution of that function but does not affect other requests.
A single handle cannot be used for multiple simultaneous requests. However it is possible to add new requests to a pool while it is running, so you can re-use a handle within the callback of a request from that same handle. It is up to the user to make sure the same handle is not used in concurrent requests.
The multi_cancel function can be used to cancel a pending request. It has no effect if the request was already completed or canceled.
The multi_fdset function returns the file descriptors curl is
polling currently, and also a timeout parameter, the number of
milliseconds an application should wait (at most) before proceeding. It
is equivalent to the curl_multi_fdset and
curl_multi_timeout calls. It is handy for applications that is
expecting input (or writing output) through both curl, and other file
descriptors.
results <- list()
success <- function(x){
results <<- append(results, list(x))
}
failure <- function(str){
cat(paste("Failed request:", str), file = stderr())
}
# This handle will take longest (3sec)
h1 <- new_handle(url = "https://eu.httpbin.org/delay/3")
multi_add(h1, done = success, fail = failure)
# This handle writes data to a file
con <- file("output.txt", open = "wb")
h2 <- new_handle(url = "https://eu.httpbin.org/post", postfields = "bla bla")
multi_add(h2, done = success, fail = failure, data = con)
# This handle raises an error
h3 <- new_handle(url = "https://urldoesnotexist.xyz")
multi_add(h3, done = success, fail = failure)
# Actually perform the requests
multi_run(timeout = 2)
multi_run()
# Check the file
close(con)
readLines("output.txt")
unlink("output.txt")