page\_content
, which provides both HTML and wikitext as possible output formats. Older, individual revisions can be retrieved with revision\_content
. These functions also return a range of possible metadata about the revisions or articles in question.
Diffs between revisions can be generated using revision\_diff
, while individual ''elements'' of a page's content - particularly links - can be extracted using page\_links
, page\_backlinks
, and page\_external\_links
. And if the interest is in changes to content, rather than content itself, recent\_changes
can be used to grab a slice of a project's Special:RecentChanges feed.
## Retrieving metadata
Page-related information can be accessed using page\_info
, while categories that a page possesses can be retrieved with categories\_in\_page
- the inverse of this operation (what pages are in a particular category?) uses pages\_in\_category
.
User-related info can be accessed with user\_information
, while user\_contributions
allows access to recent contributions by a particular user: this can be conveniently linked up with revision\_content
, mentioned above, to retrieve the content of the last N edits by a particular editor, or metadata about those edits. WikipediR/NEWS 0000644 0001762 0000144 00000006362 14601320617 012647 0 ustar ligges users Version 1.5.0
------------------------------------------------------------------------------
BUG FIXES
* https is now the default for requests (Thanks to Hiroaki Yutani)
* UTF-8 page names and similar are properly handled (Thanks to Hiroaki Yutani)
Version 1.4.0
------------------------------------------------------------------------------
NEW FEATURES
* `random_page` now allows you to return more than one page.
* `page_backlinks` does as well; thanks to Brock Tibert for the code.
DOCUMENTATION
* Vignette tweaks made
* Package-level documentation improved.
DEVELOPMENT
* "Depends" switched out for "Imports" to avoid annoying loading messages.
Version 1.3.0
------------------------------------------------------------------------------
BUG FIXES
* Various miscellaneous internal improvements.
* `limit` introduced as an argument to pages_in_category - thanks to Ben Marwick for finding the bug.
* `limit` introduced as an argument to page_links - thanks to Hui Li of the Universität Heidelberg for the initial report.
Version 1.2.0
------------------------------------------------------------------------------
NEW FEATURES
*Random pages can now be retrieved with random_page
*pageID based querying is available
*Custom user
DEVELOPMENT
*Doc tweaks and some simplification of argument parsing around page_content and random_page
*Additional unit tests
*query() marked as exportable to allow simple third-party plugins into WikipediR
Version 1.0.1
------------------------------------------------------------------------------
* Roxygenised documentation, some R CHECK errors fixed.
* Test suite added for error handlers.
Version 1.0.0
------------------------------------------------------------------------------
* First release version
* Introduction of wiki_usercontribs, which retrieves recent contributions for a specified username
* Bugfixes to error handling routines
* Bugfix to problems around wiki_con, where species or commons users were unable to take advantage of CurlOpts.
Version 0.7.0
------------------------------------------------------------------------------
* wiki_page now returns the DOM of the relevant page, not the wikimarkup, in line with a suggestion from Scott Chamberlain. This should be parsable by the HTML tree parser in the XML package.
Version 0.6.3
------------------------------------------------------------------------------
* Standardisation and cleanup of error handlers, and the introduction of a single central LimitHandler.
Version 0.6.2
------------------------------------------------------------------------------
* Bug fix from Scott Chamberlain around wiki_page, switching the content type over to x-wikitext.
* Introduction of w_timeout as a field in the connector object; will be extended to other CURL options in time.
Version 0.6.1
------------------------------------------------------------------------------
* Support for Commons and Wikispecies
* Addition of wiki_recentchanges(), which allows a useR to query the recentchanges feed.
Version 0.5.1
------------------------------------------------------------------------------
* Format for wiki_revision changed from the (unacceptable-to-API) text/css to x/wikitext.
* Error handling improved to pass the full error explanation through to the user, in the case of API errors. WikipediR/R/ 0000755 0001762 0000144 00000000000 14601320617 012342 5 ustar ligges users WikipediR/R/categories.R 0000644 0001762 0000144 00000014473 14604033612 014622 0 ustar ligges users #'@title Retrieves categories associated with a page.
#'
#'@description
#'Retrieves categories associated with a page (or list of pages) on a MediaWiki instance
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param pages A vector of page titles, with or without spaces, that you want to retrieve
#'categories for.
#'
#'@param properties The properties you want to retrieve about the categories.
#'Options are "sortkey" (the key that sorts the way the page is stored in each category),
#'"timestamp" (when the category was added to that page) and "hidden" (tags those categories
#'in the returned list that are 'hidden' from readers).
#'
#'@param limit The maximum number of categories you want to retrieve for each page. Set
#'to 50 by default.
#'
#'@param show_hidden Whether or not to include 'hidden' categories in the categories
#'that are retrieved - these are usually associated with the maintenance of Wikipedia
#'and its internal processes. Set to FALSE by default.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@seealso \code{\link{pages_in_category}} for pages in a specified category.
#'
#'@examples
#'\dontrun{
#'#Retrieve the categories for the "New Age" article on en.wiki
#'cats <- categories_in_page("en", "wikipedia", pages = "New Age")
#'
#'#Retrieve the categories for the "New Age" article on rationalwiki.
#'rw_cats <- categories_in_page(domain = "rationalwiki.org", pages = "New Age")
#'}
#'@export
categories_in_page <- function(language = NULL, project = NULL, domain = NULL,
pages, properties = c("sortkey","timestamp","hidden"),
limit = 50, show_hidden = FALSE, clean_response = FALSE,
...){
#Check, construct URL
properties <- match.arg(properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
pages <- handle_limits(pages, 50)
if(show_hidden){
show_hidden <- "hidden"
} else {
show_hidden <- "!hidden"
}
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
prop = "categories",
clprop = properties,
clshow = show_hidden,
cllimit= limit,
titles = pages
)
#Retrieve, check, return
content <- query(url, "pagecats", clean_response, query_param = query_param, ...)
page_names <- names(unlist(content))
missing_pages <- sum(grepl(x = page_names, pattern = "missing"))
if(missing_pages){
warning("This request contained ",missing_pages," invalid page title(s)", call. = FALSE)
}
return(content)
}
#'@title
#'Retrieves a list of category members.
#'
#'@description
#'wiki_catpages retrieves a list of pages, subcategories, files or all of the above
#'in a specified category (or series of specified categories)
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param categories The names of the categories you want to gather information for.
#'
#'@param properties The properties you want to gather for each member of the category.
#'Options are "title" (the name of the member, including namespace),
#'"id" (the unique numeric identifier of the member), "sortkey"
#'(the hexadecimal key used to sort that member within the category),
#'"sortkeyprefix" (the human-readable sort key), "type"
#'(whether the member is a page, a subcategory or a file) and
#'"timestamp" (when the member was added to the category)
#'
#'@param type The type of member you're interested in returning;
#'options are any permutation of "page" (pages), "subcat" (subcategories) and "file" (files).
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param limit The maximum number of members to retrieve for each category. Set
#'to 50 by default.
#'
#'@param ... further arguments to pass to httr's GET().
#'
#'@section warnings:
#'Because of the way MediaWiki stores this data, both "the category you asked for doesn't exist"
#'and "the category you asked for exists, but has no members" return in the same way.
#'
#'@seealso \code{\link{categories_in_page}} for finding categories that a specified page is a member of.
#'
#'@examples
#'\dontrun{
#'#Retrieve the pages in the "New Age" category on en.wiki
#'cats <- pages_in_category("en", "wikipedia", categories = "New Age")
#'
#'#Retrieve the pages in the "New Age" category on rationalwiki.
#'rw_cats <- pages_in_category(domain = "rationalwiki.org", categories = "New Age")
#'}
#'@export
pages_in_category <- function(language = NULL, project = NULL, domain = NULL, categories,
properties = c("title","ids","sortkey","sortkeyprefix","type","timestamp"),
type = c("page","subcat","file"), clean_response = FALSE, limit = 50,
...){
#Format and check
categories <- gsub(x = categories, pattern = "^", replacement = "Category:")
categories <- handle_limits(categories, 50)
properties <- match.arg(properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
type <- match.arg(type, several.ok = TRUE)
type <- paste(type, collapse = "|")
#Construct URL
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
list = "categorymembers",
cmtitle = categories,
cmprop = properties,
cmtype = type,
cmlimit = limit
)
#Query and return
content <- query(url, "catpages", clean_response, query_param = query_param, ...)
return(content)
} WikipediR/R/metadata.R 0000644 0001762 0000144 00000021177 14604033721 014255 0 ustar ligges users #'@title Retrieve a page's backlinks
#'
#'@description
#'page_backlinks, when provided with a page title, retrieves backlinks to that
#'page. Output can be filtered to specific namespaces.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param page the title of the page you want the backlinks of.
#'
#'@param limit the number of backlinks to return. Set to 50 (the maximum) by default.
#'
#'@param direction the direction to order the backlinks in, by linking page ID: "ascending"
#'or "descending". Set to "ascending" by default.
#'
#'@param namespaces The namespaces to filter to. By default, backlinks from any namespace
#'are retrieved: alternately, a numeric vector of accepted namespaces (which are described
#'\href{https://www.mediawiki.org/wiki/Manual:Namespace#Built-in_namespaces}{here}) can be
#'provided, and only backlinks from pages within those namespaces will be returned.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@section Warnings: as with \code{\link{pages_in_category}}, if the page
#'you are linking to does not exist, an empty list will be returned, without
#'any indication of an error.
#'
#'@examples
#'\dontrun{
#'#Backlink
#'all_bls <- page_backlinks("en","wikipedia", page = "Aaron Halfaker")
#'
#'#Namespace-specific backlinks
#'mainspace_bls <- page_backlinks("en","wikipedia", page = "Aaron Halfaker", namespaces = 0)
#'}
#'@export
page_backlinks <- function(language = NULL, project = NULL, domain = NULL,
page, limit = 50, direction = "ascending", namespaces = NULL,
clean_response = FALSE, ...){
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
list = "backlinks",
bltitle = page,
bldir = direction,
bllimit = limit
)
if(!is.null(namespaces)){
query_param$blnamespace <- paste(namespaces, collapse = "|")
}
content <- query(url, "blink", clean_response, query_param = query_param, ...)
return(content)
}
#'@title Retrieve a page's links
#'
#'@description
#'page_links, when provided with a page title, retrieves internal wikilinks from the
#'current revision of that page.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param page the title of the page you want the links of.
#'
#'@param limit the number of links to retrieve. 50 by default; a maximum of 500 is set server-side.
#'
#'@param direction the direction to order the links in, by destination page ID: "ascending"
#'or "descending". Set to "ascending" by default.
#'
#'@param namespaces The namespaces to filter to. By default, links to any namespace
#'are retrieved: alternately, a numeric vector of accepted namespaces (which are described
#'\href{https://www.mediawiki.org/wiki/Manual:Namespace#Built-in_namespaces}{here}) can be
#'provided, and only backlinks from pages within those namespaces will be returned.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@examples
#'\dontrun{
#'#Links
#'links <- page_links("en","wikipedia", page = "Aaron Halfaker")
#'
#'#Namespace-specific links
#'mainspace_links <- page_links("en","wikipedia", page = "Aaron Halfaker", namespaces = 0)
#'}
#'@export
page_links <- function(language = NULL, project = NULL, domain = NULL,
page, limit = 50, direction = "ascending", namespaces = NULL,
clean_response = FALSE, ...){
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
prop = "links",
titles = page,
pldir = direction,
pllimit = limit
)
if(!is.null(namespaces)){
query_param$plnamespace <- paste(namespaces, collapse = "|")
}
content <- query(url, "plink", clean_response, query_param = query_param, ...)
return(content)
}
#'@title Retrieve a page's links
#'
#'@description
#'page_external_links, when provided with a page title, retrieves external wikilinks from the
#'current revision of that page.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param page the title of the page you want the links of.
#'
#'@param protocol limit links to those with certain link protocols. Options are listed
#'in Special:ApiSandbox's
#'\href{https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&prop=extlinks}{elprotocol field}.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@examples
#'\dontrun{
#'#Links
#'external_links <- page_external_links("en","wikipedia", page = "Aaron Halfaker")
#'
#'#Protocol-specific links
#'external_http_links <- page_external_links("en","wikipedia",
#' page = "Aaron Halfaker", protocol = "http")
#'}
#'@export
page_external_links <- function(language = NULL, project = NULL, domain = NULL,
page, protocol = NULL, clean_response = FALSE,
...){
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
prop = "extlinks",
titles = page
)
if(!is.null(protocol)){
query_param$elprotocol <- protocol
}
content <- query(url, "elink", clean_response, query_param = query_param, ...)
return(content)
}
#'@title Retrieve information about a particular page
#'
#'@description
#'page_info, when provided with a page title, retrieves metadata about that page.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param page the title of the page you want the metadata of.
#'
#'@param properties the properties you'd like to retrieve. Some properties (the pageID, namespace,
#'title, language, length and most recent revision ID, for example) are retrieved by default,
#'whatever is passed to \code{properties}: properties that can be explicitly retrieved include
#'the page's protection level ("protection"), the ID of the associated talk page, if applicable
#'("talkid"), the full, canonical URL ("url"), and the displayed page title ("displaytitle").
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@examples
#'\dontrun{
#'#Metadata
#'page_metadata <- page_info("en","wikipedia", page = "Aaron Halfaker")
#'}
#'@export
page_info <- function(language = NULL, project = NULL, domain = NULL,
page, properties = c("protection","talkid","url", "displaytitle"),
clean_response = FALSE, ...){
properties <- match.arg(arg = properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
prop = "info",
inprop = properties,
titles = page
)
content <- query(url, "pageinfo", clean_response, query_param = query_param, ...)
return(content)
} WikipediR/R/parse.R 0000644 0001762 0000144 00000005426 14601343044 013605 0 ustar ligges users #' Response parser
#'
#' Should not be externally used
#'
#' @title parse_response: Parse WikipediR responses internally
#' @param x result from a WikipediR query
#' @export
parse_response <- function(x){
UseMethod("parse_response", x)
}
#' @export
parse_response.rchanges <- function(x){
x <- x$query$recentchanges
return(x)
}
#' @export
parse_response.rcontent <- function(x){
x <- x$query$pages
names(x) <- NULL
return(x)
}
#' @export
parse_response.pcontent <- function(x){
x <- x$parse
return(x)
}
#' @export
parse_response.rdiff <- function(x){
x <- x$query$pages
names(x) <- NULL
return(x)
}
#' @export
parse_response.uinfo <- function(x){
x <- x$query$users
return(x)
}
#' @export
parse_response.ucontribs <- function(x){
x <- x$query$usercontribs
results <- unlist(x)
results <- data.frame(matrix(results, nrow = length(x), byrow = TRUE),
stringsAsFactors = FALSE)
names(results) <- names(x[[1]])
return(results)
}
#' @export
parse_response.catpages <- function(x){
x <- x$query$categorymembers
results <- unlist(x)
results <- data.frame(matrix(results, nrow = length(x), byrow = TRUE),
stringsAsFactors = F)
names(results) <- names(x[[1]])
return(results)
}
#' @export
parse_response.pagecats <- function(x){
x <- x$query$pages
names(x) <- NULL
results <- lapply(x,function(x){
cats <- unlist(x$categories)
cats <- data.frame(matrix(cats, nrow = length(x$categories), byrow = TRUE),
stringsAsFactors = FALSE)
names(cats) <- names(x$categories[[1]])
x$categories <- cats
return(x)
})
return(results)
}
#' @export
parse_response.blink <- function(x){
x <- x$query$backlinks
results <- lapply(x,unlist)
return(results)
}
#' @export
parse_response.plink <- function(x){
x <- x$query$pages
names(x) <- NULL
results <- lapply(x, function(x){
x$links <- lapply(x$links,unlist)
return(x)
})
return(results)
}
#' @export
parse_response.elink <- function(x){
x <- x$query$pages
names(x) <- NULL
results <- lapply(x, function(x){
x$extlinks <- unlist(x$extlinks)
names(x$extlinks) <- NULL
return(x)
})
return(results)
}
#' @export
parse_response.pageinfo <- function(x){
x <- x$query$pages
names(x) <- NULL
results <- lapply(x, function(x){
x$restrictiontypes <- unlist(x$restrictiontypes)
return(x)
})
return(results)
}
#' @export
parse_response.prelogintoken <- function(x){
x <- x$login$token
return(x)
}
#' @export
parse_response.login <- function(x){
x <- x$clientlogin$status == "PASS"
return(x)
}
#' @export
parse_response.actiontoken <- function(x){
x <- x$query$tokens$csrftoken
return(x)
}
#' @export
parse_response.createpage <- function(x){
x <- x$edit$result == "Success"
return(x)
} WikipediR/R/login.R 0000644 0001762 0000144 00000004053 14601320617 013577 0 ustar ligges users #' request token to start client login
#'
#' helper function to request a user login token
#'
#' @param url a URL body
#' @param user a username of a registered user
#'
#' @return a token string
#'
#' @importFrom magrittr "%>%"
#'
get_prelogin_token <- function(url, user) {
# get login token
response <- httr::modify_url(
url,
query = list(
action = "login",
lgname = user,
format = "json"
)
) %>% httr::POST(
) %>% httr::stop_for_status(
)
# parse the response, check for API errors
parsed_response <- response_parse(
response = response, out_class = "prelogintoken"
)
if(!is.null(parsed_response$error)){
stop(
"The API returned an error: ",
parsed_response$error$code,
" - ", parsed_response$error$info
)
}
parse_response(parsed_response) %>%
return()
}
#' wikimedia api user login
#'
#' Login to a wikimedia instance to trigger api requests
#' as a registered user. This function only allows the
#' very basic login with username and password. Wikimedia
#' setups that require more sophisticated login methods
#' are not supported.
#'
#' @param url a URL body
#' @param user a username of a registered user
#' @param pw the password of said user
#'
#' @return TRUE
#'
#' @export
login <- function(url, user, pw){
# get prelogin token
token <- get_prelogin_token(url, user)
# login
response <- httr::modify_url(
url,
query = list(
action = "clientlogin",
username = user,
rememberMe = 1,
loginreturnurl = url,
format = "json"
)
) %>%
httr::POST(
body = list(
logintoken = token,
password = pw
)
) %>% httr::stop_for_status(
)
# parse the response, check for API errors
parsed_response <- response_parse(
response = response, out_class = "login"
)
if(!is.null(parsed_response$error)){
stop(
"The API returned an error: ",
parsed_response$error$code,
" - ", parsed_response$error$info
)
}
parse_response(parsed_response) %>%
return()
}
WikipediR/R/recent_changes.R 0000644 0001762 0000144 00000007474 14601320617 015451 0 ustar ligges users #'@title Retrieves entries from the RecentChanges feed
#'
#'@description
#'wiki_recentchanges retrieves a stream of entries from Special:RecentChanges, with a variety of
#'associated metadata and filtering (of both entries *and* that metadata.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param properties Properties you're trying to retrieve about each entry, Options include
#'"user" (the username of the person responsible for that entry), "userid" (the userID of said
#'person), "comment" (the edit summary associated with the entry), "parsedcomment" (the same,
#'but parsed, generating HTML from any wikitext in that comment), "flags" (whether the revision
#'was 'minor' or not), "timestamp", "title" (the name of the page the entry affected), "ids"
#'(the page id, along with the old and new revision IDs when applicable) "sizes" (the size,
#'in uncompressed bytes, of the entry, and, in the case of revisions, the size of the edit
#'it displaced), "tags" (any tags associated with the revision) and "loginfo" (applicable only
#'to log entries, and consisting of log ID numbers, log types and actions, and so on) and "sha1"
#'(the SHA-1 hash of the revision text).
#'
#'@param type The type of entry you want to retrieve; can be any permutation of "edit" (edits to existing pages),
#'"external" (external actions that impact on the project - primarily wikidata changes),
#'"new" (the creation of new pages) and "log" (log entries). By default, all of these entry types
#'are included.
#'
#'@param tag Only return items with particular "tags", such as "mobile edit". NULL by
#'default.
#'
#'@param dir Should it go from newest to oldest ("newer"), or oldest to newest ("older")?
#'By default, set to "newer".
#'
#'@param limit The number of entries you'd like to return. By default, set to 50, which is
#'also the maximum number per-request for logged-out users.
#'
#'@param top Should the request only return "top" entries - in other words, the most recent
#'entry on a page? Set to FALSE by default.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@export
recent_changes <- function(language = NULL, project = NULL, domain = NULL,
properties = c("user","userid","comment",
"parsedcomment","flags","timestamp",
"title","ids","sizes","redirect",
"loginfo","tags","sha1"),
type = c("edit","external","new","log"),
tag = NULL, dir = "newer", limit = 50, top = FALSE,
clean_response = FALSE, ...) {
#Format and standardise, construct URL
type <- match.arg(arg = type, several.ok = TRUE)
type <- paste(type, collapse = "|")
properties <- match.arg(arg = properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
list = "recentchanges",
rcdir = dir,
rcprop = properties,
rctype = type,
rclimit= limit
)
if(!is.null(tag)){
query_param$rctag <- paste(tag, collapse = "|")
}
if(top){
query_param$rctoponly <- ""
}
#Query
content <- query(url, "rchanges", clean_response, query_param = query_param, ...)
#Return
return(content)
} WikipediR/R/generic_handlers.R 0000644 0001762 0000144 00000001227 14601320617 015763 0 ustar ligges users #Handles limitations on the number of unique values you can pass in.
#Takes an input set of params and a limit, and warns of the length of one
#is greather than the other, limiting the set if so, and either way,
#collapsing for incorporation into the API query.
handle_limits <- function(parameters, limit){
if(length(parameters) > limit){
warning("This option accepts ", limit, " values; you have provided ", length(parameters),
". Only the first ", limit, " will be returned.", call. = FALSE)
parameters <- paste(parameters[1:limit], collapse = "|")
} else {
parameters <- paste(parameters, collapse = "|")
}
return(parameters)
} WikipediR/R/content.R 0000644 0001762 0000144 00000031626 14604033647 014156 0 ustar ligges users #Checks for invalid revision IDs, warns if they're found.
invalid_revs <- function(parsed_response){
if(!is.null(parsed_response$query$badrevids)){
warning("This request contained ",length(parsed_response$query$badrevids)," invalid revisionID(s)", call. = FALSE)
}
return(invisible())
}
#'@title Retrieve the page content of a random MediaWiki page
#'
#'@description
#'wiki_page retrieves the DOM of a particular MediaWiki page,
#'as a HTML blob inside a JSON object.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param namespaces The namespaces to consider pages from. By default, pages from any namespace are
#'considered; alternately, a numeric vector of accepted namespaces (which are described
#'\href{https://www.mediawiki.org/wiki/Manual:Namespace#Built-in_namespaces}{here}) can be
#'provided, and only pages within those namespaces will be considered.
#'
#'@param as_wikitext whether to retrieve the wikimarkup (TRUE) or the HTML (FALSE).
#'Set to FALSE by default.
#'
#'@param limit the number of pages to return. 1 by default.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@seealso \code{\link{page_content}} for retrieving the content of a specific page,
#'\code{\link{revision_diff}} for retrieving 'diffs' between revisions,
#'\code{\link{revision_content}} for retrieving the text of specified revisions.
#'
#'@examples
#'\dontrun{
#'#A page from Wikipedia
#'wp_content <- random_page("en","wikipedia")
#'
#'#A page from the mainspace on Wikipedia
#'wp_article_content <- random_page("en","wikipedia", namespaces = 0)
#'}
#'@export
random_page <- function(language = NULL, project = NULL, domain = NULL,
namespaces = NULL, as_wikitext = FALSE, limit = 1,
clean_response = FALSE, ...){
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
list = "random",
rnlimit = limit
)
if(!is.null(namespaces)){
query_param$rnnamespace <- paste(namespaces, collapse = "|")
}
pages <- query(url, NULL, FALSE, query_param = query_param)$query$random
return(lapply(pages, function(page, language, project, domain, page_name, as_wikitext,
clean_response, ...){
content <- page_content(language = language, project = project, domain = domain,
page_name = page$title, as_wikitext = as_wikitext,
clean_response = clean_response, ...)
content$parse$text <- content$parse$text[[1]]
return(content$parse)
}, language = language, project = project, domain = domain, as_wikitext = as_wikitext,
clean_response = clean_response, ...))
}
#'@title Retrieves MediaWiki page content
#'
#'@description
#'wiki_page retrieves the DOM of a particular MediaWiki page,
#'as a HTML blob inside a JSON object.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param page_name The title of the page you want to retrieve
#'
#'@param page_id the pageID of the page you want to retrieve. Set to NULL by default,
#'and an alternative to page_name; if both are provided, page_id will be used.
#'
#'@param as_wikitext whether to retrieve the wikimarkup (TRUE) or the HTML (FALSE).
#'Set to FALSE by default.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@seealso \code{\link{revision_diff}} for retrieving 'diffs' between revisions,
#'\code{\link{revision_content}} for retrieving the text of specified revisions.
#'
#'@examples
#'\dontrun{
#'#Content from a Wikimedia project
#'wp_content <- page_content("en","wikipedia", page_name = "Aaron Halfaker")
#'
#'#Content by ID
#'wp_content <- page_content("en", "wikipedia", page_id = 12)
#'
#'#Content from a non-Wikimedia project
#'rw_content <- page_content(domain = "rationalwiki.org", page_name = "New Age")
#'}
#'@export
page_content <- function(language = NULL, project = NULL, domain = NULL,
page_name, page_id = NULL, as_wikitext = FALSE, clean_response = FALSE, ...){
#Format and construct URL.
if(as_wikitext){
properties <- "wikitext|revid"
} else {
properties <- "text|revid"
}
properties <- paste(properties, collapse = "|")
url <- url_gen(language, project, domain)
query_param <- list(
action = "parse",
prop = properties
)
if(!is.null(page_id)){
query_param$pageid <- handle_limits(page_id, 1)
} else {
query_param$page <- handle_limits(page_name, 1)
}
#Run
content <- query(url, "pcontent", clean_response, query_param = query_param, ...)
#Return
return(content)
}
#'@title Retrieves MediaWiki revisions
#'
#'@description
#'Retrieves the content of a provided list of revisions from whichever MediaWiki instance you're
#'querying. Returns as wikimarkup.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param revisions The revision IDs of each desired revision.
#'
#'@param properties Properties you're trying to retrieve about that revision, should you want to;
#'options include "ids" (the revision ID of the revision...which is pointless),
#'"flags" (whether the revision was 'minor' or not), "timestamp" (the timestamp of the revision),
#'"user" (the username of the person who made that revision), "userid"
#'(the userID of the person who made the revision),
#'"size" (the size, in uncompressed bytes, of the revision), "sha1" (the SHA-1 hash of
#'the revision text), "contentmodel" (the content model of the page, usually "wikitext"),
#'"comment" (the revision summary associated with the revision), "parsedcomment" (the same,
#'but parsed, generating HTML from any wikitext in that comment), "tags" (any tags associated
#'with the revision) and "flagged" (the revision's status under Flagged Revisions).
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@seealso
#'\code{\link{revision_diff}} for diffs between revisions,
#'and \code{\link{page_content}} for the content a specific page currently has.
#'
#'@examples
#'\dontrun{
#'#Revision content from a Wikimedia project
#'wp_content <- revision_content("en","wikipedia", revisions = 552373187)
#'
#'#Revision content from a non-Wikimedia project
#'rw_content <- revision_content(domain = "rationalwiki.org", revisions = 88616)
#'}
#'@export
revision_content <- function(language = NULL, project = NULL, domain = NULL,
revisions, properties = c("content","ids","flags","timestamp",
"user","userid","size",
"sha1","contentmodel","comment",
"parsedcomment","tags"),
clean_response = FALSE, ...){
#Format, construct URL.
properties <- match.arg(arg = properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
revisions <- handle_limits(revisions, 50)
url <- url_gen(language, project, domain)
query_param <- list(
rvcontentformat = "text/x-wiki",
action = "query",
prop = "revisions",
rvprop = properties,
revids = revisions
)
#Run
content <- query(url, "rcontent", clean_response, query_param = query_param, ...)
#Check for invalid RevIDs
invalid_revs(content)
#Return
return(content)
}
#'@title Generates a "diff" between a pair of revisions
#'
#'@description
#'revision_diff generates a diff between two revisions in a MediaWiki page.
#'This is provided as an XML-parsable blob inside the returned JSON object.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param revisions The revision IDs of each "start" revision.
#'
#'@param properties Properties you're trying to retrieve about that revision, should you want to;
#'options include "ids" (the revision ID of the revision...which is pointless),
#'"flags" (whether the revision was 'minor' or not), "timestamp","user" (the username of the person
#'who made that revision), "userid" (the userID of the person who made the revision),
#'"size" (the size, in uncompressed bytes, of the revision), "sha1" (the SHA-1 hash of
#'the revision text), "contentmodel" (the content model of the page, usually "wikitext"),
#'"comment" (the revision summary associated with the revision), "parsedcomment" (the same,
#'but parsed, generating HTML from any wikitext in that comment), "tags" (any tags associated
#'with the revision) and "flagged" (the revision's status under Flagged Revisions).
#'
#'@param direction The direction you want the diff to go in from the revisionID you have provided.
#'Options are "prev" (compare to the previous revision on that page), "next" (compare to the next
#'revision on that page) and "cur" (compare to the current, extant version of the page).
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@section Warnings:
#'
#'MediaWiki's API is deliberately designed to restrict users' ability to make computing-intense requests
#'- such as diff computation. As a result, the API only allows requests for one uncached diff in
#'each request. If you ask for multiple diffs, some uncached and some cached, you will be provided
#' with the cached diffs, one of the uncached diffs, and a warning.
#'
#'If you're going to be asking for a lot of diffs, some of which may not be cached, it may be more
#'sensible to retrieve the revisions themselves using \code{\link{revision_content}} and compute the
#'diffs yourself.
#'
#'@seealso \code{\link{page_content}} for retrieving the current content of a specific page, and
#'\code{\link{revision_content}} for retrieving the text of specific revisions.
#'
#'@examples
#'\dontrun{
#'#Wikimedia diff
#'wp_diff <- revision_diff("en","wikipedia", revisions = 552373187, direction = "next")
#'
#'#Non-Wikimedia diff
#'rw_diff <- revision_diff(domain = "rationalwiki.org", revisions = 88616, direction = "next")
#'}
#'@export
revision_diff <- function(language = NULL, project = NULL, domain = NULL,
revisions, properties = c("ids","flags","timestamp","user","userid","size",
"sha1","contentmodel","comment","parsedcomment",
"tags","flagged"),
direction, clean_response = FALSE, ...){
#Check and construct URL
direction <- match.arg(direction, c("prev","next","cur"))
properties <- match.arg(properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
revisions <- handle_limits(revisions, 50)
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
prop = "revisions",
rvprop = properties,
rvdiffto = direction,
rvcontentformat = "text/css",
revids = revisions
)
#Retrieve the content, check for invalid RevIDs and uncached diffs,
#return.
content <- query(url, "rdiff", clean_response, query_param = query_param, ...)
invalid_revs(content)
if(sum(grepl(x = names(unlist(content)), pattern = "diff.notcached"))){
warning("This request contained uncached diffs; these will not be returned", call. = FALSE)
}
return(content)
} WikipediR/R/create_content.R 0000644 0001762 0000144 00000006022 14601320617 015462 0 ustar ligges users #' request token for api action as signed in user
#'
#' helper function to request a user action token
#'
#' @param url a URL body
#'
#' @return a token string
#'
#' @importFrom magrittr "%>%"
#'
get_action_token <- function(url) {
# get login token
response <- httr::modify_url(
url,
query = list(
action = "query",
meta = "tokens",
format = "json"
)
) %>% httr::GET(
) %>% httr::stop_for_status(
)
# parse the response, check for API errors
parsed_response <- response_parse(
response = response, out_class = "actiontoken"
)
if(!is.null(parsed_response$error)){
stop(
"The API returned an error: ",
parsed_response$error$code,
" - ", parsed_response$error$info
)
}
parse_response(parsed_response) %>%
return()
}
#' wikimedia api page creation
#'
#' Create pages or category-pages on a wikimedia instance.
#'
#' @param url a URL body
#' @param p_title vector with page title strings of new pages
#' @param p_text vector with page content strings of new pages
#' @param category switch to decide, if the pages should be
#' created as category-pages
#'
#' @return TRUE
#'
#' @export
create_pages <- function(url, p_title, p_text, category = FALSE){
if(length(p_title) != length(p_text)) {
stop(
"The length of the vectors p_title and p_text
is not equal."
)
}
# get action token
token <- NA
token <- get_action_token(url)
if(token %>% is.na){
stop("Problems with the action token request.")
}
# create page(s) (with progress bar)
pb <- utils::txtProgressBar(
min = 0, max = length(p_title), style = 3
)
res <- c()
for (i in 1:length(p_title)) {
res[i] <- create_page(
url, p_title[i], p_text[i], category, token
)
utils::setTxtProgressBar(pb, i)
}
close(pb)
all(res) %>%
return()
}
#' wikimedia api page creation (single pages)
#'
#' helper function to do the actual api requests for page
#' and category-page creation
#'
#' @param url a URL body
#' @param p_title page title string of new page
#' @param p_text page content string of new page
#' @param category switch to decide, if the page should be
#' created as category-page
#' @param token action token to perform the request
#'
#' @return TRUE
#'
create_page <- function(url, p_title, p_text, category, token){
# create page
response <- httr::modify_url(
url,
query = list(
action = "edit",
title = ifelse(
category,
paste0("Category:", p_title),
p_title
),
text = p_text,
format = "json"
)
) %>% httr::POST(
body = list(token = token)
) %>% httr::stop_for_status(
)
# parse the response, check for API errors
parsed_response <- response_parse(
response = response, out_class = "createpage"
)
if(!is.null(parsed_response$error)){
stop(
"The API returned an error: ",
parsed_response$error$code,
" - ", parsed_response$error$info
)
}
parse_response(parsed_response) %>%
return()
}
WikipediR/R/query.R 0000644 0001762 0000144 00000004711 14601320617 013635 0 ustar ligges users #General functions and error handlers for
#generic queries and query construction.
#'@title base query function
#'@description not designed to be used by anyone except
#'a third-party reuser package, such as WikidataR
#'@param url a URL body
#'@param out_class the class to set on the output object; used within
#'WikidataR to indicate what response-cleaning method should be applied.
#'
#'@param clean_response whether to clean the response, using the method assigned
#'by out_class, or not.
#'
#'@param query_param query parameters
#'
#'@param ... further arguments to httr's GET.
#'@export
query <- function(url, out_class, clean_response = FALSE, query_param = list(), ...){
# Common query parameters
if(is.null(query_param$format)) {
query_param$format <- "json"
}
args <- list(...)
if(length(args) > 0 && "config" %in% class(args[[1]]) && "useragent" %in% names(args[[1]])){
response <- httr::GET(url, query = query_param, ...)
} else {
response <- httr::GET(url, query = query_param, httr::user_agent("WikipediR - https://github.com/Ironholds/WikipediR"), ...)
}
#Check the validity of the response
httr::stop_for_status(response)
#Parse the response, check for API errors, return
parsed_response <- response_parse(response = response, out_class = out_class)
if(!is.null(parsed_response$error)){
stop("The API returned an error: ", parsed_response$error$code,
" - ", parsed_response$error$info)
}
if(clean_response){
parsed_response <- parse_response(parsed_response)
}
return(parsed_response)
}
#Constructor for the URL
url_gen <- function(language, project, domain = NULL, ...){
if(is.null(domain)){
#Commons and Wikispecies have different URL formats, so those have to be handled in a hinky way.
if(project %in% c("commons","species")){
url <- sprintf("https://%s.wikimedia.org/w/api.php", project)
} else {
url <- sprintf("https://%s.%s.org/w/api.php", language, project)
}
} else {
url <- sprintf("http://%s/w/api.php", domain)
}
#Return
return(url)
}
#'@importFrom httr content
#'@importFrom jsonlite fromJSON
response_parse <- function(response, out_class){
#Convert it into a character vector
response_text <- httr::content(x = response, as = "text")
#From there, turn it into an R object from JSON
parsed_text <- jsonlite::fromJSON(txt = response_text, simplifyVector = FALSE)
class(parsed_text) <- out_class
#Return
return(parsed_text)
}
WikipediR/R/WikipediR.R 0000644 0001762 0000144 00000001106 14601321137 014350 0 ustar ligges users #' @title A client library for MediaWiki's API
#' @name WikipediR
#' @description This package provides functions for accessing the MediaWiki API, either for
#' Wikimedia projects or any other MediaWiki instance. For more information, see the
#' \href{https://CRAN.R-project.org/package=WikipediR/vignettes/WikipediR.html}{vignette}.
#'
#' @seealso The \href{https://CRAN.R-project.org/package=WikipediR/vignettes/WikipediR.html}{package vignette}.
#' @importFrom httr GET user_agent stop_for_status
#' @importFrom utils URLencode
#' @aliases WikipediR WikipediR-package
"_PACKAGE" WikipediR/R/user_info.R 0000644 0001762 0000144 00000021020 14604033757 014462 0 ustar ligges users #Missing user handler for user_information/user_contributions
missing_users <- function(parsed_response){
#Check for missing values
names_to_check <- names(unlist(parsed_response))
missing_names <- sum(grepl(x = names_to_check, pattern = "users\\.missing"))
if(missing_names){
warning(missing_names," of the provided usernames did not exist", call. = FALSE)
}
return(invisible())
}
#'@title Retrieve user contributions
#'
#'@description Retrieves metadata associated with the most recent contributions by a
#'specified user.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param username The username of the user whose contributions you want to retrieve.
#'Due to limitations at the API end, you can only retrieve edits for one user at a time.
#'
#'@param properties The metadata you want associated with each edit. Potential metadata includes "ids"
#'(the revision ID of the revision, which can be passed into \code{\link{revision_content}}),
#'"title" (the name of the page that was edited), "timestamp", "comment" (the edit summary associated
#'with the revision), "parsedcomment" (the same, but parsed, generating HTML from any wikitext
#'in that comment), "size" (the size, in uncompressed bytes, of the edit), "sizediff" (the size
#'delta between this edit, and the last edit to the page), "flags" (whether the revision was
#''minor' or not), and "tags" (any tags associated with the revision).
#'
#'@param mainspace A boolean flag; FALSE retrieves all of the most recent contributions, while
#'TRUE limits the retrieved contributions to those in the 'mainspace' - in other words, edits to
#'actual articles. Set to FALSE by default
#'
#'@param limit The number of edits to be retrieved. 50 is the maximum for logged-out API users,
#'and putting in more than 50 will generate a warning.
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@seealso \code{\link{user_information}} for information about a specific user (or group of users),
#'and \code{recent_changes} for non-user-specific recent actions.
#'
#'@examples
#'\dontrun{
#'#Retrieve the timestamps of a user's recent contributions to the English-language Wikipedia
#'contribs <- user_contributions("en", "wikipedia", username = "Ironholds",
#' properties = "timestamp")
#'
#'#Retrieve the timestamps of a user's recent contributions to a non-Wikimedia wiki.
#'rw_contribs <- user_contributions(domain = "rationalwiki.org", username = "David Gerard",
#' properties = "ids", limit = 1)
#'}
#'@export
user_contributions <- function(language = NULL, project = NULL, domain = NULL,
username, properties = c("ids", "title", "timestamp",
"comment", "parsedcomment", "size",
"sizediff", "flags", "tags"),
mainspace = FALSE, limit = 50, clean_response = FALSE,
...){
#Perform checks, construct URL
properties <- match.arg(properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
username <- handle_limits(username, 1)
url <- url_gen(language, project, domain)
query_param <- list(
action = "query",
list = "usercontribs",
uclimit = limit,
ucuser = username,
ucprop = properties
)
#If only article contributions are desired, note that.
if(mainspace){
query_param$ucnamespace <- 0
}
#Get, check and return
contribs_content <- query(url, "ucontribs", clean_response, query_param = query_param, ...)
missing_users(contribs_content)
return(contribs_content)
}
#'@title Retrieve user information
#'
#'@description
#'Retrieves information about a user, or set of users, from the MediaWiki API,
#'including registration date, gender and editcount.
#'
#'@param language The language code of the project you wish to query,
#'if appropriate.
#'
#'@param project The project you wish to query ("wikiquote"), if appropriate.
#'Should be provided in conjunction with \code{language}.
#'
#'@param domain as an alternative to a \code{language} and \code{project} combination,
#'you can also provide a domain ("rationalwiki.org") to the URL constructor, allowing
#'for the querying of non-Wikimedia MediaWiki instances.
#'
#'@param user_names The username(s) of the users you want information on - this should be provided
#'as a vector. There is a hard limit of 50 distinct users per query, set by MediaWiki's API;
#'in the event that you go over this, a warning will be issued and the query will only be
#'performed for the first 50 names in the vector.
#'
#'@param properties The user properties you're interested in. Applicable properties are
#'"blockinfo" (details about the user's block, if they are currently blocked), "groups"
#'(the user groups the user is a member of), "implicitgroups" (groups they are a member of
#'through inheritance, as a result of membership in other groups), "rights" (what permissions
#'their group membership grants them), "editcount" (how many non-deleted edits they have),
#'"registration" (the date when they registered), "emailable" (whether they are contactable
#'through Special:EmailUser) and "gender" (their provided gender).
#'
#'@param clean_response whether to do some basic sanitising of the resulting data structure.
#'Set to FALSE by default.
#'
#'@param ... further arguments to pass to httr's GET.
#'
#'@section Warnings:
#'There are a few caveats with the data provided by \code{user_information}, mostly stemming from
#'historical inconsistencies and peculiarities in MediaWiki.
#'
#'\code{groups} and \code{implicitgroups} gives you the user's permissions and group membership
#'on the project you are querying, not their membership on all projects - while you can find out
#'if "Ironholds" is not a sysop on, say, enwiki, that doesn't mean they aren't a sysop elsewhere
#'- there is no universal, API-accessible user groups listing.
#'
#'As an extension of the lack of centrality in Wikimedia's infrastructure, \code{registration}
#'tells you the date their account was created on the wiki you are querying. If they initially
#'registered on that wiki, this is accurate - if they registered on a different wiki,
#'this instead reflects the date and time that they first visited the wiki you're querying
#'while logged-in. For users registered before 2006, when registration logging was introduced,
#'the \code{registration} value represents not when they first registered, but when their first
#'edit was, since that was used as an estimator for existing accounts when the field was first
#'populated.
#'
#'@seealso \code{\link{user_contributions}} for retrieving recent contributions made by
#'a particular user.
#'
#'@examples
#'\dontrun{
#'#Retrieving information from a Wikimedia project
#'user_info <- user_information("en", "wikipedia", user_names = "David Gerard",
#' properties = "registration")
#'
#'#Non-Wikimedia projects
#'user_info <- user_information(domain = "rationalwiki.org", user_names = "David Gerard",
#' properties = "registration")
#'}
#'@export
user_information <- function(language = NULL, project = NULL, domain = NULL,
user_names, properties = c("blockinfo","groups","implicitgroups",
"rights","editcount","registration",
"emailable","gender"),
clean_response = FALSE, ...){
#Check, construct URL
properties <- match.arg(properties, several.ok = TRUE)
properties <- paste(properties, collapse = "|")
user_names <- handle_limits(user_names, 50)
url <- url_gen(language, project, domain)
query_param = list(
action = "query",
list = "users",
usprop = properties,
ususers = user_names
)
#Retrieve the content, check it, return.
user_content <- query(url, "uinfo", clean_response, query_param = query_param, ...)
missing_users(user_content)
return(user_content)
} WikipediR/MD5 0000644 0001762 0000144 00000004655 14604037454 012472 0 ustar ligges users 8a1187bc162bec655cd593c2e9780e60 *DESCRIPTION
1d9678dbfe1732b5d2c521e07b2ceef0 *LICENSE
1aa74073b35fca87f224628c2c05b2df *NAMESPACE
4f85f4bc8d56477833d6e5518a0b93a5 *NEWS
7abe9dbaf67fb6edb3d06f7e431457d2 *R/WikipediR.R
5067ad5283bcd433dba907eea0796562 *R/categories.R
73503b4191ac966ca97186c30c9e4306 *R/content.R
4bcf6a5b0429c26143f57a5754cc181a *R/create_content.R
3310e6008ce3c185829aa4fe5087baf2 *R/generic_handlers.R
83bb45ed7646a35c5a067aa48b86af4b *R/login.R
77d1566ddebc8dd53c5d4d5ffe44be99 *R/metadata.R
b28c261d06ea9722155ba633658d76c7 *R/parse.R
7d6b7516a10fa66ee830d20dcde26594 *R/query.R
9e782c267ce639c3c333b567b93bf8d8 *R/recent_changes.R
cbf979fa8a25eabe82bdcec7584afebc *R/user_info.R
4ff28c992f97f12f9d2d1a584a1f7996 *README.md
8427e4d2e5c7326c57c871aff38d68cc *build/vignette.rds
e5214179299ab80dd7b16e98346cd8f1 *inst/doc/WikipediR.Rmd
8bd185cd6e77ed314d9c518ca3418dcc *inst/doc/WikipediR.html
61ea5b5e2197b3e95cda09ba8026bc8b *man/WikipediR.Rd
bc835670547b03a522697e7bbc37bc91 *man/categories_in_page.Rd
87376fcad8f502c1e5096509c7dce3ea *man/create_page.Rd
a6047d320bc1e72988140343ec07c28f *man/create_pages.Rd
835829e6cae2db0512fb8642c605e5e1 *man/get_action_token.Rd
55b22500bc5ad9b135e3d9d0049efa15 *man/get_prelogin_token.Rd
43294b43d3f0beb196af50d47cd0b5d1 *man/login.Rd
b5f06d794c1980deb7a2d13d52435a91 *man/page_backlinks.Rd
08e4b35bfefb7e8f7d76be7006ac876e *man/page_content.Rd
baf00e4335d0d7bc0d02305af0c19e35 *man/page_external_links.Rd
646476cdcfb2cb2fff2f7f4a60c6c82c *man/page_info.Rd
7d79d48d4f581db9d41c98eb1bbf5657 *man/page_links.Rd
eafb1a42cdec850391c0530a1c3daf00 *man/pages_in_category.Rd
f488e4051f977b098ea337ff8479542b *man/parse_response.Rd
a81ee6a673391569ebfcf7869f7a59fa *man/query.Rd
2f9ff7f1dc98a584748f2744a3d56190 *man/random_page.Rd
c2c0e8414545359e45ef101cd8531991 *man/recent_changes.Rd
36b024ad0f73ebbe9d7195b37d2e59d6 *man/revision_content.Rd
ca9c157a8432d70d4043d983cdf6a54d *man/revision_diff.Rd
e3457b8cba3674b546fa670db8d11073 *man/user_contributions.Rd
9a4f5ff01273099b0a5691383d4c19c7 *man/user_information.Rd
0258bf928f7abd1db4cf997cbff76363 *tests/testthat.R
700d7e68d80d76edb1f345f679aef317 *tests/testthat/test_category_retrieval.R
e5113c7e7fe442a49f79492861102792 *tests/testthat/test_content_retrieval.R
82c679290bcde1b558517bf945cb41ae *tests/testthat/test_metadata_retrieval.R
fe113c80acc16e643f44d946c03927c1 *tests/testthat/test_recent_changes.R
e5214179299ab80dd7b16e98346cd8f1 *vignettes/WikipediR.Rmd
WikipediR/inst/ 0000755 0001762 0000144 00000000000 14604034112 013111 5 ustar ligges users WikipediR/inst/doc/ 0000755 0001762 0000144 00000000000 14604034112 013656 5 ustar ligges users WikipediR/inst/doc/WikipediR.html 0000644 0001762 0000144 00000010354 14604034112 016436 0 ustar ligges users
#WikipediR: A MediaWiki API client library Many websites run on versions of MediaWiki, most prominently Wikipedia and its sister sites. WikipediR is an API client library that allows you to conveniently make requests for content and associated metadata against MediaWiki instances.
“content” can mean a lot of different things - but mostly, we mean the text of an article, either its current version or any previous versions. Current versions can be retrieved using page_content
, which provides both HTML and wikitext as possible output formats. Older, individual revisions can be retrieved with revision_content
. These functions also return a range of possible metadata about the revisions or articles in question.
Diffs between revisions can be generated using revision_diff
, while individual ''elements'' of a page's content - particularly links - can be extracted using page_links
, page_backlinks
, and page_external_links
. And if the interest is in changes to content, rather than content itself, recent_changes
can be used to grab a slice of a project's Special:RecentChanges feed.
Page-related information can be accessed using page_info
, while categories that a page possesses can be retrieved with categories_in_page
- the inverse of this operation (what pages are in a particular category?) uses pages_in_category
.
User-related info can be accessed with user_information
, while user_contributions
allows access to recent contributions by a particular user: this can be conveniently linked up with revision_content
, mentioned above, to retrieve the content of the last N edits by a particular editor, or metadata about those edits.
page\_content
, which provides both HTML and wikitext as possible output formats. Older, individual revisions can be retrieved with revision\_content
. These functions also return a range of possible metadata about the revisions or articles in question.
Diffs between revisions can be generated using revision\_diff
, while individual ''elements'' of a page's content - particularly links - can be extracted using page\_links
, page\_backlinks
, and page\_external\_links
. And if the interest is in changes to content, rather than content itself, recent\_changes
can be used to grab a slice of a project's Special:RecentChanges feed.
## Retrieving metadata
Page-related information can be accessed using page\_info
, while categories that a page possesses can be retrieved with categories\_in\_page
- the inverse of this operation (what pages are in a particular category?) uses pages\_in\_category
.
User-related info can be accessed with user\_information
, while user\_contributions
allows access to recent contributions by a particular user: this can be conveniently linked up with revision\_content
, mentioned above, to retrieve the content of the last N edits by a particular editor, or metadata about those edits.