irlba/ 0000755 0001760 0000144 00000000000 14316626071 011401 5 ustar ripley users irlba/NAMESPACE 0000644 0001760 0000144 00000000564 13452171265 012625 0 ustar ripley users # Generated by roxygen2: do not edit by hand
S3method(summary,irlba_prcomp)
export(irlba)
export(partial_eigen)
export(prcomp_irlba)
export(ssvd)
export(svdr)
import(Matrix)
importFrom(methods,slot)
importFrom(methods,slotNames)
importFrom(stats,prcomp)
importFrom(stats,rnorm)
importFrom(stats,sd)
importFrom(stats,var)
useDynLib(irlba, .registration=TRUE, .fixes="C_")
irlba/README.md 0000644 0001760 0000144 00000012445 14153272154 012664 0 ustar ripley users # irlba
Implicitly-restarted Lanczos methods for fast truncated singular value
decomposition of sparse and dense matrices (also referred to as partial SVD).
IRLBA stands for Augmented, Implicitly Restarted Lanczos
Bidiagonalization Algorithm. The package provides the following
functions (see help on each for details and examples).
* `irlba()` partial SVD function
* `ssvd()` l1-penalized matrix decompoisition for sparse PCA (based on Shen and Huang's algorithm)
* `prcomp_irlba()` principal components function similar to the `prcomp` function in stats package for computing the first few principal components of large matrices
* `svdr()` alternate partial SVD function based on randomized SVD (see also the [rsvd](https://cran.r-project.org/package=rsvd) package by N. Benjamin Erichson for an alternative implementation)
* `partial_eigen()` a very limited partial eigenvalue decomposition for symmetric matrices (see the [RSpectra](https://cran.r-project.org/package=RSpectra) package for more comprehensive truncated eigenvalue decomposition)
Help documentation for each function includes extensive documentation and
examples. Also see the package vignette, `vignette("irlba", package="irlba")`.
An overview web page is here: https://bwlewis.github.io/irlba/.
## New in 2.3.2
- Fixed a regression in `prcomp_irlba()` discovered by Xiaojie Qiu, see https://github.com/bwlewis/irlba/issues/25, and other related problems reported in https://github.com/bwlewis/irlba/issues/32.
- Added rchk testing to pre-CRAN submission tests.
- Fixed a sign bug in `ssvd()` found by Alex Poliakov.
## What's new in Version 2.3.1?
- Fixed an `irlba()` bug associated with centering (PCA), see https://github.com/bwlewis/irlba/issues/21.
- Fixed `irlba()` scaling to conform to `scale`, see https://github.com/bwlewis/irlba/issues/22.
- Improved `prcomp_irlba()` from a suggestion by N. Benjamin Erichson, see https://github.com/bwlewis/irlba/issues/23.
- Significanty changed/improved `svdr()` convergence criterion.
- Added a version of Shen and Huang's Sparse PCA/SVD L1-penalized matrix decomposition (`ssvd()`).
- Fixed valgrind errors.
## Deprecated features
I will remove `partial_eigen()` in a future version. As its documentation
states, users are better off using the RSpectra package for eigenvalue
computations (although not generally for singular value computations).
The `mult` argument is deprecated and will be removed in a future version. We
now recommend simply defining a custom class with a custom multiplcation
operator. The example below illustrates the old and new approaches.
```{r}
library(irlba)
set.seed(1)
A <- matrix(rnorm(100), 10)
# ------------------ old way ----------------------------------------------
# A custom matrix multiplication function that scales the columns of A
# (cf the scale option). This function scales the columns of A to unit norm.
col_scale <- sqrt(apply(A, 2, crossprod))
mult <- function(x, y)
{
# check if x is a vector
if (is.vector(x))
{
return((x %*% y) / col_scale)
}
# else x is the matrix
x %*% (y / col_scale)
}
irlba(A, 3, mult=mult)$d
## [1] 1.820227 1.622988 1.067185
# Compare with:
irlba(A, 3, scale=col_scale)$d
## [1] 1.820227 1.622988 1.067185
# Compare with:
svd(sweep(A, 2, col_scale, FUN=`/`))$d[1:3]
## [1] 1.820227 1.622988 1.067185
# ------------------ new way ----------------------------------------------
setClass("scaled_matrix", contains="matrix", slots=c(scale="numeric"))
setMethod("%*%", signature(x="scaled_matrix", y="numeric"), function(x ,y) x@.Data %*% (y / x@scale))
setMethod("%*%", signature(x="numeric", y="scaled_matrix"), function(x ,y) (x %*% y@.Data) / y@scale)
a <- new("scaled_matrix", A, scale=col_scale)
irlba(a, 3)$d
## [1] 1.820227 1.622988 1.067185
```
We have learned that using R's existing S4 system is simpler, easier, and more
flexible than using custom arguments with idiosyncratic syntax and behavior.
We've even used the new approach to implement distributed parallel matrix
products for very large problems with amazingly little code.
## Wishlist / help wanted...
- More Matrix classes supported in the fast code path
- Help improving the solver for singular values in tricky cases (basically, for ill-conditioned problems and especially for the smallest singular values); in general this may require a combination of more careful convergence criteria and use of harmonic Ritz values; Dmitriy Selivanov has proposed alternative convergence criteria in https://github.com/bwlewis/irlba/issues/29 for example.
## References
* Baglama, James, and Lothar Reichel. "Augmented implicitly restarted Lanczos bidiagonalization methods." SIAM Journal on Scientific Computing 27.1 (2005): 19-42.
* Halko, Nathan, Per-Gunnar Martinsson, and Joel A. Tropp. "Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions." (2009).
* Shen, Haipeng, and Jianhua Z. Huang. "Sparse principal component analysis via regularized low rank matrix approximation." Journal of multivariate analysis 99.6 (2008): 1015-1034.
* Witten, Daniela M., Robert Tibshirani, and Trevor Hastie. "A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis." Biostatistics 10.3 (2009): 515-534.
irlba/man/ 0000755 0001760 0000144 00000000000 13430560161 012145 5 ustar ripley users irlba/man/ssvd.Rd 0000644 0001760 0000144 00000021733 13430560161 013421 0 ustar ripley users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/ssvd.R
\name{ssvd}
\alias{ssvd}
\title{Sparse regularized low-rank matrix approximation.}
\usage{
ssvd(x, k = 1, n = 2, maxit = 500, tol = 0.001, center = FALSE,
scale. = FALSE, alpha = 0, tsvd = NULL, ...)
}
\arguments{
\item{x}{A numeric real- or complex-valued matrix or real-valued sparse matrix.}
\item{k}{Matrix rank of the computed decomposition (see the Details section below).}
\item{n}{Number of nonzero components in the right singular vectors. If \code{k > 1},
then a single value of \code{n} specifies the number of nonzero components
in each regularized right singular vector. Or, specify a vector of length
\code{k} indicating the number of desired nonzero components in each
returned vector. See the examples.}
\item{maxit}{Maximum number of soft-thresholding iterations.}
\item{tol}{Convergence is determined when \eqn{\|U_j - U_{j-1}\|_F < tol}{||U_j - U_{j-1}||_F < tol}, where \eqn{U_j} is the matrix of estimated left regularized singular vectors at iteration \eqn{j}.}
\item{center}{a logical value indicating whether the variables should be
shifted to be zero centered. Alternately, a centering vector of length
equal the number of columns of \code{x} can be supplied. Use \code{center=TRUE}
to perform a regularized sparse PCA.}
\item{scale.}{a logical value indicating whether the variables should be
scaled to have unit variance before the analysis takes place.
Alternatively, a vector of length equal the number of columns of \code{x} can be supplied.
The value of \code{scale} determines how column scaling is performed
(after centering). If \code{scale} is a numeric vector with length
equal to the number of columns of \code{x}, then each column of \code{x} is
divided by the corresponding value from \code{scale}. If \code{scale} is
\code{TRUE} then scaling is done by dividing the (centered) columns of
\code{x} by their standard deviations if \code{center=TRUE}, and the
root mean square otherwise. If \code{scale} is \code{FALSE}, no scaling is done.
See \code{\link{scale}} for more details.}
\item{alpha}{Optional scalar regularization parameter between zero and one (see Details below).}
\item{tsvd}{Optional initial rank-k truncated SVD or PCA (skips computation if supplied).}
\item{...}{Additional arguments passed to \code{\link{irlba}}.}
}
\value{
A list containing the following components:
\itemize{
\item{u} {regularized left singular vectors with orthonormal columns}
\item{d} {regularized upper-triangluar projection matrix so that \code{x \%*\% v == u \%*\% d}}
\item{v} {regularized, sparse right singular vectors with columns of unit norm}
\item{center, scale} {the centering and scaling used, if any}
\item{lambda} {the per-column regularization parameter found to obtain the desired sparsity}
\item{iter} {number of soft thresholding iterations}
\item{n} {value of input parameter \code{n}}
\item{alpha} {value of input parameter \code{alpha}}
}
}
\description{
Estimate an \eqn{{\ell}1}{l1}-penalized
singular value or principal components decomposition (SVD or PCA) that introduces sparsity in the
right singular vectors based on the fast and memory-efficient
sPCA-rSVD algorithm of Haipeng Shen and Jianhua Huang.
}
\details{
The \code{ssvd} function implements a version of an algorithm by
Shen and Huang that computes a penalized SVD or PCA that introduces
sparsity in the right singular vectors by solving a penalized least squares problem.
The algorithm in the rank 1 case finds vectors \eqn{u, w}{u, w} that minimize
\deqn{\|x - u w^T\|_F^2 + \lambda \|w\|_1}{||x - u w^T||_F^2 + lambda||w||_1}
such that \eqn{\|u\| = 1}{||u|| = 1},
and then sets \eqn{v = w / \|w\|}{v = w / ||w||} and
\eqn{d = u^T x v}{d = u^T x v};
see the referenced paper for details. The penalty \eqn{\lambda}{lambda} is
implicitly determined from the specified desired number of nonzero values \code{n}.
Higher rank output is determined similarly
but using a sequence of \eqn{\lambda}{lambda} values determined to maintain the desired number
of nonzero elements in each column of \code{v} specified by \code{n}.
Unlike standard SVD or PCA, the columns of the returned \code{v} when \code{k > 1} may not be orthogonal.
}
\note{
Our \code{ssvd} implementation of the Shen-Huang method makes the following choices:
\enumerate{
\item{The l1 penalty is the only available penalty function. Other penalties may appear in the future.}
\item{Given a desired number of nonzero elements in \code{v}, value(s) for the \eqn{\lambda}{lambda}
penalty are determined to achieve the sparsity goal subject to the parameter \code{alpha}.}
\item{An experimental block implementation is used for results with rank greater than 1 (when \code{k > 1})
instead of the deflation method described in the reference.}
\item{The choice of a penalty lambda associated with a given number of desired nonzero
components is not unique. The \code{alpha} parameter, a scalar between zero and one,
selects any possible value of lambda that produces the desired number of
nonzero entries. The default \code{alpha = 0} selects a penalized solution with
largest corresponding value of \code{d} in the 1-d case. Think of \code{alpha} as
fine-tuning of the penalty.}
\item{Our method returns an upper-triangular matrix \code{d} when \code{k > 1} so
that \code{x \%*\% v == u \%*\% d}. Non-zero
elements above the diagonal result from non-orthogonality of the \code{v} matrix,
providing a simple interpretation of cumulative information, or explained variance
in the PCA case, via the singular value decomposition of \code{d \%*\% t(v)}.}
}
What if you have no idea for values of the argument \code{n} (the desired sparsity)?
The reference describes a cross-validation and an ad-hoc approach; neither of which are
in the package yet. Both are prohibitively computationally expensive for matrices with a huge
number of columns. A future version of this package will include a revised approach to
automatically selecting a reasonable sparsity constraint.
Compare with the similar but more general functions \code{SPC} and \code{PMD} in the \code{PMA} package
by Daniela M. Witten, Robert Tibshirani, Sam Gross, and Balasubramanian Narasimhan.
The \code{PMD} function can compute low-rank regularized matrix decompositions with sparsity penalties
on both the \code{u} and \code{v} vectors. The \code{ssvd} function is
similar to the PMD(*, L1) method invocation of \code{PMD} or alternatively the \code{SPC} function.
Although less general than \code{PMD}(*),
the \code{ssvd} function can be faster and more memory efficient for the
basic sparse PCA problem.
See \url{https://bwlewis.github.io/irlba/ssvd.html} for more information.
(* Note that the s4vd package by Martin Sill and Sebastian Kaiser, \url{https://cran.r-project.org/package=s4vd},
includes a fast optimized version of a closely related algorithm by Shen, Huang, and Marron, that penalizes
both \code{u} and \code{v}.)
}
\examples{
set.seed(1)
u <- matrix(rnorm(200), ncol=1)
v <- matrix(c(runif(50, min=0.1), rep(0,250)), ncol=1)
u <- u / drop(sqrt(crossprod(u)))
v <- v / drop(sqrt(crossprod(v)))
x <- u \%*\% t(v) + 0.001 * matrix(rnorm(200*300), ncol=300)
s <- ssvd(x, n=50)
table(actual=v[, 1] != 0, estimated=s$v[, 1] != 0)
oldpar <- par(mfrow=c(2, 1))
plot(u, cex=2, main="u (black circles), Estimated u (blue discs)")
points(s$u, pch=19, col=4)
plot(v, cex=2, main="v (black circles), Estimated v (blue discs)")
points(s$v, pch=19, col=4)
# Let's consider a trivial rank-2 example (k=2) with noise. Like the
# last example, we know the exact number of nonzero elements in each
# solution vector of the noise-free matrix. Note the application of
# different sparsity constraints on each column of the estimated v.
# Also, the decomposition is unique only up to sign, which we adjust
# for below.
set.seed(1)
u <- qr.Q(qr(matrix(rnorm(400), ncol=2)))
v <- matrix(0, ncol=2, nrow=300)
v[sample(300, 15), 1] <- runif(15, min=0.1)
v[sample(300, 50), 2] <- runif(50, min=0.1)
v <- qr.Q(qr(v))
x <- u \%*\% (c(2, 1) * t(v)) + .001 * matrix(rnorm(200 * 300), 200)
s <- ssvd(x, k=2, n=colSums(v != 0))
# Compare actual and estimated vectors (adjusting for sign):
s$u <- sign(u) * abs(s$u)
s$v <- sign(v) * abs(s$v)
table(actual=v[, 1] != 0, estimated=s$v[, 1] != 0)
table(actual=v[, 2] != 0, estimated=s$v[, 2] != 0)
plot(v[, 1], cex=2, main="True v1 (black circles), Estimated v1 (blue discs)")
points(s$v[, 1], pch=19, col=4)
plot(v[, 2], cex=2, main="True v2 (black circles), Estimated v2 (blue discs)")
points(s$v[, 2], pch=19, col=4)
par(oldpar)
}
\references{
\itemize{
\item{Shen, Haipeng, and Jianhua Z. Huang. "Sparse principal component analysis via regularized low rank matrix approximation." Journal of multivariate analysis 99.6 (2008): 1015-1034.}
\item{Witten, Tibshirani and Hastie (2009) A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. _Biostatistics_ 10(3): 515-534.}
}
}
irlba/man/prcomp_irlba.Rd 0000644 0001760 0000144 00000007023 13430560161 015107 0 ustar ripley users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/prcomp.R
\name{prcomp_irlba}
\alias{prcomp_irlba}
\title{Principal Components Analysis}
\usage{
prcomp_irlba(x, n = 3, retx = TRUE, center = TRUE, scale. = FALSE, ...)
}
\arguments{
\item{x}{a numeric or complex matrix (or data frame) which provides
the data for the principal components analysis.}
\item{n}{integer number of principal component vectors to return, must be less than
\code{min(dim(x))}.}
\item{retx}{a logical value indicating whether the rotated variables should be returned.}
\item{center}{a logical value indicating whether the variables should be
shifted to be zero centered. Alternately, a centering vector of length
equal the number of columns of \code{x} can be supplied.}
\item{scale.}{a logical value indicating whether the variables should be
scaled to have unit variance before the analysis takes place.
The default is \code{FALSE} for consistency with S, but scaling is often advisable.
Alternatively, a vector of length equal the number of columns of \code{x} can be supplied.
The value of \code{scale} determines how column scaling is performed
(after centering). If \code{scale} is a numeric vector with length
equal to the number of columns of \code{x}, then each column of \code{x} is
divided by the corresponding value from \code{scale}. If \code{scale} is
\code{TRUE} then scaling is done by dividing the (centered) columns of
\code{x} by their standard deviations if \code{center=TRUE}, and the
root mean square otherwise. If \code{scale} is \code{FALSE}, no scaling is done.
See \code{\link{scale}} for more details.}
\item{...}{additional arguments passed to \code{\link{irlba}}.}
}
\value{
A list with class "prcomp" containing the following components:
\itemize{
\item{sdev} {the standard deviations of the principal components (i.e.,
the square roots of the eigenvalues of the
covariance/correlation matrix, though the calculation is
actually done with the singular values of the data matrix).}
\item{rotation} {the matrix of variable loadings (i.e., a matrix whose columns
contain the eigenvectors).}
\item {x} {if \code{retx} is \code{TRUE} the value of the rotated data (the centred
(and scaled if requested) data multiplied by the \code{rotation}
matrix) is returned. Hence, \code{cov(x)} is the diagonal matrix
\code{diag(sdev^2)}.}
\item{center, scale} {the centering and scaling used, or \code{FALSE}.}
}
}
\description{
Efficient computation of a truncated principal components analysis of a given data matrix
using an implicitly restarted Lanczos method from the \code{\link{irlba}} package.
}
\note{
The signs of the columns of the rotation matrix are arbitrary, and
so may differ between different programs for PCA, and even between
different builds of R.
NOTE DIFFERENCES WITH THE DEFAULT \code{\link{prcomp}} FUNCTION!
The \code{tol} truncation argument found in \code{prcomp} is not supported.
In place of the truncation tolerance in the original function, the
\code{prcomp_irlba} function has the argument \code{n} explicitly giving the
number of principal components to return. A warning is generated if the
argument \code{tol} is used, which is interpreted differently between
the two functions.
}
\examples{
set.seed(1)
x <- matrix(rnorm(200), nrow=20)
p1 <- prcomp_irlba(x, n=3)
summary(p1)
# Compare with
p2 <- prcomp(x, tol=0.7)
summary(p2)
}
\seealso{
\code{\link{prcomp}}
}
irlba/man/svdr.Rd 0000644 0001760 0000144 00000007745 13430560161 013427 0 ustar ripley users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/svdr.R
\name{svdr}
\alias{svdr}
\title{Find a few approximate largest singular values and corresponding
singular vectors of a matrix.}
\usage{
svdr(x, k, tol = 1e-05, it = 100L, extra = min(10L, dim(x) - k),
center = NULL, Q = NULL, return.Q = FALSE)
}
\arguments{
\item{x}{numeric real- or complex-valued matrix or real-valued sparse matrix.}
\item{k}{dimension of subspace to estimate (number of approximate singular values to compute).}
\item{tol}{stop iteration when the largest absolute relative change in estimated singular
values from one iteration to the next falls below this value.}
\item{it}{maximum number of algorithm iterations.}
\item{extra}{number of extra vectors of dimension \code{ncol(x)}, larger values generally improve accuracy (with increased
computational cost).}
\item{center}{optional column centering vector whose values are implicitly subtracted from each
column of \code{A} without explicitly forming the centered matrix (preserving sparsity).
Optionally specify \code{center=TRUE} as shorthand for \code{center=colMeans(x)}.
Use for efficient principal components computation.}
\item{Q}{optional initial random matrix, defaults to a matrix of size \code{ncol(x)} by \code{k + extra} with
entries sampled from a normal random distribution.}
\item{return.Q}{if \code{TRUE} return the \code{Q} matrix for restarting (see examples).}
}
\value{
Returns a list with entries:
\describe{
\item{d:}{ k approximate singular values}
\item{u:}{ k approximate left singular vectors}
\item{v:}{ k approximate right singular vectors}
\item{mprod:}{ total number of matrix products carried out}
\item{Q:}{ optional subspace matrix (when \code{return.Q=TRUE})}
}
}
\description{
The randomized method for truncated SVD by P. G. Martinsson and colleagues
finds a few approximate largest singular values and corresponding
singular vectors of a sparse or dense matrix. It is a fast and
memory-efficient way to compute a partial SVD, similar in performance
for many problems to \code{\link{irlba}}. The \code{svdr} method
is a block method and may produce more accurate estimations with
less work for problems with clustered large singular values (see
the examples). In other problems, \code{irlba} may exhibit faster
convergence.
}
\details{
Also see an alternate implementation (\code{rsvd}) of this method by N. Benjamin Erichson
in the https://cran.r-project.org/package=rsvd package.
}
\examples{
set.seed(1)
A <- matrix(runif(400), nrow=20)
svdr(A, 3)$d
# Compare with svd
svd(A)$d[1:3]
# Compare with irlba
irlba(A, 3)$d
\dontrun{
# A problem with clustered large singular values where svdr out-performs irlba.
tprolate <- function(n, w=0.25)
{
a <- rep(0, n)
a[1] <- 2 * w
a[2:n] <- sin( 2 * pi * w * (1:(n-1)) ) / ( pi * (1:(n-1)) )
toeplitz(a)
}
x <- tprolate(512)
set.seed(1)
tL <- system.time(L <- irlba(x, 20))
tR <- system.time(R <- svdr(x, 20))
S <- svd(x)
plot(S$d)
data.frame(time=c(tL[3], tR[3]),
error=sqrt(c(crossprod(L$d - S$d[1:20]), crossprod(R$d - S$d[1:20]))),
row.names=c("IRLBA", "Randomized SVD"))
# But, here is a similar problem with clustered singular values where svdr
# doesn't out-perform irlba as easily...clusters of singular values are,
# in general, very hard to deal with!
# (This example based on https://github.com/bwlewis/irlba/issues/16.)
set.seed(1)
s <- svd(matrix(rnorm(200 * 200), 200))
x <- s$u \%*\% (c(exp(-(1:100)^0.3) * 1e-12 + 1, rep(0.5, 100)) * t(s$v))
tL <- system.time(L <- irlba(x, 5))
tR <- system.time(R <- svdr(x, 5))
S <- svd(x)
plot(S$d)
data.frame(time=c(tL[3], tR[3]),
error=sqrt(c(crossprod(L$d - S$d[1:5]), crossprod(R$d - S$d[1:5]))),
row.names=c("IRLBA", "Randomized SVD"))
}
}
\references{
Finding structure with randomness: Stochastic algorithms for constructing
approximate matrix decompositions N. Halko, P. G. Martinsson, J. Tropp. Sep. 2009.
}
\seealso{
\code{\link{irlba}}, \code{\link{svd}}, \code{rsvd} in the rsvd package
}
irlba/man/irlba.Rd 0000644 0001760 0000144 00000023733 13430560161 013535 0 ustar ripley users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/irlba.R
\name{irlba}
\alias{irlba}
\title{Find a few approximate singular values and corresponding
singular vectors of a matrix.}
\usage{
irlba(A, nv = 5, nu = nv, maxit = 1000, work = nv + 7, reorth = TRUE,
tol = 1e-05, v = NULL, right_only = FALSE, verbose = FALSE,
scale = NULL, center = NULL, shift = NULL, mult = NULL,
fastpath = TRUE, svtol = tol, smallest = FALSE, ...)
}
\arguments{
\item{A}{numeric real- or complex-valued matrix or real-valued sparse matrix.}
\item{nv}{number of right singular vectors to estimate.}
\item{nu}{number of left singular vectors to estimate (defaults to \code{nv}).}
\item{maxit}{maximum number of iterations.}
\item{work}{working subspace dimension, larger values can speed convergence at the cost of more memory use.}
\item{reorth}{if \code{TRUE}, apply full reorthogonalization to both SVD bases, otherwise
only apply reorthogonalization to the right SVD basis vectors; the latter case is cheaper per
iteration but, overall, may require more iterations for convergence. Automatically \code{TRUE}
when \code{fastpath=TRUE} (see below).}
\item{tol}{convergence is determined when \eqn{\|A^TU - VS\| < tol\|A\|}{||A^T U - VS|| < tol*||A||},
and when the maximum relative change in estimated singular values from one iteration to the
next is less than \code{svtol = tol} (see \code{svtol} below),
where the spectral norm ||A|| is approximated by the
largest estimated singular value, and U, V, S are the matrices corresponding
to the estimated left and right singular vectors, and diagonal matrix of
estimated singular values, respectively.}
\item{v}{optional starting vector or output from a previous run of \code{irlba} used
to restart the algorithm from where it left off (see the notes).}
\item{right_only}{logical value indicating return only the right singular vectors
(\code{TRUE}) or both sets of vectors (\code{FALSE}). The right_only option can be
cheaper to compute and use much less memory when \code{nrow(A) >> ncol(A)} but note
that obtained solutions typically lose accuracy due to lack of re-orthogonalization in the
algorithm and that \code{right_only = TRUE} sets \code{fastpath = FALSE} (only use this option
for really large problems that run out of memory and when \code{nrow(A) >> ncol(A)}).
Consider increasing the \code{work} option to improve accuracy with \code{right_only=TRUE}.}
\item{verbose}{logical value that when \code{TRUE} prints status messages during the computation.}
\item{scale}{optional column scaling vector whose values divide each column of \code{A};
must be as long as the number of columns of \code{A} (see notes).}
\item{center}{optional column centering vector whose values are subtracted from each
column of \code{A}; must be as long as the number of columns of \code{A} and may
not be used together with the deflation options below (see notes).}
\item{shift}{optional shift value (square matrices only, see notes).}
\item{mult}{DEPRECATED optional custom matrix multiplication function (default is \code{\%*\%}, see notes).}
\item{fastpath}{try a fast C algorithm implementation if possible; set \code{fastpath=FALSE} to use the
reference R implementation. See the notes for more details.}
\item{svtol}{additional stopping tolerance on maximum allowed absolute relative change across each
estimated singular value between iterations.
The default value of this parameter is to set it to \code{tol}. You can set \code{svtol=Inf} to
effectively disable this stopping criterion. Setting \code{svtol=Inf} allows the method to
terminate on the first Lanczos iteration if it finds an invariant subspace, but with less certainty
that the converged subspace is the desired one. (It may, for instance, miss some of the largest
singular values in difficult problems.)}
\item{smallest}{set \code{smallest=TRUE} to estimate the smallest singular values and associated
singular vectors. WARNING: this option is somewhat experimental, and may produce poor
estimates for ill-conditioned matrices.}
\item{...}{optional additional arguments used to support experimental and deprecated features.}
}
\value{
Returns a list with entries:
\describe{
\item{d:}{ max(nu, nv) approximate singular values}
\item{u:}{ nu approximate left singular vectors (only when right_only=FALSE)}
\item{v:}{ nv approximate right singular vectors}
\item{iter:}{ The number of Lanczos iterations carried out}
\item{mprod:}{ The total number of matrix vector products carried out}
}
}
\description{
The augmented implicitly restarted Lanczos bidiagonalization algorithm
(IRLBA) finds a few approximate largest (or, optionally, smallest) singular
values and corresponding
singular vectors of a sparse or dense matrix using a method of Baglama and
Reichel. It is a fast and memory-efficient way to compute a partial SVD.
}
\note{
The syntax of \code{irlba} partially follows \code{svd}, with an important
exception. The usual R \code{svd} function always returns a complete set of
singular values, even if the number of singular vectors \code{nu} or \code{nv}
is set less than the maximum. The \code{irlba} function returns a number of
estimated singular values equal to the maximum of the number of specified
singular vectors \code{nu} and \code{nv}.
Use the optional \code{scale} parameter to implicitly scale each column of
the matrix \code{A} by the values in the \code{scale} vector, computing the
truncated SVD of the column-scaled \code{sweep(A, 2, scale, FUN=`/`)}, or
equivalently, \code{A \%*\% diag(1 / scale)}, without explicitly forming the
scaled matrix. \code{scale} must be a non-zero vector of length equal
to the number of columns of \code{A}.
Use the optional \code{center} parameter to implicitly subtract the values
in the \code{center} vector from each column of \code{A}, computing the
truncated SVD of \code{sweep(A, 2, center, FUN=`-`)},
without explicitly forming the centered matrix. \code{center}
must be a vector of length equal to the number of columns of \code{A}.
This option may be used to efficiently compute principal components without
explicitly forming the centered matrix (which can, importantly, preserve
sparsity in the matrix). See the examples.
The optional \code{shift} scalar valued argument applies only to square matrices; use it
to estimate the partial svd of \code{A + diag(shift, nrow(A), nrow(A))}
(without explicitly forming the shifted matrix).
(Deprecated) Specify an optional alternative matrix multiplication operator in the
\code{mult} parameter. \code{mult} must be a function of two arguments,
and must handle both cases where one argument is a vector and the other
a matrix. This option is deprecated and will be removed in a future version.
The new preferred method simply uses R itself to define a custom matrix class
with your user-defined matrix multiplication operator. See the examples.
Use the \code{v} option to supply a starting vector for the iterative
method. A random vector is used by default (precede with \code{set.seed()}
for reproducibility). Optionally set \code{v} to
the output of a previous run of \code{irlba} to restart the method, adding
additional singular values/vectors without recomputing the solution
subspace. See the examples.
The function may generate the following warnings:
\itemize{
\item{"did not converge--results might be invalid!; try increasing work or maxit"
means that the algorithm didn't
converge -- this is potentially a serious problem and the returned results may not be valid. \code{irlba}
reports a warning here instead of an error so that you can inspect whatever is returned. If this
happens, carefully heed the warning and inspect the result. You may also try setting \code{fastpath=FALSE}.}
\item{"You're computing a large percentage of total singular values, standard svd might work better!"
\code{irlba} is designed to efficiently compute a few of the largest singular values and associated
singular vectors of a matrix. The standard \code{svd} function will be more efficient for computing
large numbers of singular values than \code{irlba}.}
\item{"convergence criterion below machine epsilon" means that the product of \code{tol} and the
largest estimated singular value is really small and the normal convergence criterion is only
met up to round off error.}
}
The function might return an error for several reasons including a situation when the starting
vector \code{v} is near the null space of the matrix. In that case, try a different \code{v}.
The \code{fastpath=TRUE} option only supports real-valued matrices and sparse matrices
of type \code{dgCMatrix} (for now). Other problems fall back to the reference
R implementation.
}
\examples{
set.seed(1)
A <- matrix(runif(400), nrow=20)
S <- irlba(A, 3)
S$d
# Compare with svd
svd(A)$d[1:3]
# Restart the algorithm to compute more singular values
# (starting with an existing solution S)
S1 <- irlba(A, 5, v=S)
# Estimate smallest singular values
irlba(A, 3, smallest=TRUE)$d
#Compare with
tail(svd(A)$d, 3)
# Principal components (see also prcomp_irlba)
P <- irlba(A, nv=1, center=colMeans(A))
# Compare with prcomp and prcomp_irlba (might vary up to sign)
cbind(P$v,
prcomp(A)$rotation[, 1],
prcomp_irlba(A)$rotation[, 1])
# A custom matrix multiplication function that scales the columns of A
# (cf the scale option). This function scales the columns of A to unit norm.
col_scale <- sqrt(apply(A, 2, crossprod))
setClass("scaled_matrix", contains="matrix", slots=c(scale="numeric"))
setMethod("\%*\%", signature(x="scaled_matrix", y="numeric"),
function(x ,y) x@.Data \%*\% (y / x@scale))
setMethod("\%*\%", signature(x="numeric", y="scaled_matrix"),
function(x ,y) (x \%*\% y@.Data) / y@scale)
a <- new("scaled_matrix", A, scale=col_scale)
irlba(a, 3)$d
# Compare with:
svd(sweep(A, 2, col_scale, FUN=`/`))$d[1:3]
}
\references{
Baglama, James, and Lothar Reichel. "Augmented implicitly restarted Lanczos bidiagonalization methods." SIAM Journal on Scientific Computing 27.1 (2005): 19-42.
}
\seealso{
\code{\link{svd}}, \code{\link{prcomp}}, \code{\link{partial_eigen}}, \code{\link{svdr}}
}
irlba/man/partial_eigen.Rd 0000644 0001760 0000144 00000004242 13430560161 015241 0 ustar ripley users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/eigen.R
\name{partial_eigen}
\alias{partial_eigen}
\title{Find a few approximate largest eigenvalues and corresponding eigenvectors of a symmetric matrix.}
\usage{
partial_eigen(x, n = 5, symmetric = TRUE, ...)
}
\arguments{
\item{x}{numeric real-valued dense or sparse matrix.}
\item{n}{number of largest eigenvalues and corresponding eigenvectors to compute.}
\item{symmetric}{\code{TRUE} indicates \code{x} is a symmetric matrix (the default);
specify \code{symmetric=FALSE} to compute the largest eigenvalues and corresponding
eigenvectors of \code{t(x) \%*\% x} instead.}
\item{...}{optional additional parameters passed to the \code{irlba} function.}
}
\value{
Returns a list with entries:
\itemize{
\item{values}{ n approximate largest eigenvalues}
\item{vectors}{ n approximate corresponding eigenvectors}
}
}
\description{
Use \code{partial_eigen} to estimate a subset of the largest (most positive)
eigenvalues and corresponding eigenvectors of a symmetric dense or sparse
real-valued matrix.
}
\note{
Specify \code{symmetric=FALSE} to compute the largest \code{n} eigenvalues
and corresponding eigenvectors of the symmetric matrix cross-product
\code{t(x) \%*\% x}.
This function uses the \code{irlba} function under the hood. See \code{?irlba}
for description of additional options, especially the \code{tol} parameter.
See the RSpectra package https://cran.r-project.org/package=RSpectra for more comprehensive
partial eigenvalue decomposition.
}
\examples{
set.seed(1)
# Construct a symmetric matrix with some positive and negative eigenvalues:
V <- qr.Q(qr(matrix(runif(100), nrow=10)))
x <- V \%*\% diag(c(10, -9, 8, -7, 6, -5, 4, -3, 2, -1)) \%*\% t(V)
partial_eigen(x, 3)$values
# Compare with eigen
eigen(x)$values[1:3]
# Use symmetric=FALSE to compute the eigenvalues of t(x) \%*\% x for general
# matrices x:
x <- matrix(rnorm(100), 10)
partial_eigen(x, 3, symmetric=FALSE)$values
eigen(crossprod(x))$values
}
\references{
Augmented Implicitly Restarted Lanczos Bidiagonalization Methods, J. Baglama and L. Reichel, SIAM J. Sci. Comput. 2005.
}
\seealso{
\code{\link{eigen}}, \code{\link{irlba}}
}
irlba/man/summary.irlba_prcomp.Rd 0000644 0001760 0000144 00000000765 13430560161 016611 0 ustar ripley users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/prcomp.R
\name{summary.irlba_prcomp}
\alias{summary.irlba_prcomp}
\title{Summary method for truncated pca objects computed by \code{prcomp_irlba}.}
\usage{
\method{summary}{irlba_prcomp}(object, ...)
}
\arguments{
\item{object}{An object returned by \code{prcomp_irlba}.}
\item{...}{Optional arguments passed to \code{summary}.}
}
\description{
Summary method for truncated pca objects computed by \code{prcomp_irlba}.
}
irlba/DESCRIPTION 0000644 0001760 0000144 00000002022 14316626071 013103 0 ustar ripley users Package: irlba
Type: Package
Title: Fast Truncated Singular Value Decomposition and Principal
Components Analysis for Large Dense and Sparse Matrices
Version: 2.3.5.1
Date: 2021-12-05
Authors@R: c(
person("Jim", "Baglama", role=c("aut", "cph"), email="jbaglama@uri.edu"),
person("Lothar", "Reichel", role=c("aut", "cph"), email="reichel@math.kent.edu"),
person("B. W.", "Lewis", role=c("aut","cre","cph"), email="blewis@illposed.net"))
Description: Fast and memory efficient methods for truncated singular value
decomposition and principal components analysis of large sparse and dense
matrices.
Depends: R (>= 3.6.2), Matrix
LinkingTo: Matrix
Imports: stats, methods
License: GPL-3
BugReports: https://github.com/bwlewis/irlba/issues
RoxygenNote: 5.0.1
NeedsCompilation: yes
Packaged: 2022-10-03 15:17:05 UTC; ripley
Author: Jim Baglama [aut, cph],
Lothar Reichel [aut, cph],
B. W. Lewis [aut, cre, cph]
Maintainer: B. W. Lewis
Repository: CRAN
Date/Publication: 2022-10-03 18:38:49 UTC
irlba/build/ 0000755 0001760 0000144 00000000000 14153273722 012500 5 ustar ripley users irlba/build/vignette.rds 0000644 0001760 0000144 00000000305 14153273722 015035 0 ustar ripley users b```b`a@&0`b fd`aҜE9IzAyh<` ļ
R$@
`
` sST楀 M wjey~L6̜T!%ps QY_/
@?{49'ݣ\)%ziE@ w
irlba/tests/ 0000755 0001760 0000144 00000000000 13430560161 012534 5 ustar ripley users irlba/tests/ssvd.R 0000644 0001760 0000144 00000001734 13430560161 013643 0 ustar ripley users # Tests for sparse SVD/PCA
require("irlba")
loc <- ""
test <- function()
{
on.exit(message("Error occured in: ", loc))
loc <<- "sparse SVD"
set.seed(1)
x <- matrix(rnorm(100), 10)
s <- ssvd(x, 1, n=5)
stopifnot(isTRUE(all.equal(sqrt(drop(crossprod(x %*% s$v - s$u %*% s$d))), 0)))
loc <<- "sparse PCA"
set.seed(1)
x <- matrix(rnorm(100), 10)
s <- ssvd(x, 1, n=5, center=TRUE)
stopifnot(isTRUE(all.equal(sqrt(drop(crossprod(scale(x, center=TRUE, scale=FALSE) %*% s$v - s$u %*% s$d))), 0)))
loc <<- "sparse PCA + scale"
set.seed(1)
x <- matrix(rnorm(100), 10)
s <- ssvd(x, 1, n=5, center=TRUE, scale.=TRUE)
isTRUE(all.equal(sqrt(drop(crossprod(scale(x, center=TRUE, scale=TRUE) %*% s$v - s$u %*% s$d))), 0))
loc <<- "sparse scaled"
set.seed(1)
x <- matrix(rnorm(100), 10)
s <- ssvd(x, 1, n=5, center=FALSE, scale.=TRUE)
isTRUE(all.equal(sqrt(drop(crossprod(scale(x, center=FALSE, scale=TRUE) %*% s$v - s$u %*% s$d))), 0))
on.exit()
}
test()
irlba/tests/edge.R 0000644 0001760 0000144 00000005045 13430560161 013567 0 ustar ripley users # Tests for a few edge cases
require("irlba")
loc <- ""
test <- function()
{
on.exit(message("Error occured in: ", loc))
# Dense matrix
loc <<- "dense"
set.seed(1)
A <- matrix(rnorm(16), 4)
L <- irlba(A, nu=1, nv=1, tol=1e-9, fastpath=FALSE)
L1 <- irlba(A, nu=1, nv=1, tol=1e-9, fastpath=TRUE)
S <- svd(A, nu=1, nv=1)
if (!isTRUE(all.equal(L$d, S$d[1])))
{
stop("Failed tiny reference example ")
}
if (!isTRUE(all.equal(L1$d, S$d[1])))
{
stop("Failed tiny fastpath example")
}
# Tickle misc. checks
loc <<- "misc"
set.seed(1)
A <- matrix(rnorm(100), 10)
L <- tryCatch(irlba(A, nv=3, tol=1e-9, fastpath=FALSE, work=2, v=rep(0, nrow(A))), error=function(e) "NULLSPACE")
S <- svd(A)
L <- irlba(A, nv=3, tol=1e-9, fastpath=FALSE, work=2, v=S$v[, 1])
A <- S$u %*% diag(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1e-12)) %*% t(S$v)
L <- irlba(A, nv=3, tol=1e-9, fastpath=FALSE, work=2, reorth=FALSE)
# Convergence
loc <<- "convergence"
A <- S$u %*% (c(1e-5, rep(1e-9, 9)) * t(S$v))
for (tol in 10 ^ - (7:11))
{
L <- irlba(A, 3, tol=tol, svtol=Inf)
converged <- svd(A %*% L$v - L$u %*% diag(L$d))$d[1] < tol * L$d[1]
stopifnot(converged)
}
# Sparse but not dgCMatrix (issue #6)
loc <<- "misc sparse"
A <- Matrix(matrix(rnorm(100), 10))
L <- irlba(A, nv=1)
S <- svd(A, nu=1, nv=1)
if (!isTRUE(all.equal(L$d, S$d[1])))
{
stop("Failed general sparse matrix example ")
}
A <- Matrix(sample(c(FALSE, TRUE), 100, replace=TRUE), 10, 10)
L <- irlba(A, nv=1)
S <- svd(A, nu=1, nv=1)
if (!isTRUE(all.equal(L$d, S$d[1])))
{
stop("Failed logical sparse matrix example ")
}
# Test for issue #7, a really dumb bug.
loc <<- "issue 7"
mx <- matrix(sample(1:10, 10 * 100, replace=TRUE), nrow=10)
S <- irlba(mx, nv=2, verbose=TRUE, center=colMeans(mx), right_only=TRUE)
# test for issue #9
loc <<- "issue 9"
set.seed(2)
s1 <- irlba(diag(c(1, 2, 3, 4, 5, 0, 0, 0, 0)), 4, fastpath=FALSE)
set.seed(2)
s2 <- irlba(diag(c(1, 2, 3, 4, 5, 0, 0, 0, 0)), 4, fastpath=TRUE)
stopifnot(all.equal(s1$d, s2$d))
# Repeat this test with different seed
set.seed(3)
s2 <- irlba(diag(c(1, 2, 3, 4, 5, 0, 0, 0, 0)), 4, fastpath=TRUE)
stopifnot(all.equal(s1$d, s2$d))
loc <<- "issue 26"
set.seed(1)
r <- 10
n <- 1000
X1 <- matrix(rnorm(n * r), n)
X2 <- matrix(rnorm(n * r), n)
X <- X1 %*% t(X2)
l <- irlba(X, 20, fastpath=TRUE)$d
stopifnot(all.equal(tail(l, 10), rep(0, 10)))
l <- irlba(X, 20, fastpath=FALSE)$d
stopifnot(all.equal(tail(l, 10), rep(0, 10)))
on.exit()
}
test()
irlba/tests/prcomp.r 0000644 0001760 0000144 00000007701 13430560161 014224 0 ustar ripley users require("irlba")
# prcomp convenience function
x <- matrix(rnorm(200), nrow=20)
p1 <- prcomp_irlba(x, n=3)
p2 <- prcomp(x, tol=0.7)
if (!isTRUE(all.equal(p1$sdev[1:2], p2$sdev[1:2])))
{
stop("Failed basic prcomp test")
}
s <- summary(p1)
# scaling bug identified in issue #21
normalize_signs <- function(X, Y) {
for (i in 1:ncol(X)) {
if (sign(X[1, i]) != sign(Y[1, i])) {
Y[, i] <- -Y[, i]
}
}
return(Y)
}
all.equal_pca <- function(X, Y) {
Y <- normalize_signs(X, Y)
return(all.equal(X, Y, check.attributes=F, tolerance=1e-4))
}
set.seed(1)
X <- matrix(rnorm(2000), ncol=40)
M <- 5 # number of PCA components
centers <- colMeans(X)
sds <- apply(X, 2, sd)
rms <- apply(X, 2, function(x) sqrt(sum(x^2) / (length(x) - 1)))
Xc <- sweep(X, 2, centers, `-`)
Xs <- sweep(X, 2, sds, `/`)
Xcs <- sweep(Xc, 2, sds, `/`)
Xrms <- sweep(X, 2, rms, `/`)
# unscaled
scaled <- FALSE
centered <- FALSE
pca <- prcomp(X, center=centered, scale.=scaled)
sv <- svd(X)
svir <- irlba(X, nv=M, nu=M)
pcair <- prcomp_irlba(X, n=M, center=centered, scale.=scaled)
Xpca <- predict(pca)[, 1:M]
Xsvl <- sv$u[, 1:M] %*% diag(sv$d[1:M])
Xsvr <- X %*% sv$v[, 1:M]
Xsvirl <- svir$u %*% diag(svir$d)
Xsvirr <- X %*% svir$v
Xpcair <- predict(pcair)
Xpcair2 <- X %*% pcair$rotation
if (! isTRUE(all.equal_pca(Xsvl, Xsvr)) &&
isTRUE(all.equal_pca(Xpca, Xsvl)) &&
isTRUE(all.equal_pca(Xsvirl, Xsvirr)) &&
isTRUE(all.equal_pca(Xpca, Xsvirl)) &&
isTRUE(all.equal_pca(Xpcair, Xpcair2)) &&
isTRUE(all.equal_pca(Xpca, Xpcair)) &&
isTRUE(all.equal_pca(Xpcair, Xsvirl)))
{
stop("failed unscaled, uncentered prcomp")
}
# scaled, uncentered
scaled <- TRUE
centered <- FALSE
pca <- prcomp(X, center=centered, scale.=scaled)
sv <- svd(Xrms)
svir <- irlba(X, nv=M, nu=M, scale=rms)
pcair <- prcomp_irlba(X, n=M, center=centered, scale.=scaled)
Xpca <- predict(pca)[, 1:M]
Xsvl <- sv$u[, 1:M] %*% diag(sv$d[1:M])
Xsvr <- Xrms %*% sv$v[, 1:M]
Xsvirl <- svir$u %*% diag(svir$d)
Xsvirr <- Xrms %*% svir$v
Xpcair <- predict(pcair)
Xpcair2 <- Xrms %*% pcair$rotation
if (! isTRUE(all.equal_pca(Xsvl, Xsvr)) &&
isTRUE(all.equal_pca(Xpca, Xsvl)) &&
isTRUE(all.equal_pca(Xsvirl, Xsvirr)) &&
isTRUE(all.equal_pca(Xpca, Xsvirl)) &&
isTRUE(all.equal_pca(Xpcair, Xpcair2)) &&
isTRUE(all.equal_pca(Xpca, Xpcair)) &&
isTRUE(all.equal_pca(Xpcair, Xsvirl)))
{
stop("failed scaled, uncentered prcomp")
}
# issue #25 prcomp_irlba regression (error in scale. handling)
set.seed(1)
x <- matrix(rnorm(100), 10)
p <- prcomp_irlba(x, 3, scale.=TRUE, fastpath=FALSE)
p <- prcomp_irlba(x, 3, scale.=TRUE, fastpath=TRUE)
# issue #32 (and also issue #25 redux) more checks for proper
# variance proportion computation
library(irlba)
set.seed(1)
x <- matrix(rnorm(200), nrow=20)
n <- 3
s1 <- summary(prcomp_irlba(x, n=n, center=TRUE, scale.=FALSE))
s2 <- summary(prcomp(x, tol=0.7, center=TRUE, scale.=FALSE))
if (! isTRUE(all.equal(s1$sdev, s2$sdev[1:n]) &&
all.equal(s1$importance, s2$importance[, 1:n])))
{
stop("center=TRUE scale.=FALSE prcomp variance computation")
}
s1 <- summary(prcomp_irlba(x, n=3, center=TRUE, scale.=TRUE))
s2 <- summary(prcomp(x, tol=0.8, center=TRUE, scale.=TRUE))
if (! isTRUE(all.equal(s1$sdev, s2$sdev[1:n]) &&
all.equal(s1$importance, s2$importance[, 1:n])))
{
stop("center=TRUE scale.=TRUE prcomp variance computation")
}
s1 <- summary(prcomp_irlba(x, n=3, center=FALSE, scale.=TRUE))
s2 <- summary(prcomp(x, tol=0.8, center=FALSE, scale.=TRUE))
if (! isTRUE(all.equal(s1$sdev, s2$sdev[1:n]) &&
all.equal(s1$importance, s2$importance[, 1:n])))
{
stop("center=FALSE scale.=TRUE prcomp variance computation")
}
s1 <- summary(prcomp_irlba(x, n=3, center=FALSE, scale.=FALSE))
s2 <- summary(prcomp(x, tol=0.7, center=FALSE, scale.=FALSE))
if (! isTRUE(all.equal(s1$sdev, s2$sdev[1:n]) &&
all.equal(s1$importance, s2$importance[, 1:n])))
{
stop("center=FALSE, scale.=FALSE prcomp variance computation")
}
irlba/tests/svdr.R 0000644 0001760 0000144 00000002316 13430560161 013637 0 ustar ripley users # Tests for svdr
require("irlba")
loc <- ""
test <- function()
{
on.exit(message("Error occured in: ", loc))
# Dense matrix
loc <<- "svdr dense"
set.seed(1)
A <- matrix(rnorm(16), 4)
L <- svdr(A, 1)
S <- svd(A, nu=1, nv=1)
stopifnot(isTRUE(all.equal(L$d, S$d[1])))
loc <<- "svdr dense m > n"
A <- matrix(rnorm(50 * 40), 50)
L <- svdr(A, 5, extra=15)
S <- svd(A, nu=5, nv=5)
stopifnot(isTRUE(all.equal(L$d, S$d[1:5])))
loc <<- "svdr dense m < n"
A <- matrix(rnorm(50 * 40), 40)
L <- svdr(A, 5, extra=15)
S <- svd(A, nu=5, nv=5)
stopifnot(isTRUE(all.equal(L$d, S$d[1:5])))
# Sparse but not dgCMatrix (issue #6)
loc <<- "svdr misc sparse"
A <- Matrix(matrix(rnorm(100), 10))
L <- svdr(A, 1)
S <- svd(A, nu=1, nv=1)
stopifnot(isTRUE(all.equal(L$d, S$d[1])))
loc <<- "svdr logical sparse"
A <- Matrix(sample(c(FALSE, TRUE), 100, replace=TRUE), 10, 10)
L <- svdr(A, 1)
S <- svd(A, nu=1, nv=1)
stopifnot(isTRUE(all.equal(L$d, S$d[1])))
loc <<- "svdr center only, sparse"
A <- Matrix(matrix(rnorm(100), 10))
m <- colMeans(A)
L <- svdr(A, 3, center=m)
S <- svd(scale(A, center=TRUE, scale=FALSE))
stopifnot(isTRUE(all.equal(L$d, S$d[1:3])))
on.exit()
}
test()
irlba/tests/test.R 0000644 0001760 0000144 00000015110 13430560161 013634 0 ustar ripley users require("irlba")
for (FAST in c(FALSE, TRUE))
{
# Dense matrix
set.seed(1)
A <- matrix(rnorm(400), 20)
L <- irlba(A, nu=2, nv=2, tol=1e-9, fastpath=FAST)
S <- svd(A, nu=2, nv=2)
if (!isTRUE(all.equal(L$d, S$d[1:2])))
{
stop("Failed simple dense singular value test", " fastpath=", FAST)
}
# restart
L1 <- irlba(A, nv=3, v=L, fastpath=FAST)
if (!isTRUE(all.equal(L1$d, S$d[1:3])))
{
stop("Failed restart", " fastpath=", FAST)
}
# unequal nu, nv
L <- irlba(A, nv=2, nu=3, fastpath=FAST)
if (!isTRUE(ncol(L$v) == 2 && ncol(L$u) == 3))
{
stop("Failed unequal nu,nv", " fastpath=", FAST)
}
# Scaling and centering, dense
s <- sqrt(apply(A, 2, crossprod))
m <- colMeans(A)
L <- irlba(A, 3, tol=1e-9, center=m, scale=s, fastpath=FAST)
S <- svd(scale(A, center=TRUE, scale=s))
if (!isTRUE(all.equal(L$d, S$d[1:3])))
{
stop("Failed scaling/centering test", " fastpath=", FAST)
}
# Scale only, non-square, dense
A <- matrix(rnorm(200), 10)
s <- seq(1, ncol(A))
m <- colMeans(A)
L <- irlba(A, 3, tol=1e-9, scale=s, fastpath=FAST)
S <- svd(scale(A, center=FALSE, scale=s))
if (!isTRUE(all.equal(L$d, S$d[1:3])))
{
stop("Failed dense scaling test", " fastpath=", FAST)
}
# Center only, non-square, dense
L <- irlba(A, 3, tol=1e-9, center=m, fastpath=FAST)
S <- svd(scale(A, center=TRUE, scale=FALSE))
if (!isTRUE(all.equal(L$d, S$d[1:3])))
{
stop("Failed dense centering test", " fastpath=", FAST)
}
# Sparse matrix
require("Matrix")
K <- 400
N <- 2000
i <- sample(K, size=N, replace=TRUE)
j <- sample(K, size=N, replace=TRUE)
A <- sparseMatrix(i, j, x=rnorm(N))
L <- irlba(A, nu=2, nv=2, tol=1e-9, fastpath=FAST)
S <- svd(A, nu=2, nv=2)
if (!isTRUE(all.equal(L$d, S$d[1:2])))
{
stop("Failed simple sparse singular value test", " fastpath=", FAST)
}
# Center only, sparse
m <- colMeans(A)
L <- irlba(A, 3, tol=1e-9, center=m, fastpath=FAST)
S <- svd(scale(A, center=TRUE, scale=FALSE))
if (!isTRUE(all.equal(L$d, S$d[1:3])))
{
stop("Failed sparse centering test", " fastpath=", FAST)
}
# scale only, spase
s <- seq(1, ncol(A))
L <- irlba(A, 3, tol=1e-9, scale=s, fastpath=FAST)
S <- svd(scale(A, center=FALSE, scale=s))
if (!isTRUE(all.equal(L$d, S$d[1:3])))
{
stop("Failed sparse scaling test", " fastpath=", FAST)
}
# Symmetric partial eigendecomposition
set.seed(1)
V <- qr.Q(qr(matrix(runif(100), nrow=10)))
x <- V %*% diag(c(10, -9, 8, -7, 6, -5, 4, -3, 2, -1)) %*% t(V)
if (!isTRUE(all.equal(partial_eigen(x, 3, fastpath=FAST)$values, c(10, 8, 6))))
{
stop("Failed partial_eigen test", " fastpath=", FAST)
}
# Test right-only option
L <- irlba(A, 2, tol=1e-3, right_only=TRUE, fastpath=FAST, work=20)
S <- svd(A, nu=2, nv=2)
if (isTRUE(max(L$d - S$d[1:2]) > 1e-3))
{
stop("Failed right_only test", " fastpath=", FAST)
}
# Dense complex-valued matrix
A <- matrix(rnorm(400), 20) + 1i * matrix(rnorm(400), 20)
L <- irlba(A, nu=2, nv=2, tol=1e-9, fastpath=FAST)
S <- svd(A, nu=2, nv=2)
if (!isTRUE(all.equal(L$d, S$d[1:2])))
{
stop("Failed complex-valued dense singular value test", " fastpath=", FAST)
}
# test extra reorthogonalization
L <- irlba(A, nu=2, nv=2, tol=1e-9, reorth=TRUE, fastpath=FAST)
if (!isTRUE(all.equal(L$d, S$d[1:2])))
{
stop("Failed reorthogonalization test", " fastpath=", FAST)
}
# very non-square dense matrices
set.seed(1)
A <- matrix(rnorm(2000), 20)
L1 <- irlba(A, nu=2, nv=2, tol=1e-9, fastpath=FAST)
L2 <- irlba(t(A), nu=2, nv=2, tol=1e-9, fastpath=FAST)
if (!isTRUE(all.equal(L1$d, L2$d)))
{
stop("Failed nonsquare test", " fastpath=", FAST)
}
# This pathological example was provided by Giuseppe Rodriguez, http://bugs.unica.it/~gppe/
# The singular values cluster at 1 and 0, making it hard to converge to a truncated
# subspace containing the largest few singular values (they are all very close).
# Or, for that matter, the smallest.
#
# Reference:
# J. M. Varah. The Prolate matrix. Linear Algebra and Appl.,
# 187:269-278, 1993.
# Michela Redivo-Zaglia, University of Padova, Italy
# Email: Michela.RedivoZaglia@unipd.it
# Giuseppe Rodriguez, University of Cagliari, Italy
# Email: rodriguez@unica.it
tprolate <- function(n, w=0.25)
{
a <- rep(0, n)
a[1] <- 2 * w
a[2:n] <- sin(2 * pi * w * (1:(n-1))) / (pi * (1:(n-1)))
toeplitz(a)
}
x <- tprolate(512)
set.seed(1)
l <- irlba(x, nv=20, fastpath=FAST)
if (isTRUE(max(abs(l$d - 1)) > 1e-3))
{
stop("Failed tprolate test fastpath=", FAST)
}
# test for issue #7 and issue #14
mx <- matrix(sample(1:100, 100 * 100, replace=TRUE), nrow=100)
set.seed(1)
l <- irlba(mx, nv=30, center=colMeans(mx), fastpath=FAST)
s <- svd(scale(mx, center=TRUE, scale=FALSE))
if (isTRUE(max(abs(l$d - s$d[1:30])) > 1e-3))
{
stop("Failed integer matrix test fastpath=", FAST)
}
# test for https://github.com/bwlewis/irlba/issues/22
set.seed(1000)
ncells <- 50
ngenes <- 1000
counts <- matrix(as.double(rpois(ncells*ngenes, lambda=100)), nrow=ncells)
centers <- colMeans(counts)
set.seed(1)
out <- irlba(scale(counts, scale=FALSE, center=centers), nu=10, nv=10)
set.seed(1)
l <- irlba(counts, center=centers, nu=10, nv=10, fastpath=FAST)
if (isTRUE(max(abs(out$d - l$d)) > 1e-3))
{
stop("Failed centering test (n > m) fastpath=", FAST)
}
# test for https://github.com/bwlewis/irlba/issues/42
set.seed(1234)
a <- matrix(rnorm(10000), ncol=20)
center <- runif(ncol(a))
scale <- runif(ncol(a))
L <- irlba(a, 5, scale=scale, center=center, fastpath=FAST)
S <- svd(scale(a, center=center, scale=scale))
if (isTRUE(max(abs(S$d[1:5] - L$d)) > 1e-3))
{
stop("Failed scale + center test fastpath=", FAST)
}
}
# smallest=TRUE, m > n (fastpath always FALSE in this case)
x <- matrix(rnorm(5000), 100)
set.seed(1)
L <- irlba(x, nv=5, smallest=TRUE)
if (!isTRUE(all.equal(L$d, tail(svd(x)$d, 5))))
{
stop("Failed smallest svd test")
}
# smallest=TRUE, n > m
x <- matrix(rnorm(5000), 50)
set.seed(1)
L <- irlba(x, nv=5, smallest=TRUE)
if (!isTRUE(all.equal(L$d, tail(svd(x)$d, 5))))
{
stop("Failed smallest svd test")
}
# test for https://github.com/bwlewis/irlba/issues/47 (again, fastpath always FALSE)
set.seed(2345)
a <- spMatrix(50, 40, x=runif(200), i=sample(50, 200, replace=TRUE), j=sample(40, 200, replace=TRUE))
center <- runif(ncol(a))
scale <- runif(ncol(a))
L <- irlba(a, 5, scale=scale, center=center)
S <- svd(scale(a, center=center, scale=scale))
if (isTRUE(max(abs(S$d[1:5] - L$d)) > 1e-3))
{
stop("Failed scale + center test for non-fastpath'able matrices")
}
irlba/src/ 0000755 0001760 0000144 00000000000 14316576142 012173 5 ustar ripley users irlba/src/irlb.h 0000644 0001760 0000144 00000007135 13430560161 013270 0 ustar ripley users /* Compute Y = Y - X * t(X) * Y */
void
orthog( double *X, // Input data matrix
double *Y, // Input data matrix
double *T, // work matrix size xn * yn
int xm, // number of columns of X
int xn, // number of columns of X
int yn); // number of columns of Y
void
convtests (int Bsz, // Number of rows of bidiagonal matrix B
int n, // requested number of singular values
double tol, // convergence tolerance
double svtol, // max change each singular value tolerane
double Smax, // largest singular value of B
double *svratio, // vector of relative singular value ratio compared to last iteration
double *residuals, // vector of residual values
int *k, // number of estimated singular values (INPUT)
// adjusted subspace size (OUTPUT)
int *converged, // 0 = FALSE, 1 = TRUE
double S); // If S == 0 then invariant subspace found.
/*
* Simple cholmod double-precision sparse matrix times dense vector multiplication interface
* Compute c = op(a) %*% b, c changed on output a and b unchanged.
* where, if transpose = 't' then op(a) = t(a) and length(b) = m, length(c) = n
* else, then op(a) = a and length(b) = n, length(c) = m
*/
void
dsdmult(char transpose, // 't' -> op(a) = t(a), non-transposed a otherwise
int m, // number of rows of a
int n, // number of columns of a
void *a, // double precision valued sparse matrix
double *b, // double precision dense vector
double *c); // output
/* IRLB function for sparse or dense double-precision valued matrices */
int
irlb(double *A, // input data matrix (dense case)
void *AS, // input data matrix (sparse case)
int mult, // 0 -> A is double *, 1-> A is sparse double *
int m, // data matrix number of rows
int n, // data matrix number of columns
int nu, // dimension of solution
int m_b, // working dimension
int maxit, // maximum number of Lanzcos iterations
int restart, // 0 -> no restart, 1 -> restarted form
double tol, // convergence tolerance
double *scale, // optional scale (NULL for no scale) length n * 2 (1st n scale values 2nd n work)
double *shift, // optional shift (NULL for no shift) length 1
double *center,// optional center (NULL for no center) length n
double *s, // output singular vectors at least length nu
double *U, // output left singular vectors length >= m x m_b
double *V, // output right singular vectors length >= n x m_b
int *ITER, // output number of iterations performed
int *MPROD, // output number of matrix vector products
double eps, // machine epsilon
int lwork, // length for some intermediate values below
double *V1, // working storage n * work
double *U1, // working storage m * work
double *W, // working storage m * work
double *F, // working storage n
double *B, // working storage work * work
double *BU, // working storage work * work
double *BV, // working storage work * work
double *BS, // working storage work
double *BW, // working storage lwork * lwork
double *res, // working storage work
double *T, // working storage lwork
double svtol, // svtol tolerance on maximum ratio change per singular value per iteration
double *SVRATIO); // working storage nu
irlba/src/Makevars 0000644 0001760 0000144 00000000061 13430560161 013652 0 ustar ripley users PKG_LIBS = $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
irlba/src/utility.c 0000644 0001760 0000144 00000005442 14153271424 014041 0 ustar ripley users /*
* irlb: Implicitly restarted Lanczos bidiagonalization partial SVD.
* Copyright (c) 2016 by Bryan W. Lewis
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
* You should have received a copy of the GNU General Public License
* along with this program. If not, see .
*/
#include
#include
#include
#include
#include
#define USE_FC_LEN_T
#include
#include
#ifndef FCONE
# define FCONE
#endif
#include "Rinternals.h"
#include "irlb.h"
/* orthog(X,Y,...)
* compute Y = Y - X * t(X) * Y
* xm,xn: nrow, ncol X
* yn: ncol Y (ASSUMED TO BE 1)
* On entry, number of rows of Y must be xm to compute t(X) * Y and
* T must be allocated of at least size xn * yn.
* Modifies contents of Y.
*/
void
orthog (double *X, double *Y, double *T, int xm, int xn, int yn)
{
double a = 1, b = 1;
int inc = 1;
memset (T, 0, xn * yn * sizeof (double));
// T = t(X) * Y
F77_CALL (dgemv) ("t", &xm, &xn, &a, X, &xm, Y, &inc, &b, T, &inc FCONE);
// Y = Y - X * T
a = -1.0;
b = 1.0;
F77_CALL (dgemv) ("n", &xm, &xn, &a, X, &xm, T, &inc, &b, Y, &inc FCONE);
}
/*
* Convergence tests
* Input parameters
* Bsz number of rows of the bidiagonal matrix B (scalar)
* tol convergence tolerance (scalar)
* svtol max change in each singular value tolerance (scalar)
* n requested number of singular values
* Smax largest singular value of B
* svratio vector of abs(current - previous) / current singular value ratios
* residuals vector of residual values
* k number of estimated signular values (scalar)
* S check for invariant subspace when S == 0
*
* Output
* converged 0 = FALSE, 1 = TRUE (all converged)
* k adjusted subspace size.
*/
void
convtests (int Bsz, int n, double tol, double svtol, double Smax,
double *svratio, double *residuals, int *k, int *converged, double S)
{
int j, Len_res = 0;
for (j = 0; j < Bsz; j++)
{
if ((fabs (residuals[j]) < tol * Smax) && (svratio[j] < svtol))
Len_res++;
}
if (Len_res >= n || S == 0)
{
*converged = 1;
return;
}
if (*k < n + Len_res)
*k = n + Len_res;
if (*k > Bsz - 3)
*k = Bsz - 3;
if (*k < 1)
*k = 1;
*converged = 0;
return;
}
irlba/src/irlb.c 0000644 0001760 0000144 00000046176 14316576142 013305 0 ustar ripley users /*
* irlb: Implicitly restarted Lanczos bidiagonalization partial SVD.
* Copyright (c) 2016 by Bryan W. Lewis
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
* You should have received a copy of the GNU General Public License
* along with this program. If not, see .
*/
#include
#include
#include
#include
#include
#define USE_FC_LEN_T
#include
#include "R_ext/BLAS.h"
#ifndef FCONE
# define FCONE
#endif
#include
#define USE_RINTERNALS
#include
#include
#include "R_ext/Lapack.h"
#include "R_ext/Rdynload.h"
#include "R_ext/Utils.h"
#include "R_ext/Parse.h"
#include "Matrix.h"
#include "Matrix_stubs.c"
#include "irlb.h"
/* helper function for calling rnorm below */
SEXP
RNORM (int n)
{
char buf[4096];
SEXP cmdSexp, cmdexpr, ans = R_NilValue;
ParseStatus status;
cmdSexp = PROTECT (allocVector (STRSXP, 1));
snprintf (buf, 4095, "rnorm(%d)", n);
SET_STRING_ELT (cmdSexp, 0, mkChar (buf));
cmdexpr = PROTECT (R_ParseVector (cmdSexp, -1, &status, R_NilValue));
if (status != PARSE_OK)
{
UNPROTECT (2);
error ("invalid call");
}
for (int i = 0; i < length (cmdexpr); i++)
{
ans = PROTECT (eval (VECTOR_ELT (cmdexpr, i), R_GlobalEnv));
UNPROTECT (1);
}
UNPROTECT (2);
return ans;
}
/* irlb C implementation wrapper for R
*
* X double precision input matrix
* NU integer number of singular values/vectors to compute must be > 3
* INIT double precision starting vector length(INIT) must equal ncol(X)
* WORK integer working subspace dimension must be > NU
* MAXIT integer maximum number of iterations
* TOL double tolerance
* EPS double invariant subspace detection tolerance
* MULT integer 0 X is a dense matrix (dgemm), 1 sparse (cholmod)
* RESTART integer 0 no or > 0 indicates restart of dimension n
* RV, RW, RS optional restart V W and S values of dimension RESTART
* (only used when RESTART > 0)
* SCALE either NULL (no scaling) or a vector of length ncol(X)
* SHIFT either NULL (no shift) or a single double-precision number
* CENTER either NULL (no centering) or a vector of length ncol(X)
* SVTOL double tolerance max allowed per cent change in each estimated singular value
*
* Returns a list with 6 elements:
* 1. vector of estimated singular values
* 2. matrix of estimated left singular vectors
* 3. matrix of estimated right singular vectors
* 4. number of algorithm iterations
* 5. number of matrix vector products
* 6. irlb C algorithm return error code (see irlb below)
*/
SEXP
IRLB (SEXP X, SEXP NU, SEXP INIT, SEXP WORK, SEXP MAXIT, SEXP TOL, SEXP EPS,
SEXP MULT, SEXP RESTART, SEXP RV, SEXP RW, SEXP RS, SEXP SCALE,
SEXP SHIFT, SEXP CENTER, SEXP SVTOL)
{
SEXP ANS, S, U, V;
double *V1, *U1, *W, *F, *B, *BU, *BV, *BS, *BW, *res, *T, *scale, *shift,
*center, *SVRATIO;
int i, iter, mprod, ret;
int m, n;
int mult = INTEGER (MULT)[0];
void *AS = NULL;
double *A = NULL;
switch (mult)
{
case 1:
AS = (void *) AS_CHM_SP (X);
int *dims = INTEGER (GET_SLOT (X, install ("Dim")));
m = dims[0];
n = dims[1];
break;
default:
A = REAL (X);
m = nrows (X);
n = ncols (X);
}
int nu = INTEGER (NU)[0];
int work = INTEGER (WORK)[0];
int maxit = INTEGER (MAXIT)[0];
double tol = REAL (TOL)[0];
double svtol = REAL (SVTOL)[0];
int lwork = 7 * work * (1 + work);
int restart = INTEGER (RESTART)[0];
double eps = REAL (EPS)[0];
PROTECT (ANS = NEW_LIST (6));
PROTECT (S = allocVector (REALSXP, nu));
PROTECT (U = allocVector (REALSXP, m * work));
PROTECT (V = allocVector (REALSXP, n * work));
if (restart == 0)
for (i = 0; i < n; ++i)
(REAL (V))[i] = (REAL (INIT))[i];
/* set up intermediate working storage */
scale = NULL;
shift = NULL;
center = NULL;
if (TYPEOF (SCALE) == REALSXP)
{
scale = (double *) R_alloc (n * 2, sizeof (double));
memcpy (scale, REAL (SCALE), n * sizeof (double));
}
if (TYPEOF (SHIFT) == REALSXP)
{
shift = REAL (SHIFT);
}
if (TYPEOF (CENTER) == REALSXP)
{
center = REAL (CENTER);
}
SVRATIO = (double *) R_alloc (work, sizeof (double));
V1 = (double *) R_alloc (n * work, sizeof (double));
U1 = (double *) R_alloc (m * work, sizeof (double));
W = (double *) R_alloc (m * work, sizeof (double));
F = (double *) R_alloc (n, sizeof (double));
B = (double *) R_alloc (work * work, sizeof (double));
BU = (double *) R_alloc (work * work, sizeof (double));
BV = (double *) R_alloc (work * work, sizeof (double));
BS = (double *) R_alloc (work, sizeof (double));
BW = (double *) R_alloc (lwork, sizeof (double));
res = (double *) R_alloc (work, sizeof (double));
T = (double *) R_alloc (lwork, sizeof (double));
if (restart > 0)
{
memcpy (REAL (V), REAL (RV), n * (restart + 1) * sizeof (double));
memcpy (W, REAL (RW), m * restart * sizeof (double));
memset (B, 0, work * work * sizeof (double));
for (i = 0; i < restart; ++i)
B[i + work * i] = REAL (RS)[i];
}
ret =
irlb (A, AS, mult, m, n, nu, work, maxit, restart, tol, scale, shift, center,
REAL (S), REAL (U), REAL (V), &iter, &mprod, eps, lwork, V1, U1, W,
F, B, BU, BV, BS, BW, res, T, svtol, SVRATIO);
SET_VECTOR_ELT (ANS, 0, S);
SET_VECTOR_ELT (ANS, 1, U);
SET_VECTOR_ELT (ANS, 2, V);
SET_VECTOR_ELT (ANS, 3, ScalarInteger (iter));
SET_VECTOR_ELT (ANS, 4, ScalarInteger (mprod));
SET_VECTOR_ELT (ANS, 5, ScalarInteger (ret));
UNPROTECT (4);
return ANS;
}
/* irlb: main computation function.
* returns:
* 0 on success,
* -1 invalid dimensions,
* -2 not converged
* -3 out of memory
* -4 starting vector near the null space of A
*
* all data must be allocated by caller, required sizes listed below
*/
int
irlb (double *A, // Input data matrix (double case)
void *AS, // input data matrix (sparse case)
int mult, // 0 -> use double *A, 1 -> use AS
int m, // data matrix number of rows, must be > 3.
int n, // data matrix number of columns, must be > 3.
int nu, // dimension of solution
int work, // working dimension, must be > 3.
int maxit, // maximum number of main iterations
int restart, // 0->no, n>0 -> restarted algorithm of dimension n
double tol, // convergence tolerance
double *scale, // optional scale (NULL for no scale) size n * 2
double *shift, // optional shift (NULL for no shift)
double *center, // optional center (NULL for no center)
// output values
double *s, // output singular values at least length nu
double *U, // output left singular vectors m x work
double *V, // output right singular vectors n x work
int *ITER, // ouput number of Lanczos iterations
int *MPROD, // output number of matrix vector products
double eps, // tolerance for invariant subspace detection
// working intermediate storage, sizes shown
int lwork, double *V1, // n x work
double *U1, // m x work
double *W, // m x work input when restart > 0
double *F, // n
double *B, // work x work input when restart > 0
double *BU, // work x work
double *BV, // work x work
double *BS, // work
double *BW, // lwork
double *res, // work
double *T, // lwork
double svtol, // svtol limit
double *svratio) // convtest extra storage vector of length work
{
double d, S, R, alpha, beta, R_F, SS;
double *x;
int jj, kk;
int converged;
int info, j, k = restart;
int inc = 1;
int mprod = 0;
int iter = 0;
double Smax = 0;
SEXP FOO;
/* Check for valid input dimensions */
if (work < 4 || n < 4 || m < 4)
return -1;
if (restart == 0)
memset (B, 0, work * work * sizeof (double));
memset(svratio, 0, work * sizeof(double));
/* Main iteration */
while (iter < maxit)
{
j = 0;
/* Normalize starting vector */
if (iter == 0 && restart == 0)
{
d = F77_CALL (dnrm2) (&n, V, &inc);
if (d < eps)
return -1;
d = 1 / d;
F77_CALL (dscal) (&n, &d, V, &inc);
}
else
j = k;
/* optionally apply scale */
x = V + j * n;
if (scale)
{
x = scale + n;
memcpy (scale + n, V + j * n, n * sizeof (double));
for (kk = 0; kk < n; ++kk)
x[kk] = x[kk] / scale[kk];
}
switch (mult)
{
case 1:
dsdmult ('n', m, n, AS, x, W + j * m);
break;
default:
alpha = 1;
beta = 0;
F77_CALL (dgemv) ("n", &m, &n, &alpha, (double *) A, &m, x,
&inc, &beta, W + j * m, &inc FCONE);
}
mprod++;
R_CheckUserInterrupt ();
/* optionally apply shift in square cases m = n */
if (shift)
{
jj = j * m;
for (kk = 0; kk < m; ++kk)
W[jj + kk] = W[jj + kk] + shift[0] * x[kk];
}
/* optionally apply centering */
if (center)
{
jj = j * m;
beta = F77_CALL (ddot) (&n, x, &inc, center, &inc);
for (kk = 0; kk < m; ++kk)
W[jj + kk] = W[jj + kk] - beta;
}
if (iter > 0)
orthog (W, W + j * m, T, m, j, 1);
S = F77_CALL (dnrm2) (&m, W + j * m, &inc);
if (S < eps && j == 0)
return -4;
SS = 1.0 / S;
F77_CALL (dscal) (&m, &SS, W + j * m, &inc);
/* The Lanczos process */
while (j < work)
{
switch (mult)
{
case 1:
dsdmult ('t', m, n, AS, W + j * m, F);
break;
default:
alpha = 1.0;
beta = 0.0;
F77_CALL (dgemv) ("t", &m, &n, &alpha, (double *) A, &m,
W + j * m, &inc, &beta, F, &inc FCONE);
}
mprod++;
R_CheckUserInterrupt ();
/* optionally apply shift, scale, center */
if (shift)
{
// Note, not a bug because shift only applies to square matrices
for (kk = 0; kk < m; ++kk)
F[kk] = F[kk] + shift[0] * W[j * m + kk];
}
if (scale)
{
for (kk = 0; kk < n; ++kk)
F[kk] = F[kk] / scale[kk];
}
if (center)
{
beta = 0;
for (kk = 0; kk < m; ++kk) beta += W[j *m + kk];
if (scale)
for (kk = 0; kk < n; ++kk)
F[kk] = F[kk] - beta * center[kk] / scale[kk];
else
for (kk = 0; kk < n; ++kk)
F[kk] = F[kk] - beta * center[kk];
}
SS = -S;
F77_CALL (daxpy) (&n, &SS, V + j * n, &inc, F, &inc);
orthog (V, F, T, n, j + 1, 1);
if (j + 1 < work)
{
R_F = F77_CALL (dnrm2) (&n, F, &inc);
R = 1.0 / R_F;
if (R_F < eps) // near invariant subspace
{
FOO = RNORM (n);
for (kk = 0; kk < n; ++kk)
F[kk] = REAL (FOO)[kk];
orthog (V, F, T, n, j + 1, 1);
R_F = F77_CALL (dnrm2) (&n, F, &inc);
R = 1.0 / R_F;
R_F = 0;
}
memmove (V + (j + 1) * n, F, n * sizeof (double));
F77_CALL (dscal) (&n, &R, V + (j + 1) * n, &inc);
B[j * work + j] = S;
B[(j + 1) * work + j] = R_F;
/* optionally apply scale */
x = V + (j + 1) * n;
if (scale)
{
x = scale + n;
memcpy (x, V + (j + 1) * n, n * sizeof (double));
for (kk = 0; kk < n; ++kk)
x[kk] = x[kk] / scale[kk];
}
switch (mult)
{
case 1:
dsdmult ('n', m, n, AS, x, W + (j + 1) * m);
break;
default:
alpha = 1.0;
beta = 0.0;
F77_CALL (dgemv) ("n", &m, &n, &alpha, (double *) A, &m,
x, &inc, &beta, W + (j + 1) * m, &inc FCONE);
}
mprod++;
R_CheckUserInterrupt ();
/* optionally apply shift */
if (shift)
{
jj = j + 1;
for (kk = 0; kk < m; ++kk)
W[jj * m + kk] = W[jj * m + kk] + shift[0] * x[kk];
}
/* optionally apply centering */
if (center)
{
jj = (j + 1) * m;
beta = F77_CALL (ddot) (&n, x, &inc, center, &inc);
for (kk = 0; kk < m; ++kk)
W[jj + kk] = W[jj + kk] - beta;
}
/* One step of classical Gram-Schmidt */
R = -R_F;
F77_CALL (daxpy) (&m, &R, W + j * m, &inc, W + (j + 1) * m,
&inc);
/* full re-orthogonalization of W_{j+1} */
orthog (W, W + (j + 1) * m, T, m, j + 1, 1);
S = F77_CALL (dnrm2) (&m, W + (j + 1) * m, &inc);
SS = 1.0 / S;
if (S < eps)
{
FOO = RNORM (m);
jj = (j + 1) * m;
for (kk = 0; kk < m; ++kk)
W[jj + kk] = REAL (FOO)[kk];
orthog (W, W + (j + 1) * m, T, m, j + 1, 1);
S = F77_CALL (dnrm2) (&m, W + (j + 1) * m, &inc);
SS = 1.0 / S;
F77_CALL (dscal) (&m, &SS, W + (j + 1) * m, &inc);
S = 0;
}
else
F77_CALL (dscal) (&m, &SS, W + (j + 1) * m, &inc);
}
else
{
B[j * work + j] = S;
}
j++;
}
memmove (BU, B, work * work * sizeof (double)); // Make a working copy of B
int *BI = (int *) T;
F77_CALL (dgesdd) ("O", &work, &work, BU, &work, BS, BU, &work, BV,
&work, BW, &lwork, BI, &info FCONE);
R_F = F77_CALL (dnrm2) (&n, F, &inc);
R = 1.0 / R_F;
F77_CALL (dscal) (&n, &R, F, &inc);
/* Force termination after encountering linear dependence */
if (R_F < eps)
R_F = 0;
Smax = 0;
for (jj = 0; jj < j; ++jj)
{
if (BS[jj] > Smax)
Smax = BS[jj];
svratio[jj] = fabs (svratio[jj] - BS[jj]) / BS[jj];
}
for (kk = 0; kk < j; ++kk)
res[kk] = R_F * BU[kk * work + (j - 1)];
/* Update k to be the number of converged singular values. */
convtests (j, nu, tol, svtol, Smax, svratio, res, &k, &converged, S);
if (converged == 1)
{
iter++;
break;
}
for (jj = 0; jj < j; ++jj)
svratio[jj] = BS[jj];
alpha = 1;
beta = 0;
F77_CALL (dgemm) ("n", "t", &n, &k, &j, &alpha, V, &n, BV, &work, &beta,
V1, &n FCONE FCONE);
memmove (V, V1, n * k * sizeof (double));
memmove (V + n * k, F, n * sizeof (double));
memset (B, 0, work * work * sizeof (double));
for (jj = 0; jj < k; ++jj)
{
B[jj * work + jj] = BS[jj];
B[k * work + jj] = res[jj];
}
/* Update the left approximate singular vectors */
alpha = 1;
beta = 0;
F77_CALL (dgemm) ("n", "n", &m, &k, &j, &alpha, W, &m, BU, &work, &beta,
U1, &m FCONE FCONE);
memmove (W, U1, m * k * sizeof (double));
iter++;
}
/* Results */
memmove (s, BS, nu * sizeof (double)); /* Singular values */
alpha = 1;
beta = 0;
F77_CALL (dgemm) ("n", "n", &m, &nu, &work, &alpha, W, &m, BU, &work, &beta,
U, &m FCONE FCONE);
F77_CALL (dgemm) ("n", "t", &n, &nu, &work, &alpha, V, &n, BV, &work, &beta,
V1, &n FCONE FCONE);
memmove (V, V1, n * nu * sizeof (double));
*ITER = iter;
*MPROD = mprod;
return (converged == 1 ? 0 : -2);
}
cholmod_common chol_c;
/* Need our own CHOLMOD error handler */
void attribute_hidden
irlba_R_cholmod_error (int status, const char *file, int line,
const char *message)
{
if (status < 0)
error ("Cholmod error '%s' at file:%s, line %d", message, file, line);
else
warning ("Cholmod warning '%s' at file:%s, line %d", message, file, line);
}
static const R_CallMethodDef CallEntries[] = {
{"IRLB", (DL_FUNC) & IRLB, 16},
{NULL, NULL, 0}
};
#ifdef HAVE_VISIBILITY_ATTRIBUTE
__attribute__ ((visibility ("default")))
#endif
void
R_init_irlba (DllInfo * dll)
{
R_RegisterCCallable("irlba", "orthog",
(DL_FUNC) &orthog);
R_RegisterCCallable("irlba", "irlb",
(DL_FUNC) &irlb);
R_registerRoutines (dll, NULL, CallEntries, NULL, NULL);
R_useDynamicSymbols (dll, 0);
M_R_cholmod_start (&chol_c);
chol_c.final_ll = 1; /* LL' form of simplicial factorization */
/* need own error handler, that resets final_ll (after *_defaults()) : */
chol_c.error_handler = irlba_R_cholmod_error;
}
void
R_unload_irlba (DllInfo * dll)
{
M_cholmod_finish (&chol_c);
}
typedef int (*cholmod_sdmult_func)
(
/* ---- input ---- */
cholmod_sparse *A, /* sparse matrix to multiply */
int transpose, /* use A if 0, or A' otherwise */
double alpha [2], /* scale factor for A */
double beta [2], /* scale factor for Y */
cholmod_dense *X, /* dense matrix to multiply */
/* ---- in/out --- */
cholmod_dense *Y, /* resulting dense matrix */
/* --------------- */
cholmod_common *Common
);
void
dsdmult (char transpose, int m, int n, void * a, double *b, double *c)
{
cholmod_sdmult_func sdmult;
sdmult = (cholmod_sdmult_func) R_GetCCallable ("Matrix", "cholmod_sdmult");
int t = transpose == 't' ? 1 : 0;
CHM_SP cha = (CHM_SP) a;
cholmod_dense chb;
chb.nrow = transpose == 't' ? m : n;
chb.d = chb.nrow;
chb.ncol = 1;
chb.nzmax = chb.nrow;
chb.xtype = cha->xtype;
chb.dtype = 0;
chb.x = (void *) b;
chb.z = (void *) NULL;
cholmod_dense chc;
chc.nrow = transpose == 't' ? n : m;
chc.d = chc.nrow;
chc.ncol = 1;
chc.nzmax = chc.nrow;
chc.xtype = cha->xtype;
chc.dtype = 0;
chc.x = (void *) c;
chc.z = (void *) NULL;
double one[] = { 1, 0 }, zero[] = { 0, 0};
sdmult (cha, t, one, zero, &chb, &chc, &chol_c);
}
irlba/vignettes/ 0000755 0001760 0000144 00000000000 14153273722 013411 5 ustar ripley users irlba/vignettes/irlba.Rnw 0000644 0001760 0000144 00000040151 13430560161 015164 0 ustar ripley users % \VignetteIndexEntry{irlba Manual}
% \VignetteDepends{irlba}
% \VignettePackage{irlba}
\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage[pdftex]{graphicx}
\usepackage{color}
\usepackage{xspace}
\usepackage{fancyvrb}
\usepackage{fancyhdr}
\usepackage[
colorlinks=true,
linkcolor=blue,
citecolor=blue,
urlcolor=blue]
{hyperref}
\usepackage{lscape}
\usepackage{Sweave}
\usepackage{tabularx}
\usepackage{listings}
\usepackage{mdwlist}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% define new colors for use
\definecolor{darkgreen}{rgb}{0,0.6,0}
\definecolor{darkred}{rgb}{0.6,0.0,0}
\definecolor{lightbrown}{rgb}{1,0.9,0.8}
\definecolor{brown}{rgb}{0.6,0.3,0.3}
\definecolor{darkblue}{rgb}{0,0,0.8}
\definecolor{darkmagenta}{rgb}{0.5,0,0.5}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newcommand{\bld}[1]{\mbox{\boldmath $#1$}}
\newcommand{\shell}[1]{\mbox{$#1$}}
\renewcommand{\vec}[1]{\mbox{\bf {#1}}}
\newcommand{\ReallySmallSpacing}{\renewcommand{\baselinestretch}{.6}\Large\normalsize}
\newcommand{\SmallSpacing}{\renewcommand{\baselinestretch}{1.1}\Large\normalsize}
\def\tm{\leavevmode\hbox{$\rm {}^{TM}$}}
\newcommand{\R}{{\mathbf R}}
\newcommand{\brho}{{\color{blue}{\rho}}}
\newcommand{\Ra}{{\mathcal R}}
\newcommand{\PP}{{\mathbf P}}
\newcommand{\N}{{\mathbf N}}
\newcommand{\K}{{\mathcal K}}
\setlength{\oddsidemargin}{-.25 truein}
\setlength{\evensidemargin}{0truein}
\setlength{\topmargin}{-0.2truein}
\setlength{\textwidth}{7 truein}
\setlength{\textheight}{8.5 truein}
\setlength{\parindent}{0.20truein}
\setlength{\parskip}{0.10truein}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\pagestyle{fancy}
\lhead{}
\chead{The {\tt irlba} Package}
\rhead{}
\lfoot{}
\cfoot{}
\rfoot{\thepage}
\renewcommand{\headrulewidth}{1pt}
\renewcommand{\footrulewidth}{1pt}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\title{The {\tt irlba} Package}
\author{Bryan W. Lewis \\
blewis@illposed.net,
\\[6pt]
adapted from the work of:\\
Jim Baglama (University of Rhode Island)\\
and Lothar Reichel (Kent State University).
}
\begin{document}
\maketitle
\thispagestyle{empty}
\section{Introduction}
The {\tt irlba} package provides a fast way to compute partial singular value
decompositions (SVD) of large sparse or dense matrices. Recent additions to the
package can also compute fast partial symmetric eigenvalue decompositions and
principal components. The package is an R implementation of the {\it augmented
implicitly restarted Lanczos bidiagonalization algorithm} of Jim Baglama and
Lothar Reichel\footnote{Augmented Implicitly Restarted Lanczos
Bidiagonalization Methods, J. Baglama and L. Reichel, SIAM J. Sci. Comput.
2005.}. Source code is maintained at
\href{https://github.com/bwlewis/irlba}{https://github.com/bwlewis/irlba}.
The {\tt irlba} package works with real- and complex-valued dense R matrices
and real-valued sparse matrices from the {\tt Matrix} package. It provides
several easy ways to define custom matrix arithmetic that works with other
matrix classes including {\tt big.matrix} from the {\tt bigmemory} package and
others. The {\tt irlba} is both faster and more memory efficient than the
usual R {\tt svd} function for computing a few of the largest singular vectors
and corresponding singular values of a matrix. It takes advantage of available
high-performance linear algebra libraries if R is compiled to use them. In
particular, the package uses the same BLAS and LAPACK libraries that R uses
(see
\href{https://cran.r-project.org/doc/manuals/R-admin.html#BLAS}{https://cran.r-project.org/doc/manuals/R-admin.html\#BLAS}),
or the CHOLMOD library from R's Matrix package for sparse matrix problems.
A whirlwind summary of the algorithm follows, along with a few basic examples.
A much more detailed description and discussion of the algorithm may be found
in the cited Baglama-Reichel reference.
\section{Partial Singular Value Decomposition}
Let $A\in\R^{\ell\times n}$ and assume $\ell\ge n$. These notes simplify the
presentation by considering only real-valued matrices and assuming without
losing generality that there are at least as many rows as columns (the
method works more generally). A singular
value decomposition of $A$ can be expressed as:
\[
A = \sum_{j=1}^n \sigma_j u_j v_j^T,
\phantom{xxxxxxxx}
v_j^Tv_k = u_j^Tu_k =
\left\{
\begin{array}{ll}
1 & \mbox{if}\phantom{x} j=k,\\
0 & \mbox{o.w.,}\\
\end{array}
\right.
\]
where $u_j\in\R^\ell $, $v_j\in\R^n $,
$j=1,2,\ldots, n$, and
$ \sigma_1 \ge \sigma_2 \ge \cdots \ge \sigma_n \ge 0 $.
Let $1 \le k library('irlba')
> set.seed(1)
> A <- matrix(rnorm(5000*5000), 5000)
> t1 <- proc.time()
> L <- irlba(A, 5)
> print(proc.time() - t1)
user system elapsed
17.440 0.192 4.417
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 1096734 58.6 1770749 94.6 1442291 77.1
Vcells 26685618 203.6 62229965 474.8 52110704 397.6
\end{lstlisting}
Compare with the standard {\tt svd} function:\newpage
\begin{lstlisting}
> t1 <- proc.time()
> S <- svd(A, nu=5, nv=5)
> print(proc.time() - t1)
user system elapsed
277.092 11.552 74.425
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 1097441 58.7 1770749 94.6 1442291 77.1
Vcells 26741910 204.1 169891972 1296.2 176827295 1349.1
\end{lstlisting}
The {\tt irlba} method uses about 1/20 elapsed time as the
{\tt svd} method in this example and less than one third the peak memory.
The defalut tolerance value yields the following relative error
in the estimated singular values:
\begin{lstlisting}
> sqrt (crossprod(S$d[1:5]-L$d)/crossprod(S$d[1:5]))
[,1]
[1,] 4.352641e-10
\end{lstlisting}
\subsection{Convergence tolerance}
IRLBA is an iterative method that estimates a few singular values
and associated singular vectors. A sketch of the algorithm is outlined
in Section \ref{sketch} below. The R {\tt tol} and {\tt svtol} arguments control
when the algorithm converges with {\tt tol} specifying
subspace convergence, and {\tt svtol} specifying convergence of estimated
singular values.
Subspace convergence occurs when the algorithm iterations find
estimated singular vectors that satisfy
\[
\|AV_k - US_k\| < \mbox{tol} \cdot \|A\|,
\]
where $\|\cdot\|$ means spectral matrix norm, $A$ is the matrix, $V_k$ and $U_k$
are the {\it estimated} right and left $k$ singular vectors computed by the
algorithm, and $\|A\|$ is the {\it estimated} spectral norm of the matrix defined
by the largest singular value computed by the algorithm. Using R notation,
the algorithm stops when
\begin{lstlisting}
L <- irlba(A, k, tol)
svd(A %*% L$v - L$u %*% diag(L$d))$d[1] < tol * L$d[1]
\end{lstlisting}
It's possible to encounter problems that fail to converge before
the maximum number of algorithm iterations specified by the {\tt maxit}
argument.
When the largest singular values are clustered together it can be hard to
detect subspace convergence. More recent versions of the IRLBA implementation
include the {\tt svtol} argument that specifies a maximum for the relative
change in each estimated singular value from one iteration to the next.
The convergence tolerance values together help improve correct subspace
detection in difficult settings when the singular values are clustered.
But in the worst cases, block methods can perform better as shown in
the documentation for the {\tt svdr} method.
Also see the related {\tt rsvd} function by N. Benjamin Erichson,
\href{https://cran.r-project.org/package=rsvd}{https://cran.r-project.org/package=rsvd}.
\subsection{Differences with {\tt svd}}
The {\tt irlba} function is designed to compute a {\it partial} singular
value decomposition. It is largely compatible with the usual R {\tt svd}
function but there are some differences. In particular:
\begin{enumerate}
\item The {\tt irlba} function only computes the number of singular values
corresponding to the maximum of the desired singular vectors,
{\tt max(nu, nv)}. For example, if 5
singular vectors are desired ({\tt nu=nv=5}), then only the five corresponding
singular values are computed. The standard R {\tt svd} function always
returns the {\it total} set of singular values for the matrix, regardless of how
many singular vectors are specified.
\item The {\tt irlba} function is an iterative method that continues until
either a tolerance or maximum number of iterations is reached.
Problems with difficult convergence properties are not likely to be
encountered, but the method will fail with an error after the iteration limit
is reached in those cases.
\end{enumerate}
Watch out especially for the first difference noted above!
\subsection{Principal Components}
Version 2.1.0 of the package introduces optional arguments and {\tt prcomp}-like
function syntax for efficiently computing partial SVDs of matrices after
centering and scaling their columns and other adjustments.
Use the following arguments to the {\tt irlba} function, or the new
{\tt irlba\_prcomp} function for PCA:
\begin{itemize}
\item {\tt center}: if {\tt center} is a numeric vector with length equal to
the number of columns of the matrix, then each column of the matrix has the
corresponding value from {\tt center} subtracted from it.
\item {\tt scale}: if 'scale' is a numeric vector with length
equal to the number of columns of the matrix, then each column is
divided by the corresponding value from {\tt scale}.
\end{itemize}
Both centering and scaling options are performed implicitly in the algorithm
and, for instance, do not affect sparsity of the input matrix or increase
storage requirements.
The following
example compares the output of the usual {\tt prcomp} function with
output from {\tt irlba}.
Note that in general, singular vectors and principal component vectors
are only unique up to sign!
\begin{lstlisting}
> set.seed(1)
> x <- matrix(rnorm(200), nrow=20)
> p1 <- prcomp_irlba(x, n=3)
> summary(p1)
Importance of components%s:
PC1 PC2 PC3
Standard deviation 1.5411 1.2513 1.1916
Proportion of Variance 0.2806 0.1850 0.1678
Cumulative Proportion 0.2806 0.4656 0.6334
> # Compare with
> p2 <- prcomp(x, tol=0.7)
> summary(p2)
Importance of components:
PC1 PC2 PC3
Standard deviation 1.5411 1.2513 1.1916
Proportion of Variance 0.2806 0.1850 0.1678
Cumulative Proportion 0.2806 0.4656 0.6334
\end{lstlisting}
Alternatively, you can compute principal components directly using the
singular value decomposition and the {\tt center} option:
\begin{lstlisting}
> p3 <- svd(scale(x, center=colMeans(x), scale=FALSE))
> p4 <- irlba(x, 3, center=colMeans(x))
> # compare with prcomp
> sqrt(crossprod(p1$rotation[,1] - p3$v[,1]))
[,1]
[1,] 9.773228e-13
> sqrt(crossprod(p1$rotation[,1] + p4$v[,1]))
[,1]
[1,] 1.652423e-12
\end{lstlisting}
\subsection{Truncated symmetric eigenvalue decomposition}
Use the {\tt partial\_eigen} function to estimate a subset of the largest (most
positive) eigenvalues and corresponding eigenvectors of a symmetric dense or
sparse real-valued matrix. The function is particularly well-suited to
estimating the largest eigenvalues and corresponding eigenvectors of symmetric
positive semi-definite matrices of the form $A^T A$.
\subsection{User-Defined Matrix Multiplication}
The {\tt irlba} function only uses matrix vector products with the input data
matrix to compute its solution. It's easy to use R's native object model to
define custom matrix classes with user-defined matrix multiplication functions.
Such functions can be used to support special matrix objects, out of core
computation of large problems, or matrix-free operators.
Here is a simple example that defines a matrix product that scales the
columns of the matrix to have unit norm (cf the {\tt scale} option).
\begin{lstlisting}
> A <- matrix(runif(400), nrow=20)
> col_scale <- sqrt(apply(A, 2, crossprod))
> setClass("scaled_matrix", contains="matrix", slots=c(scale="numeric"))
> setMethod("%*%", signature(x="scaled_matrix", y="numeric"),
+ function(x ,y) x@.Data %*% (y / x@scale))
> setMethod("%*%", signature(x="numeric", y="scaled_matrix"),
+ function(x ,y) (x %*% y@.Data) / y@scale)
> a <- new("scaled_matrix", A, scale=col_scale)
> irlba(a, 3)$d
[1] 3.9298391 0.9565016 0.8266859
# Compare with
> svd(sweep(A, 2, col_scale, FUN=`/`))$d[1:3]
[1] 3.9298391 0.9565016 0.8266859
\end{lstlisting}
See the following link for an example that uses large-scale out of core computation:
\href{https://bwlewis.github.io/1000_genomes_examples/PCA_whole_genome.html}{http://bwlewis.github.io/1000\_genomes\_examples/PCA\_whole\_genome.html}
NOTE! The reference R algorithm implementation is used whenever user-defined
matrix multiplication is specified (instead of the faster C code path).
\section{A Quick Summary of the IRLBA Method}\label{sketch}
\subsection{Partial Lanczos Bidiagonalization}
Start with a given vector $p_1$. Compute $m$ steps of the Lanczos process:
\begin{eqnarray*}
A P_m &=& Q_m B_m \\
A^T Q_m &=& P_m B_m^T + r_m e_m^T,\\
\end{eqnarray*}
$B_m\in\R^{m\times m}, P_m \in \R^{n\times m}, $
$Q_m \in \R^{\ell \times m},$
$P_m^TP_m=Q_m^TQ_m=I_m, $
$r_m\in\R^n, P_m^Tr_m=0,$
$P_m = [p_1, p_2, \ldots, p_m]$.
\subsection{Approximating Partial SVD with A Partial Lanczos bidiagonalization}
\begin{eqnarray*}
A^TA P_m &=& A^TQ_m B_m \\
&=& P_m {\color{blue}{B_m^TB_m}} + r_m e_m^TB_m,\\
\end{eqnarray*}
\begin{eqnarray*}
AA^T Q_m &=& AP_m B_m^T + Ar_m e_m^T,\\
&=& Q_m{\color{blue}{B_mB_m^T}} + Ar_me_m^T.
\end{eqnarray*}
Compute the SVD of $B_m$:
\[
B_m = \sum_{j=1}^m\sigma^B_ju^B_j\left(v_j^B\right)^T.
\]
\\[6pt]
\[
\left(\mbox{i.e., } B_mv_j^B = \sigma_j^Bu_j^B, \mbox{ and }
B_m^Tu_j^b = \sigma_j^Bv_j^B.\right)
\]
Define:
$
\tilde{\sigma_j} := \sigma_j^B, \phantom{xxx}
\tilde{u}_j := Q_m u_j^B, \phantom{xxx}
\tilde{v}_j := P_m v_j^B.
$
Then:
\begin{eqnarray*}
A\tilde{v}_j &=& AP_mv_j^B \\
&=& Q_mB_mv_j^B \\
&=& \sigma^B_jQ_mu_j^B \\
&=& \tilde{\sigma}_j \tilde{u}_j,
\end{eqnarray*}
and
\begin{eqnarray*}
A^T\tilde{u}_j &=& A^TQ_mu_j^B \\
&=& P_mB^T_mu_j^B + r_me_m^Tu_j^B \\
&=& \sigma^B_jP_mv_j^B + r_me_m^Tu_j^B\\
&=& \tilde{\sigma}_j \tilde{v}_j + {\color{red} {r_me_m^Tu_j^B}}.
\end{eqnarray*}
The part in red above represents the error with respect to the exact SVD.
The IRLBA strategy is to iteratively reduce the norm of that error term
by augmenting and restarting.
Here is the overall method:
\begin{enumerate}
\item Compute the Lanczos process up to step $m$.
\item Compute $k= k_org) return(list(converged=TRUE, k=k))
if (S == 0) return(list(converged=TRUE, k=k))
# Not converged yet...
# Adjust k to include more vectors as the number of vectors converge, but not
# too many (maxritz):
augment <- min(sum(subspace_converged), maxritz)
k <- min(max(k, k_org + augment), work - 1)
list(converged=FALSE, k=k)
}
message_once <- function(..., flag)
{
if (flag$flag) return()
flag$flag <- TRUE
message(...)
}
irlba/R/ssvd.R 0000644 0001760 0000144 00000026743 13430560161 012711 0 ustar ripley users #' Sparse regularized low-rank matrix approximation.
#'
#' Estimate an \eqn{{\ell}1}{l1}-penalized
#' singular value or principal components decomposition (SVD or PCA) that introduces sparsity in the
#' right singular vectors based on the fast and memory-efficient
#' sPCA-rSVD algorithm of Haipeng Shen and Jianhua Huang.
#' @param x A numeric real- or complex-valued matrix or real-valued sparse matrix.
#' @param k Matrix rank of the computed decomposition (see the Details section below).
#' @param n Number of nonzero components in the right singular vectors. If \code{k > 1},
#' then a single value of \code{n} specifies the number of nonzero components
#' in each regularized right singular vector. Or, specify a vector of length
#' \code{k} indicating the number of desired nonzero components in each
#' returned vector. See the examples.
#' @param maxit Maximum number of soft-thresholding iterations.
#' @param tol Convergence is determined when \eqn{\|U_j - U_{j-1}\|_F < tol}{||U_j - U_{j-1}||_F < tol}, where \eqn{U_j} is the matrix of estimated left regularized singular vectors at iteration \eqn{j}.
#' @param center a logical value indicating whether the variables should be
#' shifted to be zero centered. Alternately, a centering vector of length
#' equal the number of columns of \code{x} can be supplied. Use \code{center=TRUE}
#' to perform a regularized sparse PCA.
#' @param scale. a logical value indicating whether the variables should be
#' scaled to have unit variance before the analysis takes place.
#' Alternatively, a vector of length equal the number of columns of \code{x} can be supplied.
#'
#' The value of \code{scale} determines how column scaling is performed
#' (after centering). If \code{scale} is a numeric vector with length
#' equal to the number of columns of \code{x}, then each column of \code{x} is
#' divided by the corresponding value from \code{scale}. If \code{scale} is
#' \code{TRUE} then scaling is done by dividing the (centered) columns of
#' \code{x} by their standard deviations if \code{center=TRUE}, and the
#' root mean square otherwise. If \code{scale} is \code{FALSE}, no scaling is done.
#' See \code{\link{scale}} for more details.
#' @param alpha Optional scalar regularization parameter between zero and one (see Details below).
#' @param tsvd Optional initial rank-k truncated SVD or PCA (skips computation if supplied).
#' @param ... Additional arguments passed to \code{\link{irlba}}.
#' @details
#' The \code{ssvd} function implements a version of an algorithm by
#' Shen and Huang that computes a penalized SVD or PCA that introduces
#' sparsity in the right singular vectors by solving a penalized least squares problem.
#' The algorithm in the rank 1 case finds vectors \eqn{u, w}{u, w} that minimize
#' \deqn{\|x - u w^T\|_F^2 + \lambda \|w\|_1}{||x - u w^T||_F^2 + lambda||w||_1}
#' such that \eqn{\|u\| = 1}{||u|| = 1},
#' and then sets \eqn{v = w / \|w\|}{v = w / ||w||} and
#' \eqn{d = u^T x v}{d = u^T x v};
#' see the referenced paper for details. The penalty \eqn{\lambda}{lambda} is
#' implicitly determined from the specified desired number of nonzero values \code{n}.
#' Higher rank output is determined similarly
#' but using a sequence of \eqn{\lambda}{lambda} values determined to maintain the desired number
#' of nonzero elements in each column of \code{v} specified by \code{n}.
#' Unlike standard SVD or PCA, the columns of the returned \code{v} when \code{k > 1} may not be orthogonal.
#'
#' @return
#' A list containing the following components:
#' \itemize{
#' \item{u} {regularized left singular vectors with orthonormal columns}
#' \item{d} {regularized upper-triangluar projection matrix so that \code{x \%*\% v == u \%*\% d}}
#' \item{v} {regularized, sparse right singular vectors with columns of unit norm}
#' \item{center, scale} {the centering and scaling used, if any}
#' \item{lambda} {the per-column regularization parameter found to obtain the desired sparsity}
#' \item{iter} {number of soft thresholding iterations}
#' \item{n} {value of input parameter \code{n}}
#' \item{alpha} {value of input parameter \code{alpha}}
#' }
#' @note
#' Our \code{ssvd} implementation of the Shen-Huang method makes the following choices:
#' \enumerate{
#' \item{The l1 penalty is the only available penalty function. Other penalties may appear in the future.}
#' \item{Given a desired number of nonzero elements in \code{v}, value(s) for the \eqn{\lambda}{lambda}
#' penalty are determined to achieve the sparsity goal subject to the parameter \code{alpha}.}
#' \item{An experimental block implementation is used for results with rank greater than 1 (when \code{k > 1})
#' instead of the deflation method described in the reference.}
#' \item{The choice of a penalty lambda associated with a given number of desired nonzero
#' components is not unique. The \code{alpha} parameter, a scalar between zero and one,
#' selects any possible value of lambda that produces the desired number of
#' nonzero entries. The default \code{alpha = 0} selects a penalized solution with
#' largest corresponding value of \code{d} in the 1-d case. Think of \code{alpha} as
#' fine-tuning of the penalty.}
#' \item{Our method returns an upper-triangular matrix \code{d} when \code{k > 1} so
#' that \code{x \%*\% v == u \%*\% d}. Non-zero
#' elements above the diagonal result from non-orthogonality of the \code{v} matrix,
#' providing a simple interpretation of cumulative information, or explained variance
#' in the PCA case, via the singular value decomposition of \code{d \%*\% t(v)}.}
#' }
#'
#' What if you have no idea for values of the argument \code{n} (the desired sparsity)?
#' The reference describes a cross-validation and an ad-hoc approach; neither of which are
#' in the package yet. Both are prohibitively computationally expensive for matrices with a huge
#' number of columns. A future version of this package will include a revised approach to
#' automatically selecting a reasonable sparsity constraint.
#'
#' Compare with the similar but more general functions \code{SPC} and \code{PMD} in the \code{PMA} package
#' by Daniela M. Witten, Robert Tibshirani, Sam Gross, and Balasubramanian Narasimhan.
#' The \code{PMD} function can compute low-rank regularized matrix decompositions with sparsity penalties
#' on both the \code{u} and \code{v} vectors. The \code{ssvd} function is
#' similar to the PMD(*, L1) method invocation of \code{PMD} or alternatively the \code{SPC} function.
#' Although less general than \code{PMD}(*),
#' the \code{ssvd} function can be faster and more memory efficient for the
#' basic sparse PCA problem.
#' See \url{https://bwlewis.github.io/irlba/ssvd.html} for more information.
#'
#' (* Note that the s4vd package by Martin Sill and Sebastian Kaiser, \url{https://cran.r-project.org/package=s4vd},
#' includes a fast optimized version of a closely related algorithm by Shen, Huang, and Marron, that penalizes
#' both \code{u} and \code{v}.)
#'
#' @references
#' \itemize{
#' \item{Shen, Haipeng, and Jianhua Z. Huang. "Sparse principal component analysis via regularized low rank matrix approximation." Journal of multivariate analysis 99.6 (2008): 1015-1034.}
#' \item{Witten, Tibshirani and Hastie (2009) A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. _Biostatistics_ 10(3): 515-534.}
#' }
#' @examples
#'
#' set.seed(1)
#' u <- matrix(rnorm(200), ncol=1)
#' v <- matrix(c(runif(50, min=0.1), rep(0,250)), ncol=1)
#' u <- u / drop(sqrt(crossprod(u)))
#' v <- v / drop(sqrt(crossprod(v)))
#' x <- u %*% t(v) + 0.001 * matrix(rnorm(200*300), ncol=300)
#' s <- ssvd(x, n=50)
#' table(actual=v[, 1] != 0, estimated=s$v[, 1] != 0)
#' oldpar <- par(mfrow=c(2, 1))
#' plot(u, cex=2, main="u (black circles), Estimated u (blue discs)")
#' points(s$u, pch=19, col=4)
#' plot(v, cex=2, main="v (black circles), Estimated v (blue discs)")
#' points(s$v, pch=19, col=4)
#'
#' # Let's consider a trivial rank-2 example (k=2) with noise. Like the
#' # last example, we know the exact number of nonzero elements in each
#' # solution vector of the noise-free matrix. Note the application of
#' # different sparsity constraints on each column of the estimated v.
#' # Also, the decomposition is unique only up to sign, which we adjust
#' # for below.
#' set.seed(1)
#' u <- qr.Q(qr(matrix(rnorm(400), ncol=2)))
#' v <- matrix(0, ncol=2, nrow=300)
#' v[sample(300, 15), 1] <- runif(15, min=0.1)
#' v[sample(300, 50), 2] <- runif(50, min=0.1)
#' v <- qr.Q(qr(v))
#' x <- u %*% (c(2, 1) * t(v)) + .001 * matrix(rnorm(200 * 300), 200)
#' s <- ssvd(x, k=2, n=colSums(v != 0))
#'
#' # Compare actual and estimated vectors (adjusting for sign):
#' s$u <- sign(u) * abs(s$u)
#' s$v <- sign(v) * abs(s$v)
#' table(actual=v[, 1] != 0, estimated=s$v[, 1] != 0)
#' table(actual=v[, 2] != 0, estimated=s$v[, 2] != 0)
#' plot(v[, 1], cex=2, main="True v1 (black circles), Estimated v1 (blue discs)")
#' points(s$v[, 1], pch=19, col=4)
#' plot(v[, 2], cex=2, main="True v2 (black circles), Estimated v2 (blue discs)")
#' points(s$v[, 2], pch=19, col=4)
#' par(oldpar)
#'
#' @export
ssvd <- function(x, k=1, n=2, maxit=500, tol=1e-3, center=FALSE, scale.=FALSE, alpha=0, tsvd=NULL, ...)
{
if (alpha < 0 || alpha >= 1) stop("0 <= alpha < 1")
if (is.logical(center) && center) center <- colMeans(x)
if (is.logical(scale.))
{
if (scale.)
{
if (is.numeric(center))
{
f <- function(i) sqrt(sum((x[, i] - center[i]) ^ 2) / (nrow(x) - 1L))
scale. <- vapply(seq(ncol(x)), f, pi, USE.NAMES=FALSE)
} else scale. <- apply(x, 2L, function(v) sqrt(sum(v ^ 2) / max(1, length(v) - 1L)))
}
}
if (all(n > ncol(x) - 1))
{
warning("no sparsity constraints specified")
return(irlba(x, k, ...))
}
n <- ncol(x) - n
if (length(n) != k) n <- rep(n, length.out=k) # warn?
s <- tsvd
if (is.null(tsvd)) s <- irlba(x, k, scale=scale., center=center, ...)
lambda <- c()
soft <- function(x, u, p)
{
y <- crossprod(x, u)
if (is.numeric(center)) y <- y - sum(u) * center
if (is.numeric(scale.)) y <- y / scale.
# apply a column-wise penalty
a <- abs(y)
z <- apply(a, 2, sort)
lambda <<- vapply(seq(length(p)), function(j) (1 - alpha) * z[p[j], j] + alpha * z[p[j] + 1, j], pi, USE.NAMES=FALSE)
sign(y) * pmax(sweep(a, 2, lambda, `-`), 0)
}
s$v <- s$d * s$v
iter <- 0
delta_u <- Inf
while (delta_u > tol && iter < maxit)
{
u <- s$u
s$v <- soft(x, s$u, n)
if (is.numeric(scale.)) s$v <- s$v / scale.
if (is.numeric(center))
{
xsv <- x %*% s$v - drop(crossprod(center, s$v))
s$u <- qr.Q(qr(xsv))
} else
{
xsv <- x %*% s$v
s$u <- qr.Q(qr(xsv))
}
# Maintain sign (possibly messed up by QR)
s$u <- sweep(s$u, 2, apply(xsv, 2, function(x) sign(head(x[x!=0], 1))) / apply(s$u, 2, function(x) sign(head(x[x!=0], 1))), `*`)
delta_u <- max(1 - diag(abs(crossprod(u, s$u))))
iter <- iter + 1
}
if (iter >= maxit)
warning("Maximum number of iterations reached before convergence: solution may not be optimal. Consider increasing 'maxit'.")
s$v <- s$v %*% diag(1 / sqrt(apply(s$v, 2, crossprod)), ncol(s$v), ncol(s$v))
d <- s$v
if (is.numeric(scale.)) d <- d / scale.
d1 <- x %*% d
if (is.numeric(center)) d1 <- d1 - drop(crossprod(center, d))
d <- crossprod(s$u, d1)
list(u = s$u, v = s$v, d = d, iter = iter, lambda = lambda, center=center, scale=scale., n=n, alpha=alpha)
}
irlba/R/eigen.R 0000644 0001760 0000144 00000005220 13430560161 013004 0 ustar ripley users #' Find a few approximate largest eigenvalues and corresponding eigenvectors of a symmetric matrix.
#'
#' Use \code{partial_eigen} to estimate a subset of the largest (most positive)
#' eigenvalues and corresponding eigenvectors of a symmetric dense or sparse
#' real-valued matrix.
#'
#' @param x numeric real-valued dense or sparse matrix.
#' @param n number of largest eigenvalues and corresponding eigenvectors to compute.
#' @param symmetric \code{TRUE} indicates \code{x} is a symmetric matrix (the default);
#' specify \code{symmetric=FALSE} to compute the largest eigenvalues and corresponding
#' eigenvectors of \code{t(x) \%*\% x} instead.
#' @param ... optional additional parameters passed to the \code{irlba} function.
#'
#' @return
#' Returns a list with entries:
#' \itemize{
#' \item{values}{ n approximate largest eigenvalues}
#' \item{vectors}{ n approximate corresponding eigenvectors}
#' }
#'
#' @note
#' Specify \code{symmetric=FALSE} to compute the largest \code{n} eigenvalues
#' and corresponding eigenvectors of the symmetric matrix cross-product
#' \code{t(x) \%*\% x}.
#'
#' This function uses the \code{irlba} function under the hood. See \code{?irlba}
#' for description of additional options, especially the \code{tol} parameter.
#'
#' See the RSpectra package https://cran.r-project.org/package=RSpectra for more comprehensive
#' partial eigenvalue decomposition.
#'
#' @references
#' Augmented Implicitly Restarted Lanczos Bidiagonalization Methods, J. Baglama and L. Reichel, SIAM J. Sci. Comput. 2005.
#'
#' @examples
#' set.seed(1)
#' # Construct a symmetric matrix with some positive and negative eigenvalues:
#' V <- qr.Q(qr(matrix(runif(100), nrow=10)))
#' x <- V %*% diag(c(10, -9, 8, -7, 6, -5, 4, -3, 2, -1)) %*% t(V)
#' partial_eigen(x, 3)$values
#'
#' # Compare with eigen
#' eigen(x)$values[1:3]
#'
#' # Use symmetric=FALSE to compute the eigenvalues of t(x) %*% x for general
#' # matrices x:
#' x <- matrix(rnorm(100), 10)
#' partial_eigen(x, 3, symmetric=FALSE)$values
#' eigen(crossprod(x))$values
#'
#' @seealso \code{\link{eigen}}, \code{\link{irlba}}
#' @export
partial_eigen <- function(x, n=5, symmetric=TRUE, ...)
{
if (n > 0.5 * min(nrow(x), ncol(x)))
{
warning("You're computing a large percentage of total eigenvalues, the standard eigen function will likely work better!")
}
if (!symmetric)
{
L <- irlba(x, n, ...)
return(list(vectors=L$v, values=L$d ^ 2))
}
L <- irlba(x, n, ...)
s <- sign(L$u[1, ] * L$v[1, ])
if (all(s > 0))
{
return(list(vectors=L$u, values=L$d))
}
i <- min(which(s < 0))
shift <- L$d[i]
L <- irlba(x, n, shift=shift, ...)
return(list(vectors=L$u, values=L$d - shift))
}
irlba/R/prcomp.R 0000644 0001760 0000144 00000014551 13430560161 013224 0 ustar ripley users #' Principal Components Analysis
#'
#' Efficient computation of a truncated principal components analysis of a given data matrix
#' using an implicitly restarted Lanczos method from the \code{\link{irlba}} package.
#'
#' @param x a numeric or complex matrix (or data frame) which provides
#' the data for the principal components analysis.
#' @param retx a logical value indicating whether the rotated variables should be returned.
#' @param center a logical value indicating whether the variables should be
#' shifted to be zero centered. Alternately, a centering vector of length
#' equal the number of columns of \code{x} can be supplied.
#' @param scale. a logical value indicating whether the variables should be
#' scaled to have unit variance before the analysis takes place.
#' The default is \code{FALSE} for consistency with S, but scaling is often advisable.
#' Alternatively, a vector of length equal the number of columns of \code{x} can be supplied.
#'
#' The value of \code{scale} determines how column scaling is performed
#' (after centering). If \code{scale} is a numeric vector with length
#' equal to the number of columns of \code{x}, then each column of \code{x} is
#' divided by the corresponding value from \code{scale}. If \code{scale} is
#' \code{TRUE} then scaling is done by dividing the (centered) columns of
#' \code{x} by their standard deviations if \code{center=TRUE}, and the
#' root mean square otherwise. If \code{scale} is \code{FALSE}, no scaling is done.
#' See \code{\link{scale}} for more details.
#' @param n integer number of principal component vectors to return, must be less than
#' \code{min(dim(x))}.
#' @param ... additional arguments passed to \code{\link{irlba}}.
#'
#' @return
#' A list with class "prcomp" containing the following components:
#' \itemize{
#' \item{sdev} {the standard deviations of the principal components (i.e.,
#' the square roots of the eigenvalues of the
#' covariance/correlation matrix, though the calculation is
#' actually done with the singular values of the data matrix).}
#' \item{rotation} {the matrix of variable loadings (i.e., a matrix whose columns
#' contain the eigenvectors).}
#' \item {x} {if \code{retx} is \code{TRUE} the value of the rotated data (the centred
#' (and scaled if requested) data multiplied by the \code{rotation}
#' matrix) is returned. Hence, \code{cov(x)} is the diagonal matrix
#' \code{diag(sdev^2)}.}
#' \item{center, scale} {the centering and scaling used, or \code{FALSE}.}
#' }
#'
#' @note
#' The signs of the columns of the rotation matrix are arbitrary, and
#' so may differ between different programs for PCA, and even between
#' different builds of R.
#'
#' NOTE DIFFERENCES WITH THE DEFAULT \code{\link{prcomp}} FUNCTION!
#' The \code{tol} truncation argument found in \code{prcomp} is not supported.
#' In place of the truncation tolerance in the original function, the
#' \code{prcomp_irlba} function has the argument \code{n} explicitly giving the
#' number of principal components to return. A warning is generated if the
#' argument \code{tol} is used, which is interpreted differently between
#' the two functions.
#'
#' @examples
#' set.seed(1)
#' x <- matrix(rnorm(200), nrow=20)
#' p1 <- prcomp_irlba(x, n=3)
#' summary(p1)
#'
#' # Compare with
#' p2 <- prcomp(x, tol=0.7)
#' summary(p2)
#'
#'
#' @seealso \code{\link{prcomp}}
#' @import Matrix
#' @importFrom stats rnorm prcomp sd var
#' @importFrom methods slotNames slot
#' @export
prcomp_irlba <- function(x, n = 3, retx = TRUE, center = TRUE, scale. = FALSE, ...)
{
a <- names(as.list(match.call()))
ans <- list(scale=scale.)
if ("tol" %in% a)
warning("The `tol` truncation argument from `prcomp` is not supported by
`prcomp_irlba`. If specified, `tol` is passed to the `irlba` function to
control that algorithm's convergence tolerance. See `?prcomp_irlba` for help.")
# Try to convert data frame to matrix...
if (is.data.frame(x)) x <- as.matrix(x)
args <- list(A=x, nv=n)
if (is.logical(center))
{
if (center) args$center <- colMeans(x)
} else args$center <- center
if (is.logical(scale.))
{
if (is.numeric(args$center))
{
f <- function(i) sqrt(sum((x[, i] - args$center[i]) ^ 2) / (nrow(x) - 1L))
scale. <- vapply(seq(ncol(x)), f, pi, USE.NAMES=FALSE)
if (ans$scale) ans$totalvar <- ncol(x)
else ans$totalvar <- sum(scale. ^ 2)
} else
{
if (ans$scale)
{
scale. <- apply(x, 2L, function(v) sqrt(sum(v ^ 2) / max(1, length(v) - 1L)))
f <- function(i) sqrt(sum((x[, i] / scale.[i]) ^ 2) / (nrow(x) - 1L))
ans$totalvar <- sum(vapply(seq(ncol(x)), f, pi, USE.NAMES=FALSE) ^ 2)
} else
{
f <- function(i) sum(x[, i] ^ 2) / (nrow(x) - 1L)
ans$totalvar <- sum(vapply(seq(ncol(x)), f, pi, USE.NAMES=FALSE))
}
}
if (ans$scale) args$scale <- scale.
} else
{
args$scale <- scale.
f <- function(i) sqrt(sum((x[, i] / scale.[i]) ^ 2) / (nrow(x) - 1L))
ans$totalvar <- sum(vapply(seq(ncol(x)), f, pi, USE.NAMES=FALSE))
}
if (!missing(...)) args <- c(args, list(...))
s <- do.call(irlba, args=args)
ans$sdev <- s$d / sqrt(max(1, nrow(x) - 1))
ans$rotation <- s$v
colnames(ans$rotation) <- paste("PC", seq(1, ncol(ans$rotation)), sep="")
ans$center <- args$center
if (retx)
{
ans <- c(ans, list(x = sweep(s$u, 2, s$d, FUN=`*`)))
colnames(ans$x) <- paste("PC", seq(1, ncol(ans$rotation)), sep="")
}
class(ans) <- c("irlba_prcomp", "prcomp")
ans
}
#' Summary method for truncated pca objects computed by \code{prcomp_irlba}.
#' @param object An object returned by \code{prcomp_irlba}.
#' @param ... Optional arguments passed to \code{summary}.
#' @method summary irlba_prcomp
#' @export
summary.irlba_prcomp <- function(object, ...)
{
chkDots(...)
vars <- object$sdev ^ 2
vars <- vars / object$totalvar
importance <- rbind("Standard deviation" = object$sdev,
"Proportion of Variance" = round(vars, 5),
"Cumulative Proportion" = round(cumsum(vars), 5))
k <- ncol(object$rotation)
colnames(importance) <- c(colnames(object$rotation), rep("", length(vars) - k))
object$importance <- importance
class(object) <- "summary.prcomp"
object
}
irlba/R/irlba.R 0000644 0001760 0000144 00000072323 13430560161 013016 0 ustar ripley users #' Find a few approximate singular values and corresponding
#' singular vectors of a matrix.
#'
#' The augmented implicitly restarted Lanczos bidiagonalization algorithm
#' (IRLBA) finds a few approximate largest (or, optionally, smallest) singular
#' values and corresponding
#' singular vectors of a sparse or dense matrix using a method of Baglama and
#' Reichel. It is a fast and memory-efficient way to compute a partial SVD.
#'
#' @param A numeric real- or complex-valued matrix or real-valued sparse matrix.
#' @param nv number of right singular vectors to estimate.
#' @param nu number of left singular vectors to estimate (defaults to \code{nv}).
#' @param maxit maximum number of iterations.
#' @param work working subspace dimension, larger values can speed convergence at the cost of more memory use.
#' @param reorth if \code{TRUE}, apply full reorthogonalization to both SVD bases, otherwise
#' only apply reorthogonalization to the right SVD basis vectors; the latter case is cheaper per
#' iteration but, overall, may require more iterations for convergence. Automatically \code{TRUE}
#' when \code{fastpath=TRUE} (see below).
#' @param tol convergence is determined when \eqn{\|A^TU - VS\| < tol\|A\|}{||A^T U - VS|| < tol*||A||},
#' and when the maximum relative change in estimated singular values from one iteration to the
#' next is less than \code{svtol = tol} (see \code{svtol} below),
#' where the spectral norm ||A|| is approximated by the
#' largest estimated singular value, and U, V, S are the matrices corresponding
#' to the estimated left and right singular vectors, and diagonal matrix of
#' estimated singular values, respectively.
#' @param v optional starting vector or output from a previous run of \code{irlba} used
#' to restart the algorithm from where it left off (see the notes).
#' @param right_only logical value indicating return only the right singular vectors
#' (\code{TRUE}) or both sets of vectors (\code{FALSE}). The right_only option can be
#' cheaper to compute and use much less memory when \code{nrow(A) >> ncol(A)} but note
#' that obtained solutions typically lose accuracy due to lack of re-orthogonalization in the
#' algorithm and that \code{right_only = TRUE} sets \code{fastpath = FALSE} (only use this option
#' for really large problems that run out of memory and when \code{nrow(A) >> ncol(A)}).
#' Consider increasing the \code{work} option to improve accuracy with \code{right_only=TRUE}.
#' @param verbose logical value that when \code{TRUE} prints status messages during the computation.
#' @param scale optional column scaling vector whose values divide each column of \code{A};
#' must be as long as the number of columns of \code{A} (see notes).
#' @param center optional column centering vector whose values are subtracted from each
#' column of \code{A}; must be as long as the number of columns of \code{A} and may
#' not be used together with the deflation options below (see notes).
#' @param shift optional shift value (square matrices only, see notes).
#' @param mult DEPRECATED optional custom matrix multiplication function (default is \code{\%*\%}, see notes).
#' @param fastpath try a fast C algorithm implementation if possible; set \code{fastpath=FALSE} to use the
#' reference R implementation. See the notes for more details.
#' @param svtol additional stopping tolerance on maximum allowed absolute relative change across each
#' estimated singular value between iterations.
#' The default value of this parameter is to set it to \code{tol}. You can set \code{svtol=Inf} to
#' effectively disable this stopping criterion. Setting \code{svtol=Inf} allows the method to
#' terminate on the first Lanczos iteration if it finds an invariant subspace, but with less certainty
#' that the converged subspace is the desired one. (It may, for instance, miss some of the largest
#' singular values in difficult problems.)
#' @param smallest set \code{smallest=TRUE} to estimate the smallest singular values and associated
#' singular vectors. WARNING: this option is somewhat experimental, and may produce poor
#' estimates for ill-conditioned matrices.
#' @param ... optional additional arguments used to support experimental and deprecated features.
#'
#' @return
#' Returns a list with entries:
#' \describe{
#' \item{d:}{ max(nu, nv) approximate singular values}
#' \item{u:}{ nu approximate left singular vectors (only when right_only=FALSE)}
#' \item{v:}{ nv approximate right singular vectors}
#' \item{iter:}{ The number of Lanczos iterations carried out}
#' \item{mprod:}{ The total number of matrix vector products carried out}
#' }
#'
#' @note
#' The syntax of \code{irlba} partially follows \code{svd}, with an important
#' exception. The usual R \code{svd} function always returns a complete set of
#' singular values, even if the number of singular vectors \code{nu} or \code{nv}
#' is set less than the maximum. The \code{irlba} function returns a number of
#' estimated singular values equal to the maximum of the number of specified
#' singular vectors \code{nu} and \code{nv}.
#'
#' Use the optional \code{scale} parameter to implicitly scale each column of
#' the matrix \code{A} by the values in the \code{scale} vector, computing the
#' truncated SVD of the column-scaled \code{sweep(A, 2, scale, FUN=`/`)}, or
#' equivalently, \code{A \%*\% diag(1 / scale)}, without explicitly forming the
#' scaled matrix. \code{scale} must be a non-zero vector of length equal
#' to the number of columns of \code{A}.
#'
#' Use the optional \code{center} parameter to implicitly subtract the values
#' in the \code{center} vector from each column of \code{A}, computing the
#' truncated SVD of \code{sweep(A, 2, center, FUN=`-`)},
#' without explicitly forming the centered matrix. \code{center}
#' must be a vector of length equal to the number of columns of \code{A}.
#' This option may be used to efficiently compute principal components without
#' explicitly forming the centered matrix (which can, importantly, preserve
#' sparsity in the matrix). See the examples.
#'
#' The optional \code{shift} scalar valued argument applies only to square matrices; use it
#' to estimate the partial svd of \code{A + diag(shift, nrow(A), nrow(A))}
#' (without explicitly forming the shifted matrix).
#'
#' (Deprecated) Specify an optional alternative matrix multiplication operator in the
#' \code{mult} parameter. \code{mult} must be a function of two arguments,
#' and must handle both cases where one argument is a vector and the other
#' a matrix. This option is deprecated and will be removed in a future version.
#' The new preferred method simply uses R itself to define a custom matrix class
#' with your user-defined matrix multiplication operator. See the examples.
#'
#' Use the \code{v} option to supply a starting vector for the iterative
#' method. A random vector is used by default (precede with \code{set.seed()}
#' for reproducibility). Optionally set \code{v} to
#' the output of a previous run of \code{irlba} to restart the method, adding
#' additional singular values/vectors without recomputing the solution
#' subspace. See the examples.
#'
#' The function may generate the following warnings:
#' \itemize{
#' \item{"did not converge--results might be invalid!; try increasing work or maxit"
#' means that the algorithm didn't
#' converge -- this is potentially a serious problem and the returned results may not be valid. \code{irlba}
#' reports a warning here instead of an error so that you can inspect whatever is returned. If this
#' happens, carefully heed the warning and inspect the result. You may also try setting \code{fastpath=FALSE}.}
#' \item{"You're computing a large percentage of total singular values, standard svd might work better!"
#' \code{irlba} is designed to efficiently compute a few of the largest singular values and associated
#' singular vectors of a matrix. The standard \code{svd} function will be more efficient for computing
#' large numbers of singular values than \code{irlba}.}
#' \item{"convergence criterion below machine epsilon" means that the product of \code{tol} and the
#' largest estimated singular value is really small and the normal convergence criterion is only
#' met up to round off error.}
#' }
#' The function might return an error for several reasons including a situation when the starting
#' vector \code{v} is near the null space of the matrix. In that case, try a different \code{v}.
#'
#' The \code{fastpath=TRUE} option only supports real-valued matrices and sparse matrices
#' of type \code{dgCMatrix} (for now). Other problems fall back to the reference
#' R implementation.
#'
#' @references
#' Baglama, James, and Lothar Reichel. "Augmented implicitly restarted Lanczos bidiagonalization methods." SIAM Journal on Scientific Computing 27.1 (2005): 19-42.
#'
#' @examples
#' set.seed(1)
#'
#' A <- matrix(runif(400), nrow=20)
#' S <- irlba(A, 3)
#' S$d
#'
#' # Compare with svd
#' svd(A)$d[1:3]
#'
#' # Restart the algorithm to compute more singular values
#' # (starting with an existing solution S)
#' S1 <- irlba(A, 5, v=S)
#'
#' # Estimate smallest singular values
#' irlba(A, 3, smallest=TRUE)$d
#'
#' #Compare with
#' tail(svd(A)$d, 3)
#'
#' # Principal components (see also prcomp_irlba)
#' P <- irlba(A, nv=1, center=colMeans(A))
#'
#' # Compare with prcomp and prcomp_irlba (might vary up to sign)
#' cbind(P$v,
#' prcomp(A)$rotation[, 1],
#' prcomp_irlba(A)$rotation[, 1])
#'
#' # A custom matrix multiplication function that scales the columns of A
#' # (cf the scale option). This function scales the columns of A to unit norm.
#' col_scale <- sqrt(apply(A, 2, crossprod))
#' setClass("scaled_matrix", contains="matrix", slots=c(scale="numeric"))
#' setMethod("%*%", signature(x="scaled_matrix", y="numeric"),
#' function(x ,y) x@.Data %*% (y / x@scale))
#' setMethod("%*%", signature(x="numeric", y="scaled_matrix"),
#' function(x ,y) (x %*% y@.Data) / y@scale)
#' a <- new("scaled_matrix", A, scale=col_scale)
#' irlba(a, 3)$d
#'
#' # Compare with:
#' svd(sweep(A, 2, col_scale, FUN=`/`))$d[1:3]
#'
#'
#' @seealso \code{\link{svd}}, \code{\link{prcomp}}, \code{\link{partial_eigen}}, \code{\link{svdr}}
#' @import Matrix
#' @importFrom stats rnorm
#' @importFrom methods slotNames
#' @useDynLib irlba, .registration=TRUE, .fixes="C_"
#' @export
irlba <-
function(A, # data matrix
nv=5, nu=nv, # number of singular vectors to estimate
maxit=1000, # maximum number of iterations
work=nv + 7, # working subspace size
reorth=TRUE, # TRUE=full reorthogonalization
tol=1e-5, # stopping tolerance
v=NULL, # optional starting vector or restart
right_only=FALSE, # TRUE=only return V
verbose=FALSE, # display status messages
scale=NULL, # optional column scaling
center=NULL, # optional column centering
shift=NULL, # optional shift for square matrices
mult=NULL, # optional custom matrix multiplication func.
fastpath=TRUE, # use the faster C implementation if possible
svtol=tol, # stopping tolerance percent change in estimated svs
smallest=FALSE, # set to TRUE to estimate subspaces associated w/smallest singular values
...) # optional experimental or deprecated arguments
{
# ---------------------------------------------------------------------
# Check input parameters
# ---------------------------------------------------------------------
ropts <- options(warn=1) # immediately show warnings
mflag <- new.env()
mflag$flag <- FALSE
on.exit(options(ropts))
interchange <- FALSE
eps <- .Machine$double.eps
# hidden support for old, removed (previously deprecated) parameters
# this is here as a convenience to keep old code working without change
# also supports experimental features not yet promoted to the api
mcall <- as.list(match.call())
random <- eval(mcall[["rng"]])
if (is.null(random)) random <- rnorm # default RNG
# Maximum number of Ritz vectors to use in augmentation, may be less
# depending on workspace size.
maxritz <- eval(mcall[["maxritz"]]) # experimental
if (is.null(maxritz)) maxritz <- 3
eps2 <- eval(mcall[["invariant_subspace_tolerance"]])
if (is.null(eps2)) eps2 <- eps ^ (4 / 5)
du <- eval(mcall[["du"]]) # deprecated
dv <- eval(mcall[["dv"]]) # deprecated
ds <- eval(mcall[["ds"]]) # deprecated
deflate <- is.null(du) + is.null(ds) + is.null(dv)
if (is.logical(scale) && ! scale) scale <- NULL
if (is.logical(shift) && ! shift) shift <- NULL
if (is.logical(center) && ! center) center <- NULL
if (smallest) fastpath <- FALSE # for now anyway
if (any(dim(A) > 2 ^ 32 - 1)) fastpath <- FALSE # for now
if (deflate == 3)
{
deflate <- FALSE
} else if (deflate == 0)
{
deflate <- TRUE
warning("The deflation options have been deprecated. Please modify your code to not use them.")
if (length(ds) > 1) stop("deflation limited to one dimension")
if (!is.null(dim(du))) du <- du[, 1]
if (!is.null(dim(dv))) dv <- dv[, 1]
} else stop("all three du ds dv parameters must be specified for deflation")
if (!is.null(center))
{
if (is.logical(center) && center) center <- colMeans(A)
if (deflate) stop("the center parameter can't be specified together with deflation parameters")
if (length(center) != ncol(A)) stop("center must be a vector of length ncol(A)")
if (fastpath && ! right_only) du <- NULL
else du <- 1
ds <- 1
dv <- center
deflate <- TRUE
}
if ("integer" == typeof(A)) A <- A + 0.0
iscomplex <- is.complex(A)
m <- nrow(A)
n <- ncol(A)
if (is.null(nu)) nu <- nv
if (!is.null(mult) && deflate) stop("the mult parameter can't be specified together with deflation parameters")
missingmult <- FALSE
if (is.null(mult))
{
missingmult <- TRUE
mult <- `%*%`
}
k <- max(nu, nv)
if (k <= 0) stop("max(nu, nv) must be positive")
if (k > min(m - 1, n - 1)) stop("max(nu, nv) must be strictly less than min(nrow(A), ncol(A))")
if (k >= 0.5 * min(m, n))
{
warning("You're computing too large a percentage of total singular values, use a standard svd instead.")
}
if (work <= 1) stop("work must be greater than 1")
if (tol < 0) stop("tol must be non-negative")
if (maxit <= 0) stop("maxit must be positive")
# work must be strictly larger than requested subspace dimension, except see right_only below
if (work <= k && ! right_only) work <- k + 1
if (work >= min(n, m))
{
work <- min(n, m)
if (work <= k)
{
k <- work - 1 # the best we can do! Need to reduce output subspace dimension
warning("Requested subspace dimension too large! Reduced to ", k)
}
}
k_org <- k
w_dim <- work
if (right_only)
{
w_dim <- 1
fastpath <- FALSE
}
if (n > m && smallest)
{
# Interchange dimensions m,n so that dim(A'A) = min(m,n) when seeking the
# smallest singular values; avoids finding zero-valued smallest singular values.
interchange <- TRUE
temp <- m
m <- n
n <- temp
}
if (verbose)
{
message("Working dimension size ", work)
}
# Check for tiny problem, use standard SVD in that case. Make definition of 'tiny' larger?
if (min(m, n) < 6)
{
A <- as.matrix(A) # avoid need to define "+" and "/" for arbitrary matrix types.
if (verbose) message("Tiny problem detected, using standard `svd` function.")
if (!is.null(scale)) {
A <- sweep(A, 2, scale, "/")
dv <- dv / scale # scale the centering vector.
}
if (!is.null(shift)) A <- A + diag(shift, nrow(A), ncol(A))
if (deflate)
{
if (is.null(du)) du <- rep(1, nrow(A))
A <- A - (ds * du) %*% t(dv)
}
s <- svd(A)
if (smallest)
{
return(list(d=tail(s$d, k), u=s$u[, tail(seq(ncol(s$u)), k), drop=FALSE],
v=s$v[, tail(seq(ncol(s$v), k)), drop=FALSE], iter=0, mprod=0))
}
return(list(d=s$d[1:k], u=s$u[, 1:nu, drop=FALSE],
v=s$v[, 1:nv, drop=FALSE], iter=0, mprod=0))
}
# Try to use the fast C-language code path
if (deflate) fastpath <- fastpath && is.null(du)
# Only matrix, dgCMatrix supported by fastpath
fastpath <- fastpath && (("Matrix" %in% attributes(class(A)) && ("dgCMatrix" %in% class(A))) || "matrix" %in% class(A))
if (fastpath && missingmult && !iscomplex && !right_only)
{
RESTART <- 0L
RV <- RW <- RS <- NULL
if (is.null(v))
{
v <- random(n)
if (verbose) message("Initializing starting vector v with samples from standard normal distribution.
Use `set.seed` first for reproducibility.")
} else if (is.list(v)) # restarted case
{
if (is.null(v$v) || is.null(v$d) || is.null(v$u)) stop("restart requires left and right singular vectors")
if (max(nu, nv) <= min(ncol(v$u), ncol(v$v))) return(v) # Nothing to do!
RESTART <- as.integer(length(v$d))
RND <- random(n)
RND <- orthog(RND, v$v)
RV <- cbind(v$v, RND / norm2(RND))
RW <- v$u
RS <- v$d
v <- NULL
}
SP <- ifelse(is.matrix(A), 0L, 1L)
if (verbose) message("irlba: using fast C implementation")
SCALE <- NULL
SHIFT <- NULL
CENTER <- NULL
if (!is.null(scale))
{
if (length(scale) != ncol(A)) stop("scale length must match number of matrix columns")
SCALE <- as.double(scale)
}
if (!is.null(shift))
{
if (length(shift) != 1) stop("shift length must be 1")
SHIFT <- as.double(shift)
}
if (deflate)
{
if (length(center) != ncol(A)) stop("the centering vector length must match the number of matrix columns")
CENTER <- as.double(center)
}
ans <- .Call(C_IRLB, A, as.integer(k), as.double(v), as.integer(work),
as.integer(maxit), as.double(tol), as.double(eps2), as.integer(SP),
as.integer(RESTART), RV, RW, RS, SCALE, SHIFT, CENTER, as.double(svtol))
if (ans[[6]] == 0 || ans[[6]] == -2)
{
names(ans) <- c("d", "u", "v", "iter", "mprod", "err")
ans$u <- matrix(head(ans$u, m * nu), nrow=m, ncol=nu)
ans$v <- matrix(head(ans$v, n * nv), nrow=n, ncol=nv)
if (tol * ans$d[1] < eps) warning("convergence criterion below machine epsilon")
if (ans[[6]] == -2) warning("did not converge--results might be invalid!; try increasing work or maxit")
return(ans[-6])
}
errors <- c("invalid dimensions",
"didn't converge",
"out of memory",
"starting vector near the null space",
"linear dependency encountered")
erridx <- abs(ans[[6]])
if (erridx > 1)
warning("fast code path error ", errors[erridx], "; re-trying with fastpath=FALSE.", immediate.=TRUE)
}
# Allocate memory for W and F:
W <- matrix(0.0, m, w_dim)
F <- matrix(0.0, n, 1)
restart <- FALSE
if (is.list(v))
{
if (is.null(v$v) || is.null(v$d) || is.null(v$u)) stop("restart requires left and right singular vectors")
if (max(nu, nv) <= min(ncol(v$u), ncol(v$v))) return(v) # Nothing to do!
right_only <- FALSE
W[, 1:ncol(v$u)] <- v$u
d <- v$d
V <- matrix(0.0, n, work)
V[, 1:ncol(v$v)] <- v$v
restart <- TRUE
} else if (is.null(v))
{
# If starting matrix v is not given then set V to be an (n x 1) matrix of
# normally distributed random numbers. In any case, allocate V appropriate to
# problem size:
V <- matrix(0.0, n, work)
V[, 1] <- random(n)
} else
{
# user-supplied starting subspace
V <- matrix(0.0, n, work)
V[1:length(v)] <- v
}
# ---------------------------------------------------------------------
# Initialize local variables
# ---------------------------------------------------------------------
B <- NULL # Bidiagonal matrix
Bsz <- NULL # Size of B
eps23 <- eps ^ (2 / 3) # Used for Smax/avoids using zero
iter <- 1 # Man loop iteration count
mprod <- 0 # Number of matrix-vector products
R_F <- NULL # 2-norm of residual vector F
sqrteps <- sqrt(eps) #
Smax <- 1 # Max value of all computed singular values of
# B est. ||A||_2
Smin <- NULL # Min value of all computed singular values of
# B est. cond(A)
lastsv <- c() # estimated sv in last iteration
# Check for user-supplied restart condition
if (restart)
{
B <- cbind(diag(d), 0)
k <- length(d)
F <- random(n)
F <- orthog(F, V[, 1:k])
V[, k + 1] <- F / norm2(F)
}
# Change du to be non-NULL, for non-fastpath'able matrices with non-NULL scale.
if (deflate && is.null(du)) du <- 1
# ---------------------------------------------------------------------
# Main iteration
# ---------------------------------------------------------------------
while (iter <= maxit)
{
# ---------------------------------------------------------------------
# Compute the Lanczos bidiagonal decomposition:
# such that AV = WB
# and t(A)W = VB + Ft(E)
# This routine updates W, V, F, B, mprod
#
# Note on scale and center: These options are applied implicitly below
# for maximum computational efficiency. This complicates their application
# somewhat, but saves a few flops.
# ---------------------------------------------------------------------
j <- 1
# Normalize starting vector:
if (iter == 1 && !restart)
{
V[, 1] <- V[, 1] / norm2(V[, 1])
}
else j <- k + 1
# j_w is used here to support the right_only=TRUE case.
j_w <- ifelse(w_dim > 1, j, 1)
# Compute W=AV
# Optionally apply scale
VJ <- V[, j]
if (!is.null(scale))
{
VJ <- VJ / scale
}
if (interchange) avj <- mult(VJ, A)
else avj <- mult(A, VJ)
# Handle non-ordinary arrays as products.
W[, j_w] <- as.vector(avj)
mprod <- mprod + 1
# Optionally apply shift
if (!is.null(shift))
{
W[, j_w] <- W[, j_w] + shift * VJ
}
# Optionally apply deflation
if (deflate)
{
W[, j_w] <- W[, j_w] - ds * drop(cross(dv, VJ)) * du
}
# Orthogonalize W
if (iter != 1 && w_dim > 1 && reorth)
{
W[, j] <- orthog(W[, j, drop=FALSE], W[, 1:(j - 1), drop=FALSE])
}
S <- norm2(W[, j_w, drop=FALSE])
# Check for linearly dependent vectors
if (is.na(S) || S < eps2 && j == 1) stop("starting vector near the null space")
if (is.na(S) || S < eps2)
{
if (verbose) message_once("invariant subspace found", flag=mflag)
W[, j_w] <- random(nrow(W))
if (w_dim > 1) W[, j] <- orthog(W[, j], W[, 1:(j - 1)])
W[, j_w] <- W[, j_w] / norm2(W[, j_w])
S <- 0
}
else W[, j_w] <- W[, j_w] / S
# Lanczos process
while (j <= work)
{
j_w <- ifelse(w_dim > 1, j, 1)
if (iscomplex)
{
if (interchange) F <- Conj(t(drop(mult(A, Conj(drop(W[, j_w]))))))
else F <- Conj(t(drop(mult(Conj(drop(W[, j_w])), A))))
}
else
{
if (interchange) F <- t(drop(mult(A, drop(W[, j_w]))))
else F <- t(drop(mult(drop(W[, j_w]), A)))
}
# Optionally apply shift, scale, deflate
if (!is.null(shift)) F <- F + shift * W[, j_w]
if (!is.null(scale)) F <- F / scale
if (deflate) {
sub <- sum(W[, j_w]) * dv
if (!is.null(scale)) sub <- sub / scale
F <- F - sub
}
mprod <- mprod + 1
F <- drop(F - S * V[, j])
# Orthogonalize
F <- orthog(F, V[, 1:j, drop=FALSE])
if (j + 1 <= work)
{
R <- norm2(F)
# Check for linear dependence
if (R < eps2)
{
if (verbose) message_once("invariant subspace found", flag=mflag)
F <- matrix(random(dim(V)[1]), dim(V)[1], 1)
F <- orthog(F, V[, 1:j, drop=FALSE])
V[, j + 1] <- F / norm2(F)
R <- 0
}
else V[, j + 1] <- F / R
# Compute block diagonal matrix
if (is.null(B)) B <- cbind(S, R)
else B <- rbind(cbind(B, 0), c(rep(0, ncol(B) - 1), S, R))
jp1_w <- ifelse(w_dim > 1, j + 1, 1)
w_old <- W[, j_w]
# Optionally apply scale
VJP1 <- V[, j + 1]
if (!is.null(scale))
{
VJP1 <- VJP1 / scale
}
if (interchange) W[, jp1_w] <- drop(mult(drop(VJP1), A))
else W[, jp1_w] <- drop(mult(A, drop(VJP1)))
mprod <- mprod + 1
# Optionally apply shift
if (!is.null(shift))
{
W[, jp1_w] <- W[, jp1_w] + shift * VJP1
}
# Optionally apply deflation
if (deflate)
{
W[, jp1_w] <- W[, jp1_w] - ds * drop(cross(dv, VJP1)) * du
}
# One step of the classical Gram-Schmidt process
W[, jp1_w] <- W[, jp1_w] - R * w_old
# Full reorthogonalization of W
if (reorth && w_dim > 1) W[, j + 1] <- orthog(W[, j + 1], W[, 1:j])
S <- norm2(W[, jp1_w])
# Check for linear dependence
if (S < eps2)
{
if (verbose) message_once("invariant subspace found", flag=mflag)
W[, jp1_w] <- random(nrow(W))
if (w_dim > 1) W[, j + 1] <- orthog(W[, j + 1], W[, 1:j])
W[, jp1_w] <- W[, jp1_w] / norm2(W[, jp1_w])
S <- 0
}
else W[, jp1_w] <- W[, jp1_w] / S
}
else
{
# Add a last block to matrix B
B <- rbind(B, c(rep(0, j - 1), S))
}
j <- j + 1
}
# ---------------------------------------------------------------------
# (End of the Lanczos bidiagonalization part)
# ---------------------------------------------------------------------
Bsz <- nrow(B)
R_F <- norm2(F)
F <- F / R_F
# Compute singular triplets of B, svd must return ordered singular
# values from largest to smallest.
Bsvd <- svd(B)
# Estimate ||A|| using the largest singular value over all iterations
# and estimate the cond(A) using approximations to the largest and
# smallest singular values. If a small singular value is less than sqrteps
# require two-sided reorthogonalization.
if (iter == 1)
{
Smax <- Bsvd$d[1]
Smin <- Bsvd$d[Bsz]
}
else
{
Smax <- max(Smax, Bsvd$d[1])
Smin <- min(Smin, Bsvd$d[Bsz])
}
Smax <- max(eps23, Smax)
if (! reorth && Smin / Smax < sqrteps)
{
warning("The matrix is ill-conditioned. Basis will be reorthogonalized.")
reorth <- TRUE
}
if (smallest)
{
jj <- seq(ncol(Bsvd$u), 1, by = -1)
Bsvd$u <- Bsvd$u[, jj]
Bsvd$d <- Bsvd$d[jj]
Bsvd$v <- Bsvd$v[, jj]
}
# Compute the residuals
R <- R_F * Bsvd$u[Bsz, , drop=FALSE]
# Check for convergence
ct <- convtests(Bsz, tol, k_org, Bsvd, abs(R), k, Smax, lastsv, svtol, maxritz, work, S)
if (verbose)
{
message("iter= ", iter,
", mprod= ", mprod,
", sv[", k_org, "]=", sprintf("%.2e", Bsvd$d[k_org]),
", %change=", sprintf("%.2e", (Bsvd$d[k_org] - lastsv[k_org])/Bsvd$d[k_org]),
", k=", ct$k)
}
lastsv <- Bsvd$d
k <- ct$k
# If all desired singular values converged, then exit main loop
if (ct$converged) break
if (iter >= maxit) break
# Compute the starting vectors and first block of B
if (smallest && (Smin / Smax > sqrteps))
{
# Update the SVD of B to be the svd of [B ||F||E_m]
Bsvd2.d <- Bsvd$d
Bsvd2.d <- diag(Bsvd2.d, nrow=length(Bsvd2.d))
Bsvd2 <- svd(cbind(Bsvd2.d, t(R)))
jj <- seq(ncol(Bsvd2$u), 1, by=-1)
Bsvd2$u <- Bsvd2$u[, jj]
Bsvd2$d <- Bsvd2$d[jj]
Bsvd2$v <- Bsvd2$v[, jj]
Bsvd$d <- Bsvd2$d
Bsvd$u <- Bsvd$u %*% Bsvd2$u
Bsvd$v <- cbind(rbind(Bsvd$v, rep(0, Bsz)), c(rep(0, Bsz), 1)) %*% Bsvd2$v
V_B_last <- Bsvd$v[Bsz + 1, 1:k]
s <- R_F * solve(B, cbind(c(rep(0, Bsz - 1), 1)))
Bsvd$v <- Bsvd$v[1:Bsz, , drop=FALSE] + s %*% Bsvd$v[Bsz + 1, ]
qrv <- qr(cbind(rbind(Bsvd$v[, 1:k], 0), rbind(-s, 1)))
Bsvd$v <- qr.Q(qrv)
R <- qr.R(qrv)
V[, 1:(k + 1)] <- cbind(V, F) %*% Bsvd$v
# Update and compute the k by k+1 part of B
UT <- t(R[1:(k + 1), 1:k] + R[, k + 1] %*% rbind(V_B_last))
B <- diag(Bsvd$d[1:k], nrow=k) %*% (UT * upper.tri(UT, diag=TRUE))[1:k, 1:(k+1)]
} else
{
# using the Ritz vectors
V[, 1:(k + dim(F)[2])] <- cbind(V[, 1:(dim(Bsvd$v)[1]), drop=FALSE] %*% Bsvd$v[, 1:k], F)
B <- cbind(diag(Bsvd$d[1:k], nrow=k), R[1:k])
}
# Update the left approximate singular vectors
if (w_dim > 1)
{
W[, 1:k] <- W[, 1:(dim(Bsvd$u)[1]), drop=FALSE] %*% Bsvd$u[, 1:k]
}
iter <- iter + 1
}
# ---------------------------------------------------------------------
# End of the main iteration loop
# Output results
# ---------------------------------------------------------------------
if (!ct$converged) warning("did not converge--results might be invalid!; try increasing maxit or work")
d <- Bsvd$d[1:k_org]
if (!right_only)
{
u <- W[, 1:(dim(Bsvd$u)[1]), drop=FALSE] %*% Bsvd$u[, 1:k_org, drop=FALSE]
}
v <- V[, 1:(dim(Bsvd$v)[1]), drop=FALSE] %*% Bsvd$v[, 1:k_org, drop=FALSE]
if (smallest)
{
reverse <- seq(length(d), 1)
d <- d[reverse]
if (!right_only) u <- u[, reverse]
v <- v[, reverse]
}
if (tol * d[1] < eps) warning("convergence criterion below machine epsilon")
if (right_only)
return(list(d=d, v=v[, 1:nv, drop=FALSE], iter=iter, mprod=mprod))
return(list(d=d, u=u[, 1:nu, drop=FALSE],
v=v[, 1:nv, drop=FALSE], iter=iter, mprod=mprod))
}
irlba/R/svdr.R 0000644 0001760 0000144 00000012177 13430560161 012704 0 ustar ripley users #' Find a few approximate largest singular values and corresponding
#' singular vectors of a matrix.
#'
#' The randomized method for truncated SVD by P. G. Martinsson and colleagues
#' finds a few approximate largest singular values and corresponding
#' singular vectors of a sparse or dense matrix. It is a fast and
#' memory-efficient way to compute a partial SVD, similar in performance
#' for many problems to \code{\link{irlba}}. The \code{svdr} method
#' is a block method and may produce more accurate estimations with
#' less work for problems with clustered large singular values (see
#' the examples). In other problems, \code{irlba} may exhibit faster
#' convergence.
#'
#' Also see an alternate implementation (\code{rsvd}) of this method by N. Benjamin Erichson
#' in the https://cran.r-project.org/package=rsvd package.
#'
#' @param x numeric real- or complex-valued matrix or real-valued sparse matrix.
#' @param k dimension of subspace to estimate (number of approximate singular values to compute).
#' @param tol stop iteration when the largest absolute relative change in estimated singular
#' values from one iteration to the next falls below this value.
#' @param it maximum number of algorithm iterations.
#' @param extra number of extra vectors of dimension \code{ncol(x)}, larger values generally improve accuracy (with increased
#' computational cost).
#' @param center optional column centering vector whose values are implicitly subtracted from each
#' column of \code{A} without explicitly forming the centered matrix (preserving sparsity).
#' Optionally specify \code{center=TRUE} as shorthand for \code{center=colMeans(x)}.
#' Use for efficient principal components computation.
#' @param Q optional initial random matrix, defaults to a matrix of size \code{ncol(x)} by \code{k + extra} with
#' entries sampled from a normal random distribution.
#' @param return.Q if \code{TRUE} return the \code{Q} matrix for restarting (see examples).
#' @return
#' Returns a list with entries:
#' \describe{
#' \item{d:}{ k approximate singular values}
#' \item{u:}{ k approximate left singular vectors}
#' \item{v:}{ k approximate right singular vectors}
#' \item{mprod:}{ total number of matrix products carried out}
#' \item{Q:}{ optional subspace matrix (when \code{return.Q=TRUE})}
#' }
#' @seealso \code{\link{irlba}}, \code{\link{svd}}, \code{rsvd} in the rsvd package
#' @references
#' Finding structure with randomness: Stochastic algorithms for constructing
#' approximate matrix decompositions N. Halko, P. G. Martinsson, J. Tropp. Sep. 2009.
#' @examples
#' set.seed(1)
#'
#' A <- matrix(runif(400), nrow=20)
#' svdr(A, 3)$d
#'
#' # Compare with svd
#' svd(A)$d[1:3]
#'
#' # Compare with irlba
#' irlba(A, 3)$d
#'
#' \dontrun{
#' # A problem with clustered large singular values where svdr out-performs irlba.
#' tprolate <- function(n, w=0.25)
#' {
#' a <- rep(0, n)
#' a[1] <- 2 * w
#' a[2:n] <- sin( 2 * pi * w * (1:(n-1)) ) / ( pi * (1:(n-1)) )
#' toeplitz(a)
#' }
#'
#' x <- tprolate(512)
#' set.seed(1)
#' tL <- system.time(L <- irlba(x, 20))
#' tR <- system.time(R <- svdr(x, 20))
#' S <- svd(x)
#' plot(S$d)
#' data.frame(time=c(tL[3], tR[3]),
#' error=sqrt(c(crossprod(L$d - S$d[1:20]), crossprod(R$d - S$d[1:20]))),
#' row.names=c("IRLBA", "Randomized SVD"))
#'
#' # But, here is a similar problem with clustered singular values where svdr
#' # doesn't out-perform irlba as easily...clusters of singular values are,
#' # in general, very hard to deal with!
#' # (This example based on https://github.com/bwlewis/irlba/issues/16.)
#' set.seed(1)
#' s <- svd(matrix(rnorm(200 * 200), 200))
#' x <- s$u %*% (c(exp(-(1:100)^0.3) * 1e-12 + 1, rep(0.5, 100)) * t(s$v))
#' tL <- system.time(L <- irlba(x, 5))
#' tR <- system.time(R <- svdr(x, 5))
#' S <- svd(x)
#' plot(S$d)
#' data.frame(time=c(tL[3], tR[3]),
#' error=sqrt(c(crossprod(L$d - S$d[1:5]), crossprod(R$d - S$d[1:5]))),
#' row.names=c("IRLBA", "Randomized SVD"))
#' }
#' @export
svdr <- function(x, k, tol=1e-5, it=100L, extra=min(10L, dim(x) - k), center=NULL, Q=NULL, return.Q=FALSE)
{
eps2 <- .Machine$double.eps ^ (4 / 5)
n <- min(ncol(x), k + extra)
if (isTRUE(center)) center <- colMeans(x)
if (is.null(Q)) Q <- matrix(rnorm(ncol(x) * n), ncol(x))
d <- rep(0, k)
for (j in 1:it)
{
if (is.null(center))
{
Q <- qr.Q(qr(x %*% Q))
B <- crossprod(x, Q)
Q <- qr.Q(qr(B))
} else
{
Q <- qr.Q(qr(x %*% Q - rep(1, nrow(x)) %*% crossprod(center, Q)))
B <- crossprod(Q, x) - tcrossprod(crossprod(Q, rep(1, nrow(x))), center)
Q <- qr.Q(qr(t(B)))
}
d1 <- svd(B, nu=0, nv=0)$d[1:k]
idx <- d1 > eps2
if (all(! idx)) break
if (max(abs((d1[idx] - d[idx]) / d[idx])) < tol) break
d <- d1
}
if (return.Q) Q1 <- Q
if (is.null(center))
{
Q <- qr.Q(qr(x %*% Q))
B <- crossprod(Q, x)
} else
{
Q <- qr.Q(qr(x %*% Q - rep(1, nrow(x)) %*% crossprod(center, Q)))
B <- crossprod(Q, x) - tcrossprod(crossprod(Q, rep(1, nrow(x))), center)
}
s <- svd(B)
s$u <- Q %*% s$u
s$u <- s$u[, 1:k]
s$d <- s$d[1:k]
s$v <- s$v[, 1:k]
s$mprod <- 2 * j + 1
if (return.Q) s$Q <- Q1
s
}
irlba/MD5 0000644 0001760 0000144 00000002500 14316626071 011706 0 ustar ripley users 0d731892a736b9ecb6e6c21c6582787a *DESCRIPTION
1f29b808f13eaa1ff708a79059d02dbf *NAMESPACE
00ec66520f5b307bfd8238441f1703fe *R/eigen.R
0c60d68f766a2f251ee7392bb12ad485 *R/irlba.R
6b7204a1ab9eed7bb3131718c4e2261e *R/prcomp.R
b409b6faa3a27b65826adad184dc255c *R/ssvd.R
139e756a8467cc9a33c0fe0914ede41e *R/svdr.R
59e1470ca8e2c4bf5672d2a44d1fa04d *R/utility.R
43df6eee2280a38271f8b4bc1d49f540 *README.md
163e38ea3ecdccb1305feddb05e54fb1 *build/vignette.rds
ffcb4c3e143d951b9c1a8dffe6267a06 *inst/doc/irlba.Rnw
7988e89815e7295e4b1f0722b8d9f155 *inst/doc/irlba.pdf
bb47bd3377e2d5e17ea77b7ba616cf89 *man/irlba.Rd
1ad1f26493f3b039251517a411b7ccca *man/partial_eigen.Rd
e993145444a4b431fbe21ca4fbcb1797 *man/prcomp_irlba.Rd
5625e993d8d8bc9491c0da70c8856d80 *man/ssvd.Rd
0178ba7102496ab5c94d631cd6cd487a *man/summary.irlba_prcomp.Rd
9ffaabd60c8037f9c4f6662730c7a679 *man/svdr.Rd
619d13ce6900fca2139e2139408d39ad *src/Makevars
b8d27a2b889586072f34afd2c341681e *src/irlb.c
c3060f4abbe417e55fbed6f15af4ca62 *src/irlb.h
8666c6ba17647deb393f3f749a265edf *src/utility.c
5d15f8c507375f5caaa07e94d3dd72e7 *tests/edge.R
0b7c4542e5f8c9aeebacd80dc8c48e8e *tests/prcomp.r
3ad627da0fccf99f31b840fec794b404 *tests/ssvd.R
c514d2213577042647987a7bcb19da78 *tests/svdr.R
d156f5fb532403b3d106b9ac4b16341a *tests/test.R
ffcb4c3e143d951b9c1a8dffe6267a06 *vignettes/irlba.Rnw
irlba/inst/ 0000755 0001760 0000144 00000000000 14153273722 012356 5 ustar ripley users irlba/inst/doc/ 0000755 0001760 0000144 00000000000 14153273722 013123 5 ustar ripley users irlba/inst/doc/irlba.Rnw 0000644 0001760 0000144 00000040151 13430560161 014676 0 ustar ripley users % \VignetteIndexEntry{irlba Manual}
% \VignetteDepends{irlba}
% \VignettePackage{irlba}
\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage[pdftex]{graphicx}
\usepackage{color}
\usepackage{xspace}
\usepackage{fancyvrb}
\usepackage{fancyhdr}
\usepackage[
colorlinks=true,
linkcolor=blue,
citecolor=blue,
urlcolor=blue]
{hyperref}
\usepackage{lscape}
\usepackage{Sweave}
\usepackage{tabularx}
\usepackage{listings}
\usepackage{mdwlist}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% define new colors for use
\definecolor{darkgreen}{rgb}{0,0.6,0}
\definecolor{darkred}{rgb}{0.6,0.0,0}
\definecolor{lightbrown}{rgb}{1,0.9,0.8}
\definecolor{brown}{rgb}{0.6,0.3,0.3}
\definecolor{darkblue}{rgb}{0,0,0.8}
\definecolor{darkmagenta}{rgb}{0.5,0,0.5}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newcommand{\bld}[1]{\mbox{\boldmath $#1$}}
\newcommand{\shell}[1]{\mbox{$#1$}}
\renewcommand{\vec}[1]{\mbox{\bf {#1}}}
\newcommand{\ReallySmallSpacing}{\renewcommand{\baselinestretch}{.6}\Large\normalsize}
\newcommand{\SmallSpacing}{\renewcommand{\baselinestretch}{1.1}\Large\normalsize}
\def\tm{\leavevmode\hbox{$\rm {}^{TM}$}}
\newcommand{\R}{{\mathbf R}}
\newcommand{\brho}{{\color{blue}{\rho}}}
\newcommand{\Ra}{{\mathcal R}}
\newcommand{\PP}{{\mathbf P}}
\newcommand{\N}{{\mathbf N}}
\newcommand{\K}{{\mathcal K}}
\setlength{\oddsidemargin}{-.25 truein}
\setlength{\evensidemargin}{0truein}
\setlength{\topmargin}{-0.2truein}
\setlength{\textwidth}{7 truein}
\setlength{\textheight}{8.5 truein}
\setlength{\parindent}{0.20truein}
\setlength{\parskip}{0.10truein}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\pagestyle{fancy}
\lhead{}
\chead{The {\tt irlba} Package}
\rhead{}
\lfoot{}
\cfoot{}
\rfoot{\thepage}
\renewcommand{\headrulewidth}{1pt}
\renewcommand{\footrulewidth}{1pt}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\title{The {\tt irlba} Package}
\author{Bryan W. Lewis \\
blewis@illposed.net,
\\[6pt]
adapted from the work of:\\
Jim Baglama (University of Rhode Island)\\
and Lothar Reichel (Kent State University).
}
\begin{document}
\maketitle
\thispagestyle{empty}
\section{Introduction}
The {\tt irlba} package provides a fast way to compute partial singular value
decompositions (SVD) of large sparse or dense matrices. Recent additions to the
package can also compute fast partial symmetric eigenvalue decompositions and
principal components. The package is an R implementation of the {\it augmented
implicitly restarted Lanczos bidiagonalization algorithm} of Jim Baglama and
Lothar Reichel\footnote{Augmented Implicitly Restarted Lanczos
Bidiagonalization Methods, J. Baglama and L. Reichel, SIAM J. Sci. Comput.
2005.}. Source code is maintained at
\href{https://github.com/bwlewis/irlba}{https://github.com/bwlewis/irlba}.
The {\tt irlba} package works with real- and complex-valued dense R matrices
and real-valued sparse matrices from the {\tt Matrix} package. It provides
several easy ways to define custom matrix arithmetic that works with other
matrix classes including {\tt big.matrix} from the {\tt bigmemory} package and
others. The {\tt irlba} is both faster and more memory efficient than the
usual R {\tt svd} function for computing a few of the largest singular vectors
and corresponding singular values of a matrix. It takes advantage of available
high-performance linear algebra libraries if R is compiled to use them. In
particular, the package uses the same BLAS and LAPACK libraries that R uses
(see
\href{https://cran.r-project.org/doc/manuals/R-admin.html#BLAS}{https://cran.r-project.org/doc/manuals/R-admin.html\#BLAS}),
or the CHOLMOD library from R's Matrix package for sparse matrix problems.
A whirlwind summary of the algorithm follows, along with a few basic examples.
A much more detailed description and discussion of the algorithm may be found
in the cited Baglama-Reichel reference.
\section{Partial Singular Value Decomposition}
Let $A\in\R^{\ell\times n}$ and assume $\ell\ge n$. These notes simplify the
presentation by considering only real-valued matrices and assuming without
losing generality that there are at least as many rows as columns (the
method works more generally). A singular
value decomposition of $A$ can be expressed as:
\[
A = \sum_{j=1}^n \sigma_j u_j v_j^T,
\phantom{xxxxxxxx}
v_j^Tv_k = u_j^Tu_k =
\left\{
\begin{array}{ll}
1 & \mbox{if}\phantom{x} j=k,\\
0 & \mbox{o.w.,}\\
\end{array}
\right.
\]
where $u_j\in\R^\ell $, $v_j\in\R^n $,
$j=1,2,\ldots, n$, and
$ \sigma_1 \ge \sigma_2 \ge \cdots \ge \sigma_n \ge 0 $.
Let $1 \le k library('irlba')
> set.seed(1)
> A <- matrix(rnorm(5000*5000), 5000)
> t1 <- proc.time()
> L <- irlba(A, 5)
> print(proc.time() - t1)
user system elapsed
17.440 0.192 4.417
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 1096734 58.6 1770749 94.6 1442291 77.1
Vcells 26685618 203.6 62229965 474.8 52110704 397.6
\end{lstlisting}
Compare with the standard {\tt svd} function:\newpage
\begin{lstlisting}
> t1 <- proc.time()
> S <- svd(A, nu=5, nv=5)
> print(proc.time() - t1)
user system elapsed
277.092 11.552 74.425
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 1097441 58.7 1770749 94.6 1442291 77.1
Vcells 26741910 204.1 169891972 1296.2 176827295 1349.1
\end{lstlisting}
The {\tt irlba} method uses about 1/20 elapsed time as the
{\tt svd} method in this example and less than one third the peak memory.
The defalut tolerance value yields the following relative error
in the estimated singular values:
\begin{lstlisting}
> sqrt (crossprod(S$d[1:5]-L$d)/crossprod(S$d[1:5]))
[,1]
[1,] 4.352641e-10
\end{lstlisting}
\subsection{Convergence tolerance}
IRLBA is an iterative method that estimates a few singular values
and associated singular vectors. A sketch of the algorithm is outlined
in Section \ref{sketch} below. The R {\tt tol} and {\tt svtol} arguments control
when the algorithm converges with {\tt tol} specifying
subspace convergence, and {\tt svtol} specifying convergence of estimated
singular values.
Subspace convergence occurs when the algorithm iterations find
estimated singular vectors that satisfy
\[
\|AV_k - US_k\| < \mbox{tol} \cdot \|A\|,
\]
where $\|\cdot\|$ means spectral matrix norm, $A$ is the matrix, $V_k$ and $U_k$
are the {\it estimated} right and left $k$ singular vectors computed by the
algorithm, and $\|A\|$ is the {\it estimated} spectral norm of the matrix defined
by the largest singular value computed by the algorithm. Using R notation,
the algorithm stops when
\begin{lstlisting}
L <- irlba(A, k, tol)
svd(A %*% L$v - L$u %*% diag(L$d))$d[1] < tol * L$d[1]
\end{lstlisting}
It's possible to encounter problems that fail to converge before
the maximum number of algorithm iterations specified by the {\tt maxit}
argument.
When the largest singular values are clustered together it can be hard to
detect subspace convergence. More recent versions of the IRLBA implementation
include the {\tt svtol} argument that specifies a maximum for the relative
change in each estimated singular value from one iteration to the next.
The convergence tolerance values together help improve correct subspace
detection in difficult settings when the singular values are clustered.
But in the worst cases, block methods can perform better as shown in
the documentation for the {\tt svdr} method.
Also see the related {\tt rsvd} function by N. Benjamin Erichson,
\href{https://cran.r-project.org/package=rsvd}{https://cran.r-project.org/package=rsvd}.
\subsection{Differences with {\tt svd}}
The {\tt irlba} function is designed to compute a {\it partial} singular
value decomposition. It is largely compatible with the usual R {\tt svd}
function but there are some differences. In particular:
\begin{enumerate}
\item The {\tt irlba} function only computes the number of singular values
corresponding to the maximum of the desired singular vectors,
{\tt max(nu, nv)}. For example, if 5
singular vectors are desired ({\tt nu=nv=5}), then only the five corresponding
singular values are computed. The standard R {\tt svd} function always
returns the {\it total} set of singular values for the matrix, regardless of how
many singular vectors are specified.
\item The {\tt irlba} function is an iterative method that continues until
either a tolerance or maximum number of iterations is reached.
Problems with difficult convergence properties are not likely to be
encountered, but the method will fail with an error after the iteration limit
is reached in those cases.
\end{enumerate}
Watch out especially for the first difference noted above!
\subsection{Principal Components}
Version 2.1.0 of the package introduces optional arguments and {\tt prcomp}-like
function syntax for efficiently computing partial SVDs of matrices after
centering and scaling their columns and other adjustments.
Use the following arguments to the {\tt irlba} function, or the new
{\tt irlba\_prcomp} function for PCA:
\begin{itemize}
\item {\tt center}: if {\tt center} is a numeric vector with length equal to
the number of columns of the matrix, then each column of the matrix has the
corresponding value from {\tt center} subtracted from it.
\item {\tt scale}: if 'scale' is a numeric vector with length
equal to the number of columns of the matrix, then each column is
divided by the corresponding value from {\tt scale}.
\end{itemize}
Both centering and scaling options are performed implicitly in the algorithm
and, for instance, do not affect sparsity of the input matrix or increase
storage requirements.
The following
example compares the output of the usual {\tt prcomp} function with
output from {\tt irlba}.
Note that in general, singular vectors and principal component vectors
are only unique up to sign!
\begin{lstlisting}
> set.seed(1)
> x <- matrix(rnorm(200), nrow=20)
> p1 <- prcomp_irlba(x, n=3)
> summary(p1)
Importance of components%s:
PC1 PC2 PC3
Standard deviation 1.5411 1.2513 1.1916
Proportion of Variance 0.2806 0.1850 0.1678
Cumulative Proportion 0.2806 0.4656 0.6334
> # Compare with
> p2 <- prcomp(x, tol=0.7)
> summary(p2)
Importance of components:
PC1 PC2 PC3
Standard deviation 1.5411 1.2513 1.1916
Proportion of Variance 0.2806 0.1850 0.1678
Cumulative Proportion 0.2806 0.4656 0.6334
\end{lstlisting}
Alternatively, you can compute principal components directly using the
singular value decomposition and the {\tt center} option:
\begin{lstlisting}
> p3 <- svd(scale(x, center=colMeans(x), scale=FALSE))
> p4 <- irlba(x, 3, center=colMeans(x))
> # compare with prcomp
> sqrt(crossprod(p1$rotation[,1] - p3$v[,1]))
[,1]
[1,] 9.773228e-13
> sqrt(crossprod(p1$rotation[,1] + p4$v[,1]))
[,1]
[1,] 1.652423e-12
\end{lstlisting}
\subsection{Truncated symmetric eigenvalue decomposition}
Use the {\tt partial\_eigen} function to estimate a subset of the largest (most
positive) eigenvalues and corresponding eigenvectors of a symmetric dense or
sparse real-valued matrix. The function is particularly well-suited to
estimating the largest eigenvalues and corresponding eigenvectors of symmetric
positive semi-definite matrices of the form $A^T A$.
\subsection{User-Defined Matrix Multiplication}
The {\tt irlba} function only uses matrix vector products with the input data
matrix to compute its solution. It's easy to use R's native object model to
define custom matrix classes with user-defined matrix multiplication functions.
Such functions can be used to support special matrix objects, out of core
computation of large problems, or matrix-free operators.
Here is a simple example that defines a matrix product that scales the
columns of the matrix to have unit norm (cf the {\tt scale} option).
\begin{lstlisting}
> A <- matrix(runif(400), nrow=20)
> col_scale <- sqrt(apply(A, 2, crossprod))
> setClass("scaled_matrix", contains="matrix", slots=c(scale="numeric"))
> setMethod("%*%", signature(x="scaled_matrix", y="numeric"),
+ function(x ,y) x@.Data %*% (y / x@scale))
> setMethod("%*%", signature(x="numeric", y="scaled_matrix"),
+ function(x ,y) (x %*% y@.Data) / y@scale)
> a <- new("scaled_matrix", A, scale=col_scale)
> irlba(a, 3)$d
[1] 3.9298391 0.9565016 0.8266859
# Compare with
> svd(sweep(A, 2, col_scale, FUN=`/`))$d[1:3]
[1] 3.9298391 0.9565016 0.8266859
\end{lstlisting}
See the following link for an example that uses large-scale out of core computation:
\href{https://bwlewis.github.io/1000_genomes_examples/PCA_whole_genome.html}{http://bwlewis.github.io/1000\_genomes\_examples/PCA\_whole\_genome.html}
NOTE! The reference R algorithm implementation is used whenever user-defined
matrix multiplication is specified (instead of the faster C code path).
\section{A Quick Summary of the IRLBA Method}\label{sketch}
\subsection{Partial Lanczos Bidiagonalization}
Start with a given vector $p_1$. Compute $m$ steps of the Lanczos process:
\begin{eqnarray*}
A P_m &=& Q_m B_m \\
A^T Q_m &=& P_m B_m^T + r_m e_m^T,\\
\end{eqnarray*}
$B_m\in\R^{m\times m}, P_m \in \R^{n\times m}, $
$Q_m \in \R^{\ell \times m},$
$P_m^TP_m=Q_m^TQ_m=I_m, $
$r_m\in\R^n, P_m^Tr_m=0,$
$P_m = [p_1, p_2, \ldots, p_m]$.
\subsection{Approximating Partial SVD with A Partial Lanczos bidiagonalization}
\begin{eqnarray*}
A^TA P_m &=& A^TQ_m B_m \\
&=& P_m {\color{blue}{B_m^TB_m}} + r_m e_m^TB_m,\\
\end{eqnarray*}
\begin{eqnarray*}
AA^T Q_m &=& AP_m B_m^T + Ar_m e_m^T,\\
&=& Q_m{\color{blue}{B_mB_m^T}} + Ar_me_m^T.
\end{eqnarray*}
Compute the SVD of $B_m$:
\[
B_m = \sum_{j=1}^m\sigma^B_ju^B_j\left(v_j^B\right)^T.
\]
\\[6pt]
\[
\left(\mbox{i.e., } B_mv_j^B = \sigma_j^Bu_j^B, \mbox{ and }
B_m^Tu_j^b = \sigma_j^Bv_j^B.\right)
\]
Define:
$
\tilde{\sigma_j} := \sigma_j^B, \phantom{xxx}
\tilde{u}_j := Q_m u_j^B, \phantom{xxx}
\tilde{v}_j := P_m v_j^B.
$
Then:
\begin{eqnarray*}
A\tilde{v}_j &=& AP_mv_j^B \\
&=& Q_mB_mv_j^B \\
&=& \sigma^B_jQ_mu_j^B \\
&=& \tilde{\sigma}_j \tilde{u}_j,
\end{eqnarray*}
and
\begin{eqnarray*}
A^T\tilde{u}_j &=& A^TQ_mu_j^B \\
&=& P_mB^T_mu_j^B + r_me_m^Tu_j^B \\
&=& \sigma^B_jP_mv_j^B + r_me_m^Tu_j^B\\
&=& \tilde{\sigma}_j \tilde{v}_j + {\color{red} {r_me_m^Tu_j^B}}.
\end{eqnarray*}
The part in red above represents the error with respect to the exact SVD.
The IRLBA strategy is to iteratively reduce the norm of that error term
by augmenting and restarting.
Here is the overall method:
\begin{enumerate}
\item Compute the Lanczos process up to step $m$.
\item Compute $k>
stream
xڍYKsW*U3xَ7JJI=@ H!K
Y}5> fzzzvtp{v9<86w T$^ $ĩw.nzq_.r9
] UIU@.g)(S1ץ92
k'Palp*T*4*N3Iia
LU cO(Q *ʂ94[
^!{X@
*{y'l3eoj43Ea: $En5=gxZ {)U|Fv H}"z^OG>| EGodǑ-y :S4 ! C!xJ9iQ [ Ľʊ!a'\vܪcS8#HT+ H(aY?!Obٝ2㏖H=8 :KTDч^pqI' @&n IPgG
+ eZ ^NO1zx0f٠FF;Pfy`)1 lD;HVD, ;m1ʚ}ԍo TfT?Pԓl:2Y6fzk',N-(H֑٠.W$; X' ecYKI@bu ζ8\Tx~8fDL!(#́X3VTH5jNA!$A J]] O EU?쫢/.#>o'dW8 Mwզw!W"`8Ru_Ϧ^6 Xh P'u&r*W% pH"fx
MPl!Jm! |O{-B'|Y)=sQ$|yZ&|^pxJ9MPiĩblECG`: \+O}%{ 1p
w'hs'W\duU}E>^#G * v
G rg +Wh
3d7N6+xNL"WO&ً`͏W*SNK W%RjϦa1V@]P;c(( ȗ<2}X) b7:X\(B1)(`WPR: 7p| bEݮgm= Q$yf^3Dئ3#{f2Qr4ёF^P1O;"_73ӿJ$;veԩu._Hw_A ̶'MU4jWnlwKf/1Hp>9X_x ?Tc/GEEޜ`gxXSIx_?h /<7)8_?.tַi)Үv5.JAN"=qCi!d XhKc 0bՎD)8lȮqԼd8WOwo$wA } V!|�P>n.#Ğ^(- @ =h{! ePKCD'OD3YDi2T98mXcr3g$`#~[\b#J5
yWQ9W80w~>{y=TzPWpFow%y!H
Bu+곯Ɓ-{"3VQhݜxY HozLs5q92@>"cD*֒.PSx dK4*k.lԴ@u, omfH O )AQĒ [#B=xO) 7? +Bb3-Ȥ `1 s
endstream
endobj
68 0 obj
<<
/Length 2404
/Filter /FlateDecode
>>
stream
x]s]o;vNjw\[u;d&DɪDQI p 2ɴ"~BdѷkU)%ZturQimiD'?|#p+Հ,]Xw?3 6VxXaw Ĺzv G++U͠ZaG?,sXm}u 7Շ \BT*
pq5ГR?FE~5\dj?\"c>NZS 㢫_Q~`Es[Ҋ75=RU} .QZ46AhW!O hTB0%_gPkL tܓ*YԒA}^jjId~)2a`V eAvhL5 c5':ٿco狯R M*B7umD)JgKV[u:ށi%/TVxk
k=I9gmU!
8b4V55lb1wj2cq]cM2ͮY/UTxz\/L۞%>ܯrHT 1Z&-M $RgK^+c\2a.% nz+l3E7ߎޓ#dB0tQ:~sC2EҋV /m$x6pF [7d~Nm2MܐsVO9A8aPUad祴 +ZkG9ƌ ӂ )SeP2. K>qIHˠXAvBƖ궚Ѵ;[ix <"-+m1+qnK>es|,gbElHėP w|R7 Qpݗ 5<'}UV$H,rp'\BG\T /Fʼn2.6ٳvad5iPS*m b80{>24CO}C)? uYCQq^*ʪå _\^Ɲ/Ac.(\Ùc6
)Lp
endstream
endobj
92 0 obj
<<
/Length 2416
/Filter /FlateDecode
>>
stream
xڭr#C\\!vq WS.;AI%\sߧ #PҦrq46Mxɫ l94:|xel՜ήNl}zq )hdw
`[~A%nmש /ͩes?Nlhz c];9hܴ] mޞ%.9s=*=; iI^?!r& A;dVXsi1^?³q/{x-?fP~;RbX Q w+B4$TLH6 ;|CNV!$[abH~7?Z JO^N-<sNFp C~`0C0ֱ"z!
>^p&sNgLkw=Gڮ*ڸhE'Hef`4W_,%8Ȃ`e`
[׀E>u*5 lE |$GNMnm-&Dǝ
@\-#/`$quzDs
V4`AM Hjg[NQz:sGGE |2 \cZ``1MA<ݖ 0+K \xҜ#.5g>?7J1Ԋ ^X=TT+Jt/q Vxу[\XKR/SY` ]iOWdV 2QhwcW=}KUT {P-ϫ I_\{̇.
bŲuש܆!Py5R{0K'#Y-$?s(Jg>%R K. $]pG:ܬ+yπ8-QZNd"<ý! )k);GPoTS*1#\nR3hF5 ٝ >m)8xÐxOwэV".+l*a/04}5zn$FҀsT(;yV@W
P}ZIHȪ'uBuTl) 5RJ j`%8V\zHH{7Ԫ E+ 3V,' `C3ocqfMMzmqG:6EWQ0Sv-4%4J` jRK/n dT~=U
t 4]- vVmx0MRx^}3T^n/xYp[V^lP5՜xIpp}8@ O9x':ƶlP@Ž}6gTEʸIL
엘 _9ĒzK* 4 J1&GL3O Eψ灀[?65yD?#ZQ÷߆I
gǖ=aӦxdeb|KM7*\~yJoQqf|/RO_}7. _K}̊մ`1
L/d.&ׯKsõO4_6UT?8i\i;]?EJVYn ]67;%v:@u xE
ɂElM}"8p05(^ZLRCt[-Pm_2 $#4qdsO5ݿauA\Ԗ/05ߥ2^ӹdmr{_W-4m1 ղ\Cqfl;.M`IfLJO4U\ŽeXQ> w[" mFXOkdžk p7"KY&ɳ
\~T>Di$
endstream
endobj
114 0 obj
<<
/Length 2725
/Filter /FlateDecode
>>
stream
xڝZKsFW`OLU9I 6{S[ J)B& )ۯy %LLwO?n8ORiQFXSS0%T ]@e Yc ~(t!8k;=^S72 แnx
t n w iGٴ;0 98z{/ͬ-Y*@ɴ_[F :Yjnͥ[Ѐj!
kQ-
zi{YLy.Q;g$cJ;R rl e]GG]% S;iǪl`%w8▬X=m;w ny>:!qJN~j.y}b.~ӕ֎n誗L=G;9'9"gĦDC 6$L$ufўQR9d3 P=Ҫ
̮Ň:v|= |` 37MUᘡ{?YƫL^ڐ+W~Wƿo)t|Zw磼`w+JHג~"ԦZA"Dk
IR'2DeaO 2akiƽ? F|K>ȍzkƖ20R vUfnl(LQ=( VQYeg18Xp w- 0mZ1( 焗yQ *n 3w"OWG" 0ogCHStݮEI18@LO6RH]~0p8ҌX."Kfhr!A&+ji5k`Ll!0)u'n3|V@ܽCW.[BV`dM^ nFHYbrG
h)*'lQ dlKp\@zp٠w/ kڼS81/l XPzq5w%X\c6`e]lD\d_[ދ_|b*%a9:XGj ESʵq=29 9=X!mF7f|pDZq$љ&'@ a> & sLh@]Aʶ@PYχky^m|́cHGcut ab'i#@ȡ泳."nd3E'
}f'h|;5D^k8/{c8)i+ZTP^fL5S l ^gVQ9_fcuyNC3S<4!#]LCdr褔0/qeD:^S=/m+^BkD$7&ɍ`rΦq)۷THLCr'r&UJ#۰ t[&dnK
7Y prh0鄀k:RhB|1lu E=BG H`z w=ovv} o]r P [9!. 6|Ewڠ-N>`vj9X6.1ͺJh]pqtyک!$4\
#y}kXJ!9%fw+ >ϴRfC m8
cƝ7auIlS2LT̥2Krmq*
_c\c(Һ]B: c jMzVP=1k~ CE{wngˠ~|nB.ڷmu6?]V>r0QU :|v>F}H
$Ytډ<=ʪ3Y? (zg=֨ɚqđ%X W6| f
V.maqy.F C1\Gɚ
ǵbF7PqCE܅jolOuN6}ğIf/ɸ8z!,$!?~y8{5RŐ8*f>
stream
xXK6WH hGHlѠ-fQP{-J |q0ՙVRQaƳ@4YdD:σ4_*㏯ά鐛(69mτχ: '@\ߡd >7LiP^
..`
?qTip種x.O?q
Y'Q\hnB0 GI ` }6z]ØS.Q!r]3g\ZZ NÑNI
r!n˖ՈBhE
)Pg HQŲ/Uf6l7+BӓFEq
7z n!Q A?ې>|Iٸ84O ~q ڶx) 2gKQ Hwn}N*-ɃDC* ).@9,s KdŤyi\(ئWW|Qޅ(` D*Ms 9"K# ⌌3β"9.T^qY xVytnI[ 9[+//i1'Hyĸ ~
i%sI(A.Yxń8> ;r%
Cy :G "j}疽?_;ʤy9,'[,QͻvyO(K{y|=)eGLNBm0v- öc7%ʡ,ggsq&PgX9%R#^DZSPІå$qd Zn}VM\UOd>93ܟKtOvB91et-ӿez?;E\T
)vɰ*'LZ}v)ט0Ԑ::$nkafI4OtzC⚈;N6q!vG{W]m;Vz 'lڄn
7iKߑo+l߅BQGyB đISG}NJx-z՚so\*zW+vq yr6}0~Ckm>-M;uѳ<^,o{ ^8s%$.f˒(IM=4=FAurqOQsYS˲
?=ks?IMHɩjuMQD9
endstream
endobj
2 0 obj
<<
/Type /ObjStm
/N 100
/First 801
/Length 1743
/Filter /FlateDecode
>>
stream
xڽX[oG~_qsT)CDM q6{7Z~gcqP% {9X$GF'(Qp$)eIR6Sd -#H(}5HLDVzfi!c⸱{r"**V x \!
,"%Ha=K䠚gqP%>zȳ|1^`$lQ,O d)8*"JZG!*k78AGـw 6Fh 3 I[!WE@qDQ`|
d)5?K$%;\
O#FJ˝&q4*AA _ıK*"c4 T-̇C @
# 9) cʮap"gM[&G&S092,ZU($ 6-Á3ܓ 2AD&js`$f0p}Cwo:*ѢM
>- h;Y=/~h 4rV-kګ[4w0Y k?YNkYW2GWJ5u'i$sҬv\go["s^|/i5'>
stream
xY[۶~ׯ`vJ [ti2$Oǥ)jWn%{n AnZO'!ùQpDדL:Zi7 N3eFẼ7TͷWS#7*210#%Wb>Ĩ ziiaukי|u3yQX3ץ2iɛQ0oHej *x5$w6*1 fYG k*3i fZ&̧$O؉#%afI7D YBZ 5Ἡ8muEV8?͘`*IsN&aDuaRȪ]ÙO,\>G 6yK ˖
4[K;Ht1\RhQCZQkmU^/{hu72Y#u_caTn`%g?@)w a70o)R2 xi>",'w$ܲ>Y!2y
!/{4#-}&RG
Ca[Qz#1! sh;Ṓ yFS"巿N-}Or (Wru z 9sAb$vM\9zcwT7']֕?p4 kh]ۏ.aB_Kד,KMB:#4+ro:-m@w/w2bB G|=b%*tgQy7#2/l֪|/?ZZz*,v" Mv9"qu{ z7. s5tQF9>
stream
xZnF}WB/4M (
$؊4Թy,Hjh"Ξ9;4N2==<{,E& ]&e҅Iͤq*?8|1sc[q-;|=LmJ3J\G毹1I=׃߳(3aN'fGWoxvgg!k)uqfB3P+ib ͕MsGSo$tƞb%?n'|97 W ͔%,<+HGLTFv3G-gC|~8v1
YώϪ Nq[./(
2tt]JY5x I 2>_\(J_)ٜO(u4\U2hm(T~])F#WUh$ݺVoa;̭tAwxYb6J^^{SPX}w\)^6aEZVq YE6'շpKt8zyZVYc-u4W,w;:w1|'ؿ{f+sqݎ a1k8SZm"~Lm}=0kiɥo\ɒY 1B^:?&b7y=nvF7O>>
stream
xZKsFWHUy?`&MmU V(Wt }1` 6!MWr Bo=պ泌`Y]VX&]Lz_]T/Wū f\KX{dzk+,I?]H7_5^]3_>f͐WNg/_Pq>TV/f?3
sǓ( Bp6>R00)|i RfM ^K)m[Ru
g? e
9SK> )-`4pDZm Z9惘jA^xt! +3!
lej\Kb-rC S+W!ij츫$; LޓY+%9 p[xohW-āt[
B1 \ƕ'ׁ {Q=%B5oHDbY2MP]Fv:*0b겈1 ] ;g4 QaH]HyIþAXzd4MTЦr lH@G<2Kw
b+r--Ms`ɓZ4%IciL=7ǒ~8= ΔYcs;6 v \ze ]3/M^ƜϹՆ:ͺ<" m娲Tf+ Ua|8rZk /G\nc+n iGx4F}]Oÿw۸n|I75k$xn kf] \Q7\>?xcK- m-X^#
HW H`8D2\%^ ߦS.v!FHi OVq7>eb5[dFjWVMu?P#J:M0a_!ԛfz&e'\ ˅a7
L`Fwd.m֧m6@WNq: Q!"yocH"*4$RA8PH NJ}Hͩ^ b97
;D7T68/l b2|[.ΒU\b B8G@!#ܙ|H̗7
i9zIuۛYՍQ#Ӹ̇H^upՇn9jOh&Oғ4 }?}BQkqGz
}뻷!%1#ĴOj& o$iU\qxX80V3
o5Jռm&n"z4xbf9 ȯMV;u؍[JRŎ Ǹ`i0æ%׳ΑR_],\ ]US]sz2M&Z
oJ ijni:Y2S"U撱nl?Q+A VzOqPjyi`İT_7 9%sOic>]By:(/|J,D+SVJGC0!Y
.u`@ȤiMilՌs e&R*8s) kLKxcE XXH4 G%끡g4T fB :tf^:f
>
stream
x337U0P0U0S01CB.c I$r9yr\ `W4K)YKE!P E? 00(?;h0a$>z A ?$h LF N8\ù\= Y
endstream
endobj
209 0 obj
<<
/Length1 2060
/Length2 14375
/Length3 0
/Length 15620
/Filter /FlateDecode
>>
stream
xڍP\ֆ 4Nݵqoh ; A
KzHޢ
6;TVc5K \XY
b 66NV66ZZuk;Qh5!`DCA.6 k j`8$@nf V,V ry=? v>>^kS@benz)66wV..@;+ޙ f`[XT͝!nf$AƊB PvC
l^
v֦ί)f 5yÿ Y[v;dj
w9xZ;X,JR. . _ ;gk>
dm2y
u@JT zUo}ΦkGgVgk4*z͒f`{{sg' v0K#P\F1&?6Ks 7/ `aj uOG_5z; 2}-_x;.Ws_:Pf֦. sKk?_ 豽 ;~2x03 1P⃨ӿ%)& xpsX8| ^n6QY?2` ߿}0 "ur]q;ߔUuoGRvv[y;ur]]^@ 7T_+3>.:X=͔]L5.khv`g^- v6{.SׇL2]=Rlזqp @u8hf`8_ E2x @? w#_ ek zS/c ; 8 @?ZSqO?z< znh| ڎ |uZrZ_dsBQ5 **'f|e|f|Uff__̓ 5kܯ\
kښ?5\ n\Hx|| 7|-|oM0u@^~W/ sssS9@MmHWQRw BӴ;Z ,ދ{ uȵhp/ʶ$%'zdGDɝ6 OGud,w}|4ma[di\a(I{/Vȡ>OjQ!#1y`\]O䍿P&2 r{npzVs8w^NO%. /قpIUCٵ(X>29Bt`(^=oE+6|`I_[2nTҹD]9
u9Qd&^s?|?ugpo>}I^7w|\)WR]DX}l݊2O $Wkh
V,vFff] biȒ{4y' אr=ɸL~ (j¯z sLpDF+!ե ЧB:p˒er eSfbfI>2 #0SKHb©\ ::Kgj\0]XaA8BDO\4L%ܤksmU8^rXOĖ"x1/]fNl Q ;#,20妛r8O 5.
o|q[C _]!s BVeqASC? Z{g
K|)gP~` [~Yw؝0kG fvcns$Gk9AQZ9 }f`B&Lf 4#tG4C?D"9oش@ WFs6 ~n'_%K !^W7ǔ ^8 $o p:MB - 0 ⇴:vUٍ
+ Bni"0Ǣ&Xl,>D\: >fiHBũG3d *|u1 !^ ߲#/P"MD$
,\h?jJTTeh-ɥ:6T67) ձ'VPr}
ރ9XGdje)ה"4:B<ζ
O)a t=>VPA3g?am?a$xR% gݧ1g|ycҍH\J >#aj_$Oeו );H\a4dNm?U~Yc#Lc3
a GHmOϴI);r:&2 7nKY~?\Br244 g7!+n#mτ0tDz>=B395mr6n\34m,odt6a/hz\cl%!k`&s Nio} \ i3`4 XR2
t``:Ct IW54nBQ
P@{:1m~9GO=G@uB&fuɕZ1)9jESUui;( Eq?[t㴖
j,?IQſ!~Փ
sRj ` dw-.ڛJXs\4
oȏ;ޖ,3N"P1qѬ O|J2/Ǩ.Yg4"qՄGQxo]2Ԅ>aœ5xg9-JV¿G9$PKgL(?(d xQ!(%z: V<ޢBkUnYPHSQ>irL씁%jwc s(dǬ; jwבҚ<&hM07A,/]RC=6CUkeFaWy B&c~FOA2"h^%'Ʃ
TrӨ7{m?Vy*/.>3e##,XMI!+Gh0sfN:ˑ lۘV}L.UGo 5AqL)PaL%}6bq)]b B膯3^πG Sؐϡ%RD"^sj
P 'ps˿wW7âTDppI7q1: rIO5#<O\,<Ⱥ <0өߗZB9 %ժ
LFB5KK?/MKg p+F퇓i5 w]5(ݮLxvfW?LÓ ne+~<ƈhE*F+3Uu*oBjX"Z5c4F%vbpA_QtDp_J0 5/?˜.¿^[yt|gHȐr(d|2 ,i?ʸ)>vy
o:4 ]RAC70LT ]>KV3\ZmzBl
ik}O7"C`Zg%G&qbT'XU2
0lӡ1x"]Ŋ"za&{#SFxOoK\e|B^PQ*Ti_)QI?O] I ǜԣ
`Ã~BvFBvndQgDki`/5H#+0bNX5 `yNqrt( h'?zòکe=$d6 ֧\BMGؿyB>Y;R 6XTo#DPe=nlLN-1y
4zH } #HRc볠p/z eͣ;ޗFudQmӮM"Y ^倡8jv7o
BݤC`60zmAٚ;uw~vyw_gl!~2!Pq3n&̽z ACX:7 dMt#Ţ~LF靪?E5\ $;)]ᗽ.M~~
JJz3yV1k^#IbK$!
Vc< J{7f~p.&@cxIW<~|yz0R{Z~`4&Cy:Ky8d}Ѐwע- U}kfggR`
.
LK xԅ}^R%XSA0tA ;de 3V44 Oh,*8y eo>v \ h.u>qgzb"oJJ`?Z- IJ
[SFu /DKy o>Ɍ~kHv eTYj
{#mCb:G(yRE:i?!&ЏP~> ܪy|h|ȡcUOpe#}H}bJn&UsPoe{X^ɑJgE}$D;*[}V,&ī-C)i+y Z7rk̙=o$|iITz#|C悗.;e`a%rQ9MCI/ /!7 =Uԟ@O jG:KYs lNUy8/qs AԭU ,"H' ^.jDPhӦ*D[3*TC\-Z dCtN~R~bodWfE V?LxoxӞ a9c4ss ;*m ?3.lt!
=]x?Z6!_V`r t$9SK]SXYv/be o ~̃k͝mp+E0/+E8aJA=mD2zvW2A^k }Jۊ:wx1.id{p:@.
ap!y"|!0]RY4㳬 /=g[zKE Z0焤K~]} .L=u{,b"]oB<;ÂO~chV]):b KweEyᛎ($br/r 9~hC2+NEw0!KW)wJ
rߗq8dXuH 2?qO| XպxfҮ|ţ[c'8RT?Ycl _lm?Y.>$Dc ]U/P~kwXy]L/ynDJHyR".D8
l7Y˽EO%xF|D5 &-m|p7u çm#kiMa훶h|B4 rc |Q&
;?;m^F
dd Ջ^FoUwP=cKʙ )!+$,lǓt#i1rުgEVz 9 Y^F9r!Vk)ǥ\<6(pRvt GyNM}J|,]t' d}eQZ^&F&>*ԏ c=eDu*j=>rn` L#:Ѫ 5 dPR)zPFDS^$%jLq4o3T}F֙|:Ϻc?'Xҿ#Yq G '$MJPpq_q ﭲf|sR9u]5j˺kL '[ڛr@ݯ#vbW bʶ^ksϟ'лf͜1'Y;YbeVMV@j!p̰_l\1+: *~9VQK
^cc}yE[Gh)O.u0Es}&1;Nɪ=a^.tYM4[H7p^.HS?R,g f9O'XL٤kŵ vQ,Lc7M ߫N po
5|T~LehM&y2|fcTdzof;OsM igeNGc%|l) pTHgj86/pHy |ȄM{;g+p,eϐG`u.<.}tPY h[`'[2! LݭhՉ^$"O$8 Ibf}F F7#|qCҽ:^a]kqcCJکw<D?YjHVD)]>\ o-P4#({~/;>֘ѥ^YixVTR6!=1hP칠[NGF.q-l_cnq,"J0RgĪrPw8@IGĉM%E_x2/ѶtP''omTn@pF!-,zȫȃ)QBh!=:jjڍ ^þ"`e=eH+}E7PYgg>Y=ȍOH+;?C`*.d@Lye 'MǺ's\{>2)WBjis>j
&{抿FLOcg-,pT#/x)ҧ_ ?dz+b
5FjԘT a!ס xIM<:c||: ݜ1Z&- @Ǘ53o``=Nw/d)a ˃[W8 ݗy8WZޘLeiJw=;٦.O/hKĈNO.H~WrOy:5uOF} :DGzԞrBE(xzϨ m@gu13ep)[10M2!|T
=ħ `(::.xnM6=R6U5DWJs)V-s #>55,4*xGs{XGZ YY~2\I!K.[u4!~57\˩FO@[>A{KZz/o0 j ]Q(3oZ^;&6}m4k@0BڷQK;F
L> ލ >*%/5FwsB%{Z`ȅY ygLRbݟ6LV.`}I]>_j9줡>Ebu XD7Zh RSfˢGB|lxGt'ęVJlO]6Ys sL U
ׅu%ӵpMRYfB K>SצL
渞ǦjsQ}} =fP ũiYEsW0]qR} }k'
»kvV'Km63YuAQoJ5ΞŃ`^o$؋A&zX/;P .9S⇳ 8j8.f1Ы;XN/ه]9Ӄx&zE]PQhμ O!vhaTw:%m?~ =758dePE>c5v oTtXwMԚY{ #5]Huij;D!i 8LPzS{-س
7WD<,K0L+9#
*n8)ZNP:18"3pD<Rjp -Qꩱ)GF#ψ0Y5«YVD0e9'wYֵ__7IJ(w%,ˮp#N1.D
Qui '$qD
p%CW pnTt
;R ZܗIw V%_HxG/ E-2<# 7:q
@G
RCB*ۤ>=Pn
wW"Ff/u|i
˩ZjlBBjYsOf-KFOÓү]qBsx|R( pnii2+H$=8=hYxp (eZɴ
:+KM3"O#y`t/{6fGxrHoc@Ӆ /eCQ(|\dA 8"Z~Wj?ӎQ ϱȊ>%z JE<:
<̎gRݤgDf7BE܍6ufu(gj>k* <;`#-(cԅkz %e %-"E *Q]>J&KSjF FCEJ0+&Z| ,Df^eamMqK= &b f٤P%p!{tgn_ja a#eiɟmZ4,S~O
({%'n" dޯ\*Xqpl,4[*(7K8L ݳB K Wpr/Hue$US5e!%@ kI{*\{4LEH*_$z!GxqבIܚVsb潻_pV{ 7)]^b {P9mm:ee 0#a8X.7K&) $R'6KK`&)Y/$uraa#M.u-ԱYbS'#UQ5'wq\ S̍M7Fzߠ4Pqna{x.7D@V&~ěwkuD&AhnP h/;*}`qt/f*$)(U2DwĞ
իA4A9}KkCd *@ Fhj,iiMu
ZuhVmX`)l '4B;QPc\+i? Pr#A_Kmʶx DQ 1?Y":a 9'ޒ 3| Hg U]~H;#daT|+ |/=Iy]儊U تsޙrm f>[B}Su5 Ap>wd5j vcS'|} ;wۘQ^WG< oKia^flMI.>]iM7!e.Elq7/ s@Wv9y3c5Hƛ7B١Wme Qb3 :J\Z2%c5dxOfy397M+ 8?̲}hOdԶS tSgQ 7"g; ys']$I<_m@ iH > `]G ]f-NݛwA~jZ+V>x~"Lvz!ij,Ko*|/y_"בc7\
LjÇwnw euV2Csa!`V W ʄ 0y癭`3ц%KQWV1rpyk1גfd (L(oB_?jxSx~= &2#ExxA@2 L 2pAat 8✎!YVܵUߙ3U$g[Xi`y'q XX=T ],zQRF.h#G4.@^<6wLB~_ 2Fְm -Ns=;~dtv8`iaO%du d8l#NB(g leQj m_E ̺
G3h,HQ-|mT㗅Nt[tr}%kL/TIoUj[s:yXҸ jSOD\afِN #a K~GX 杫. kGfC c> oL72H lH5T9MVż^H tL!|{VDƥTKY7`M3[R|$ c
MTn1}:+P/4Z|~z0fHNk51ōު=X~'psʏ%Ӊsca>4!vn8gڀS`ua `ג
{C6~C|
Ov{b:;",t4 r
Md'S|LzqTcXa撌+ͽ zŶTr%ZK]Fبy,$K dkAeeWD JWC W4.#\U oz.}͗siu$Lvw/%Ú>#n- 긋ǟ8{#]!;{z
8S4YJl}Ic
H:3̚klW?zh.lpOj]ߙUs|}YhyX
7]m N{ FhZ6٠`?`r'1?;?,;jفjۘ-^.
;G N k+VE)&"FsԱWDт`
endstream
endobj
211 0 obj
<<
/Length1 1647
/Length2 7087
/Length3 0
/Length 8131
/Filter /FlateDecode
>>
stream
xڍT6L#)
2@Ca#6Vn$ iAJ$|1~sm}]w>0jK``y%Q3pqrpq`30B ` C"l@ Q
(: y"܂"\\ ..Ὲ0@8 0(A "d dr!V@(@
;e: t`V0?!l'NNwww n#p l`0
ltik q
@8pgpX.w.PQRh8ɪ l?psp @/g @m 0@C^` D 8 -J䥴 b8!\8\ ?{nrP E`O[ݓCaPN(g W'N=($sgfF y `g ʖg]O'/ o'
|tpW ` @~g[>? 0ẓ 7fw
]1 럖a ov > ; (o M O!rVZ ¿_5ӟ
a7:N` ?J7粺/?UV$ gM@G Ɲt]wk[R
wW
:/ ޭAB\! `&ae[1z?wk\ ?_ ;7`w fe۟Z@?_ Áw|wxsm$K N( qk`
cS % hv [#,!66?_?o !A%p/x.Sv?Y 8]\ = _]Ϥ\wa:z`4J4Ȯ:[Cw͑ 6
Rٽ୮1+VR]rLg47{MZWϯ_io6cώIRcQJn=q~oڈܮ̐*CͽGxCV
u{^$Ce ёɳD?h_bG{\Ny-tP>4F=#0[z;I|ƻ(9w6釂ɹ?5LWu\z@:q_ƕ0A vHA?{óR5
&jS#]ozR<@ `J6.k Io~(t2'fX`B[.&Oa(HJ2(^%Y
]E΅|y2 Yw}H*[f 2YzJ3C4 N(#&0[3|we=KoyӜo~>5Х~FuV2ngO Q=Z N/?7lFt2EɐaQ 51
E"2Fo A d3kVoHx6W2/[*J%c0ϠI ̪@\g/_$~#*-p.amz 'vQhV-|0?k 7i'ڷe<~ -ی{pꁪX,<@v*o2;>^c #]?RSYz|G.j<1e\$lQ>O*j>R*2,y.2V',HT|Ejd(:OfxƗMT7.KΏ1zG]k2͖fK26ֽ!>( ^!2l" V fs|i&A&f`R ]Z7GK"X_ 'mh;$Th+\v)5+/ +biqӨG
n
Hίy1Zd=I6^?XOW2M+)f̉
z5\V!TG 'SAb?>5EE:7&5q l5榃^hY3 J(hx%bhyvp6F=erV4G6^c폛³THKAwiOTܚ7Uu|PD`@nݤ/-RŽu{e~
/MP H9dA/XM̈́ڻsܭ ^wyBrfcUgW=nٛfFD,i1@[ xY
*HMQGfj@mU Jj
oPɎt}0$ `{)ҞV)^d}v p]Ǐ3/{wj8| Q@Qjׯ_jg#6L tigڎ"M$6 ^ rR.7++eR<60"{k |l3rꞒO( EH~
4 b]P 0*R })}9Oc֫~p=t,9`mPGLR)
>M 8Or֨ud7V RuWыTmV$VU Gb)a OC' oZ<
qۗ-̹=XC ,:BO bw]+!Pwhc왶5vG\۴w]dztM +[2Ti]q:?^Y*h[ .éw2aM rs/|OqT>?%]+OΠeLVk%CC@+h)Jk e /# re{\ n U{Xu^Fy-_>:<#qI(܋{] # J{ 2/̨?oe;!z#_$M ){0@⣰t"s\mlд6 g9 ښ^Dqrw0G: ;\-7JNJ _ ~]?5V~!2Ex
5՞u Q.
%*
tb_es:i◛uzٽNN 3wzX# kŞ"v?G {QڧKk&xrdF L9I
:}Lohżƴ1+3@|B
mjʄŀEUB+f9z%|Vk؋@6(=)!y][&Q#eܑ'|@Jj }K4VL/'FTJho#n\GOW 2(EJDzH=vA_\6VL_%h:,,\䘾tzV|3+b-s,ӹ-/3R{u 0O4{E]+c(ټ-t<={'ڿ`RrzX.4[m| j@TB'D3d Om)ʸ}Vڎ9 hۏQgTe ɈyԊ0cV :9Ew R&!(J ,{5넢uhmdfB
x0dhPӊҞMSu͏Q>Cwc :ZG4HE[ VolBv>SŎO^̯ V\4y/`7ryc7[p. ު}I^fgUH`8+
,ߒ㺆6D n0EkJϵlann
qseӤ@ O.\*6y Y^j%ۛ?G؊(]^`h;)a0A37ɛs/co~3cnRB wCK~tw6Xzf1yC#Wbi I fRG.)4s{6-tTٷۏQL Wd]MW}J[ehF4#\n2k,z)]jQl v. Dl;r$`x_VS ƫjeƬS g v bUYӸ :ykxМ)T2ڧ1} +1hSpC%o"{ٍLSdJ#/%EJ v-k8\LGbzS㠀( _q AIALE(SGƠV"/> "c%,Y-Μ4 s\X[{?E23<{v%! $*6)TȾzG.9-h
){of/oP֚mITDIͳ8;\tYA,n|[
avho)a3CeTC APgd;J(niU[춣z}W- 9ζl8 (&PhWOXJ:e-hu*Fgַ:\䝳Mjp#>n*`
1Lϭp"mjvF
!r\|܃)|ue
\#ټGZ8 f7ҺMݧ`jH@\x |^43vo &$.8|tA(~WcqǶ,| (ƾHA!b, r fTzgPۡFU ͪ3?_kҘΥYD6t! lH 9VS3tnTQ umW;bݲ1ET33xx9 k騭
0mHԨP{OءQI^(+mұiVdTE*3^6Z~ف^l^|J$NC#i_E<0d2yY3qXW 20R.ٞ
=&CO%竰iMT`W}.(4:,RWco#YM
?]>SKeLն+(K {k 8#ڹ$춗$4| c| rb;2Eы_M+S[L9EXG X+d|!K"{4v_] zG)"_x
PV:|BC>b?Kj,`6@Q큞@-X 8ͨÆ%j4 Lffgq&{tQ[p0dצS
Κ[.Die@sиG eyAh[# s&D=)Ӓ}QГrʹoW.% YUZF=k ǞEQ(Of\-,2bo4p@53 C݃m _
endstream
endobj
213 0 obj
<<
/Length1 1752
/Length2 11236
/Length3 0
/Length 12365
/Filter /FlateDecode
>>
stream
xڍT- OpC
hh\KwKp
! ;3Z-j]Nթ]\EIbؘY⊊l VVfVVvTjj
?vTj-!4u~I:?!`
`ge0uY r0J-qpYY;?O 9 p= dn
(:[O47CA@gJA+hlje8Y 1@ 5
ndj5fTj5C bf<@@09lt<PU (; $0 3# b`
; @YJݙ`
M4BM]MAvfτ?J7HL;? wk[C`g($@N@{`k`?݆&lBftppr 54< 8~{r8 , ,P@ ߎFll 3 h ? g} _
<1b UIeZ)&qx1qعXl Σb
be?}h:'S KAssXDD Eo
`QXTALsNsN ɰ7w3_bOEv> y> ,l ;ែxѳ陻89={ )4G T1m
"3!;JNS _*5MȦCI&sekM]7f13OW㽮I(f0arzaTH1;Q-zթ1ŻK*wVg(i 97
hlq]Z܈0F!/~V?4+_5`*KR`DRk)!
LG O]1\ u@<7I&u<F[nN5/;79;EG/ e)5СIyM-֟|UI0cb|Ŧw3g}y$͖,ٛ ̯ RCV{:gzݠV
cmg۟j%ʗ~ b myJvJcgusG~sr rmq=#Q7a8M#CYgB9%Q i吝Rhx_=IЦZ0h?$AsP,NVo_4xmYy^ysaU~7px Y>
.?+L$ܔ"["2d=p-y0<Y צ~\(/ҹw-,m2$gx]%y7'+D'[&ʽo"d*ږw#We qHD \#47{DwDV
ʢs2D`3922F{Yb#*r]} oY~P,*v!MZ%O
,!MW}1|q;[ڎ/x+ aTegA81W yVsy%NOH!V-%oᘉ@oz5enMZ-i9I< ʧF<ޏhk?}T!fnǓ6Ո.>wwi5 Ds7jW[+C/?et *HPx! Z8q(=SKp[}ZұmI۳.o )M/fY/O1 )_I?ݾ~op+|9'Ź_FtVx
QJ,,6h(Vo77]ܒ)
8pMfV,:
T;LJ db d>n2- {(Qٹq"4JkL*^虡zC` !+ u$f?BTndzݡ芰 /
{>\][ëd&aA>ޡ$Dx|yn9b߉ BD{BJE9gaiE\`L ëPu57j<ė x[(}ڜ' 6%*qzInֿTuwCza}VBr>`h+N-Ara
Wkg0ځtғBͭŢOp_Խ$Ȍ4W-FoT@[ g'qG&od#{4S1it 7 ?5* OPvC ~w7wh*ҷ.b!O\eڼ m$RkZ.yb:S& xx3HgLW:oJ rg,9 EN jU܌
[VGY=LHoUv &k68W=|GSlKo$|XNN@ U._I
똘- 8h3qr;;uƟbzu|7Md.&%&nO0)K5pw~įܔ@Є׆{'^nx foUQ?KϢ^cqU5 Ly``_ǕuR
0aLGQ9U.h U:E g)>Do7Qm\D/ EҤ1 >41?fOsԶY. tH Q.L^bF'Z,鵹7hyNkG~!'|f2zo:`ǹ۩(
u1|eTQJjYQ3&C$`hXE R<
D-p4w ڣ[6;ZWyJqnõR} n 7q
k8 7ebɬ Z ^'L4[ $& ; X9Q0_ r߯\nb*m,6g?!Pq W˺g Y?DуQE
&W82z㣸n=g>nq|Zݣzkɦ! ,_%Kxh] ǿ*$!V@gkS ~wetL^ ,9ل"xمlG!NJ|=m(HbިdE{)M2F&mURL)̴BB5#Yy>}u~dѽ-Q}$+k?y `t_Â`n2;S$[Z&c, 'T
-C{fIUoA fzVVGΌI *ѵ/_v'(^4'Ioq$x#(5ó^ҵEBܲtO{*6w'j Ě(Hƻ!3a~~iʆvQ( S1G9,PT!ʬdݒKԙ[à̇ {MvO"B2Â3ku&QH_Lɶl_eFW $nҤ4K0^#ނ2A. !d,HUI4uYfd#f,>[Fu!´8AR&@(ۻ) ba+=Y %=o[bS(nKr#ReknL9چR#ԼnAPymJ{儕e(EH'jb^^]0E*?gEm{4
^;v<)Aba"cHY.
>Z ΅0uhO~]TA& 8=ct+:7칬J8 %JdBI⩡L`Q!YBG} t
F kHI>BkEwy3ɺ2E:U,A%}j5齹ydHk摟)xFf #=h}(
1ƚ1%E|LT>`gdvYr~Is%Dl pRڭoOC
5PztS\ m$CFܶHsvfj' ,.U9͓HϾMuT b[oF0^5seIg;}/%7 <2~2ò3F!ª}Nj^9Nء% t;fo`^sn%2f5qrm<]xe!3Sc'AæW QHF }+~y?~lLqȢ®efp5.MzZy6Gi P5C 7>gZrC57ːqXr@8#ht9gaM8 8IM ߋ~Rz{)Lij밚k'&am{8昗zWJ[R+Nhvn>Y1QVs5 6CM bppO2}X3ڔ瑴2Q Oq"=_Wƻn"XM݂Z 4#
ӭƉJ$X{ C"-Zgl`kXAB܋#c IGy,X
*z_~]nz > ukfurmZ $:W4h6| 5[YmOWTHgGphs6ZY#ŀ T]eqj&yu5 Ǻ(gd?-? MhG)QJͿ
?TT}/~ X9imi"nt{͏@ #j? Lh@ЛxD U 斓 _)Wý '(f WONj(H^KyOoV7~ A9R\&B`ò2gfڶ,շu4_Ϭr87L{y˳-W~|c{Z167G" O+ ]FCЏrfU|`]2+>9ڡԢ!~y3 UphC ^JK
mn(KJdL>c]N|A j9"_L6ۡ`2TP0n=< nPfks
0dM̥bMzIIsz2TB toIĢU6z/'
y%Mu}} { ^':41x.XB&IyRZD}rwS o!@ޖk 9<$2e. kV,u1y[45HAQz_uUrL ՝Q6C[=c!0(0=}]ѵNRFvݬ5w!> /zHpd*v˹:
,~ak|@
-!}^ӷb8rj^5!nyϚ) #Hjזpx |y 5 ag
pdښi5S ?JH ;+dպ?# h}lE{J'p}:efW=RЙ𑂱D|
f
a | a "bP
s0kGxʑ<ёEAwls\!WSN4O_!J 1ϲQvᅑ7RrLq}'nL/4}-d3BahaNۅS$YL#2;=w' 9Cɺ^<(J7hL60yW'g veď6]}f*'Uߓ,~&Q}Gޟ(HS"ʨ#D9nE+ܗI
zTڹ; [$[i0 g,IOlǔT xl#炠SaηaX )gSD[+032_Yr>~Ō9 Wu;E*R=aCb }>ͭYrl2yԆr0GU? sI3&8*y=ۂiI<GC'oc٫0 !5- ur -* 9+J!.a 2BNr ՛k1wD/ kvGX % j^Žv}uK.LE3kQBgGbm߿G NJ|aӉvP6{, ɆNY0S6 ?@1樱<^db )=:nsNyYVjI;БGLB$뱗%%*!^
74> *7ϚU6oE貴+{5Yi
c"oH/ ?%M!:^.^߇#dÉ!x{0~^*SC=_dKեoF~.'ךWG_kj\ -
哹;F ,Ǔ5k5?\4ݯkaI پ,f s)N;)~\R#S²X2FH;'Rbw.!A ~ :dbG[~sPئe/5%Gq1 M i0I/D1MެJ 8" ⸇A1
[ZSav雗P L}ﶶY 6h߄8աuMJ X!@ ;/P|ۜ։;_i b UyuX$Ӏ}7J Ǯ?gЙl"TGO= yF"_u"^o6DdLc0K2 % WkTjrkLЄAH7U6yŮ3ɿ٢J@&@
"/D+{FJia2Z&d9JԾRk4=oJGi.KMɹuqv"|"Q=i25[>D ifX |\M{]ễ6$l!wyLWGrI$ V;g:zU0cЬ\ ,?^^^, XnlB\y_d,P-PȧL^0͇سz_Xj'_h6k79!t2lwnsoq OWc]6zq|UT 9Y9tdVi\hnb'I P ٺ%ZFb9G.)\5C_0p!5f ɋ.K}qo\=1˅yTl`DtC fFu'l74K:d
}g}jZG١3vdjM'p\OmxR;8$œwZ.sJG
e
!ͲG%w
\< Lr5T .9٢Yŋ_f_}t,@&>
stream
xڍtTm5!ݠRC7P
ҝ 0CwwtHJttHI4ȇ>}ϹϾoFZu-NI+DEp iUUEA `32j! !u!p;T? Ү0>'FTaP#
P_@0@ngP(864q=X,Y ! NW;K0
FBh vh, `E =< '? ~ k
߽Pk {s)/w99^ *7-@gnIהM2S;9zW{ @ZUW{'HBm $ҏT i5ɰR? cD٦`4em3.y;B3'"l}D4[>D A[ Xwimbɡ}]~ ?(cYeiɥW &)"fezuZQAL 5F.$;<-R!8WS<:ɿw
C!8BŪA`>+d9E3ႆ:i֕J_LXv< M_
SI(7aZi
U]8{d fLg~dDapsV}9`ҙz896;5Q+(erq?5lo0y&"G < &
ᨚ4msXAw|BSR21?Op8Db` OUsnuaH}De}vȀK= SY*!]XgD ?\ g7\[H8#1_,X4ļӲ ۑ?Q@DuUHg}#o틩
;l5Bo u@?s1jd?P4 X ah \=3 X Oj#{!Vݮ(j i8s](٠|wb=^(BlpKȥ AOG}FRJ&eR83Dzt
L4gV LUn &esf*+ILU?JW,m؛F)
~Lt6-pM@Ӡ/ C5uu+PZapXqY9I6+ITʧ]G|zr6-[#ܣ&~ )p&!ڤNb(Eᕠv(~69x]UMR\QK9Ct&F3=mC Ya]z "WzL__㉟c!) Gd,5!Vm]x?{q/a i_ڥH8還?ncixxR.ºJױ|W%]UXMvy൜[ҬMmnNbfN39}oaV\rwbgh{_*V<N
oG_XWxZx dcn
bWt]Ym{{; 壖Xkݦ pD!?`F~1YB!a퐴6 y;Be9»k7coxvD% +&NJmR ӯ]^7\gc?#
kym Rߴ귯52?IVԣm e5v_џ_;;nkyb2qڑt1n3rA~ʖ %;ĭҝ%VjWSh3whҪҀ̗9$2RQcYB c Ռ _H)J?X\ɞh+8y)Q+%W$V($(gX;T'E0# T1 ?AI:K\nsӝ?yq:i#zaL*ZbLpƖIOf1{QSoMZ#sbz29EEY.!ӶqQom4H5^K`<"(Q|BCf:6}*st chNF1n#uT2 HMXɯ,@u8Ept씃a0w wS nj/:[i=$ɠ\yz,,Ge Pu Wא~g
60a'Zuږ8G ˬ2/iKKVuFp|eq:=rK`X[W?jg&xɫ8>(99"!ٶ]elb }UmZT2HnpG>GM+ ļ;3!jR
Pz
N!uƎyXOYbȖPbY ˻ F5wR;N[yH;k)vâDo- æ19I]ji+x bb}l "0pb? A釃8 4 fCI?'-ȺofPZ,; ;T^A q2tl^?{{N}Eo,抪~|=xW^=3Kgen+}KRb3L #Da.:סM%ؔ-?jgH䜕*P?9VD,-1[%5+9A WͿPm@փ1xy;gzZB+voJT^Y˧![TBecy2Ulr~lr9W;BV(T#8d6B 1^ա_8g 3EG1zrͶ/jUޕ+A#?cͷL ?rDX#8g24^t.i;̯(dF.T5?Esŧ_I%i|/G#4%3U8{:^#ZQcWsZ% .El"! !AٱWbv4 Ĝf6sح0+J|r$s
\ ^%D*mw+ ?D5N)Fԏ&;m?zv8FHڍy`g_#67j"xLb x.$툛a|J)Kω'Kk-)\!"R p 7Ժ>|w/
8FԗKU : hTQ6q_◑.1Tqab@ɵ_ĝ0[녁P /$q#]\rFX˃t>Rҋ Q g7Z4 qW aYaw|ʃL2[nTi(Z˄>3ya#ǞI&!Il iX5ikfː'ǺόI`z$ոYWfdA^T`$A 1:Tr $_T57)2/v~[#xkmd>14! 04cykkm踃D)#\\R1 V3Mae 4pJHRr8!9{-nZŗ]OA%Uc==awg*Aю2U˩/| 4[ݎG?GGsɫ `bz`܌f)N? 8Jc]nMȄ
X }vG86")_rMt߹ 3 Q=!d}#2(XtA~tG *>(#/)h'?)}_ޥN4 J͝IXRolS!A~ Tp%iΔ5s WT;rR U3y*[[#|sy?.Kf !UΉ3
? &- G:%4Ia2=x^Igba~GWm2L&$5"Ijݙ-)#?2v`"əxFb.02 wS;UFV:Wpz<,`n+4Ap_& EDRl15ae
5oѦލSbZbD1$>T
!1vg,.9 i/LR-Vr5D8ⳇnDj"gn H"~?t -ez&e ' =Th_Z~
\khnv^B^; Z
;ti`τ߈@_7]$eV cy'Kɉ^M* yhvԳr:K ~M[v}2J6lunrY\$}c]$ ua) JgliC_,!/dY+4p[i3 t{4 PdO533 [vBŀ6W!IxXl@?65H rTmJ"t+\xuZv2Akq 49wҮ܍(;f3
l^T m: wˁiQi 88qh gy?]Gpny5I a ;H3!ԏ]O.% rV֔Tuj~hyy:iځmh쫯v;DlRQ(ӣ/_
Ӌ?I;M7.Zȿ֙I#JKiy}4M%") ';*i+S %qXo1IǞ1LqY_ rcjÃfg TYouYQ
06)sCUKLx0c~c CLiO. Ue}ÖMd3&8?,g^,m[(*-;9??{T:qf[C 33|ѫ%n^a5}V] W杰 Bv7?ÎGIƺ 6oN}UVu?L1dc.[+~i+Z E\gK};#"đY8K~X"Vb;RE=n_z2qe9,x}z*z97߁9Ƈ9:hyա8@мƛ#h.OܶQ">,LО7$ʓ2ɵsyYx`3\ IJ G2e%ٴ 2:rOΣWؾnG!ƶ N0Frdx`ٍ3~8a
ivêvȧd߰|ǻe1Sh'e^ *D_
gF,;ewD){PQ:|^[aIZ`=7@4
Ɉ l=Ҭp7ha,I7^X,H:
w+@b -DPOu6#ח +[Gt`uJ
zAx[ވ̤NkzөP ASB%0EIrN;!zkHCSR ugsiE^,]jRPnzv F4m{A-IGQ
z'2v Mnȓ}cp^Ӫ:D+҇ L~!@ ưv |fVufto@YdùZىSAC%hÉ N^CNWed?Ҩ|- *Й PE9ʬK LSix#q&c;x
!{ F5BdXnn s${ -!oMUli^5#0T(EX8^ %3 N9@'= ;[E8cCf<4oY%#FT䒃ʏ"xz:l+]^RB|ih[cQ# U|H|(= zq
kZ4QMS!XvJZ2תL:& *2yxX.F 2/3U ҼGLkiH!r wWjx=͈cߩ803[Kгr8OA3i̼v̢XhsWwVg_Y:/51?9HYqோlt"S7"M4&")`\"/VN)I<]49F4Ipg FlD+u!i++FۙQ8B"2InqK䣏ncTw4ŵ s(#i䐡׳ S֔.nMC!g>zrPWqMG9si Dz!㯙C- 8p^>*ﺖoS#/ yC/YWɛD ԏaU tu÷;%.^ ?jxƹ7jJka5֞g FׄUK<
endstream
endobj
217 0 obj
<<
/Length1 1830
/Length2 13907
/Length3 0
/Length 15056
/Filter /FlateDecode
>>
stream
xڍpk. ;s :mcbv4dbcbĶm3u{WS]\kj
Ez![#=#7@DNȌ@Ajd[@rp]&jn'gkv0 ع8̌\ch
5t01 mm@"vfNi@mL` d
r06:3ZTl-@NɎtuue0vdu0㧡Z8A ֠:c@ [8%W5ur5t V wg=9@EJ`X/:߳010' `ja(289 mL04r}7t124z7rCs4vsrdpE aާ,fc"bkm
qrD>Q ݁[WϿ M8l,AR l \, = fl# O% =l M-LA?. 3ߊFLL c'b_ , ڌc0ϓ;Llm1|ҪBju @`d F2+ecj 1O.w,ywւ \1(7ĝTS
-6x'پ6 r gr2|_!3Q
dhdl[eV6 E[G? /jy: )
9Rcfc:8# ;bx2 O 6N.(; (/
8 @ P
JdA~rAGQz4qAoh/ #Z 4|_ | P@̶-S0ޙmRs; v|p\N>% {r $o1si?ϛr#,YW }t盡FC62#`V(iuuGFpkKR׳~^+$DPm?!<fNi {gy}n%+!{J2%Qj:?f)r2pIa >ΠgOK"xFzjm2G?y2;viBޠNQz
&K,x,5g#ӥң2 2WZ(G4V.u0$>`&T7b;*`UG`8U| v
ݪwZ,sMf5O=׆߅7!/@`I\;|u;0i K$J mp!ϴ]Ȩ >J-"ڻ\csm]7-]ͨ3 ϜE #:7];,w=JT?X<,f-v'f[$/d[ p|g^˖D{A?vX?$ ukr}[Z*udp^*< ֡娐*!ngވ엪 uc *XgW)Q
%~#ޯ^KxU~a7~5
`Hk+dÍ UT<6zvHFv=u~5x"S!(IvҮ6#_MÛ\ g@CPd,㧒buBZ̷,mLG&
GuR>.Mp
QPcgi7\%ҏixY$ S`8Yih;4$"vx0jb/1YrI蹫Ѩ6_'>HT2@/ՈHI;QzrIJW#ONvK}L `J*ΖzJUo;~08eIXzd}V>l~Q~W__U.0g pő|?q4Gx4q/ Þ t&m]'T^35}K4cLDaҲϪB'ՒWuV[5o ϸAXz)' X6ɾ8s pȶ N#e㻃LT֔aX= Ñ)i*%,Վ@DB!\*YqF=]J豬>gS d46 q?'3@LahTNz/Gc9>5,Oߑ{K:Nj6eipG&tͲ)w'l`TO\sLuD Ɨn' 5݈9IdM+v>ڞ2# 0wkz W(0(dmZеqrem
o>ʈ߫=}P2\6.Z#i:,Kzy0dUb©rm۫6-;}hn#<{@6O710$E^#BN\J2Ah㖍;WiqK^űh!!ܟ.! Vy{ uTK`˓{]y2W<4Ҵ3Q~`d\6 8;åv[syI&6k"k"1< ל@tf
4gN
]s[AoMbl0 [9YnȏHɪc7*3Õ3%+~s:_)
ʍ 7!ohCxLngS6 fqQn7hSd >)?\.3 CRo ȱl4EE JL8&ϙ:MA\
γUhZ < yS[8ܤ~ HǮ\1"bGy}XJ ⅶ}Sh%S,,mw\ \ZQ\_ vʉO 5 O gnaL25ւ)Y]ы<g50j\ׂNHgc $n^T
:QSɪ*EԡG""B c ǠpzFψcidl̈́~Jr|6R
ɫ'a/t|ý%xKFأkf68㫶
I8
W⚹."sk
\gC?X
%'g0> 6' m<.0
0Wtc :xY D ef[zߒ!)ՆgJ Oe&Ub!*rU `F}]솥D_bs,A±ɟXK 2ڽƢ /eȲV(BOYXPYЪĭV?gA3MG%̳pXrEO@˂\u'~C%Ja$7~ :I1,i\.`:$_ڬ!w(<8 C"$G@cadaDUM*~k^%kBOn~_`7b҃ID _ׇ4 Ub仮f
n ݡ,QdHu{{uyncMĺjFѶiq Mg0u^[.>8 B{Acͯ ֥
-
rz20 ]9 j01*
}jH)cR0gS~wF4lWo6>x~*H t؎^vr( Lƀ `PWDc{sرDeu@O9S<ϭ"y#`ڕ,[R jm SWffcѱu9S ךJ&! ?A ݾ
|fe-LK_L랴d|SutW ?J3Pp*iRT.Az) dvz ~pKui~q-b?QH&{I?Ip|Z$#8>\A&^:F-lnC_FI#gwQWD #[Myc @ lW D]Bv L`STl_ 5| n ُRET$Z$rBOK>[䷤07ؕk [~mRJii=C!|g@uHh[~|7t8L xjNG;HB|HDT.ceN_ ,R7L%!# 7hb+bQK#_+MNq@
o3 {taiϲu2 l;Pr'yP)0,u啺.%gǮ'T . ȿ!ª|^c-i5Ԙog5mm4YaXUS'
q~C ՒoÚCDR{ޞM 4,:z}b3Ow
T _?5HBEm ]
Q85`{X] b;')Foz RE,3ea] Sѥō4b(j4f#Xܛ&d͕Xm͂2 1;^J_!K Z ^}su*~%sm?zXbUkM3tVk .Oq P`F:,҅ ^ñ**^F]zd/'cc3=+iu&
X^0ج%ߺ*r ] UE;ߗQkvzԶgTB]
SvH )OPޫQ[B#d"% @o gD} ϐa4/CQ1V!T"A؇ݷS;9=*vƼ8)эSIKsWcF|]L.nA7mH8l; LWyg \ohh7ly仝T!ţ1- CLj7/lzU@R;ۂ
r|AC!=
ĊUF
V
mƀ6~2kb_E:yAUk cg h%eӭNǖ
6~Yc$] (@UҲ^3W6j3
m#` $a\Kmx8;a'/!,[ݽ8
y[4 hKu ? +@!k(9|=;1`m{Er+9 B qF
,&iՕB@!>mPqLrǬfc[FOy/e'F >} Ta/W^KUԯG]B x3р=;vO7*EX_GP Fn$FIgؓ= c )XMX]c׆ !0 0G;6;zs+4$
K\t~w|SaEG=@Cw-"hՋͭ<=_kX3 o:/n9rXNva,U# >gܫ<->~3=ylxZ~@r$\ ZiHKsIQj;V4
@X@dl ~a@EJ h~sԕ1tVQ@ylWF,j)'&~>:b?#~Wh~֮9ɍW.5 Ay#Fd:/XL4jROC8X %:|4@ @G9PNQh
@&%بmGcgۼ8+iE ICFScgj箯hʣ 5 ^`X6x$v,h,xDJ+N"Ept$ C>d7Z߽ e<%ޖa="8NTp{@LӚF8\®]a9rPکQP^z1 M7:K^lk6r^.)o
tUlxe}vӮI7a_bOrf7 &eHbr}?: ;tlm
6i@NF稂{ c_F%mo 8m> LkV`4D5C{:J*{*y
~=2!g!G,Z)]*pG֤Q
Ii
&1lˊ.o E0Ma9j R7םLָ7EڧF痣\O=18s8$E9c^m%Icby33ԑ,BbOӨϟt
SFK($'Z|hYyf7Zh kgl@ܻ> `Ẻ4b .Qj~+l G쿫
5 ڑWws Ǵ=NiJ4j"CZ}V tG5")e5{%Lf-!cӟ G"1|MsĒV]n6[i !+tP5ZG"_
HxUH`NGm*EDTnT_=ۢ@j-u*33?Cg5nK0 ؒ_h>HDt#[UK^qlLpIBIPWM=$`ڹja&N6hXrG9^I>eS v{c]K}p."6^7Z)G"jlVO-ܳ"]0\Vl{yi^Q#9|M#.+u1 >ޑz2|7>=UK 'K#&( >EK3Qb /ɚG2?0oxήQC,&E*i"gߝY< 0W$qE*
~=WmJX5/_T!X}C>K؟<~ewH3 8'$-ȾPrT\8)ZوTe(2 yxL-g3Q16ֲq.i1d9PFX!ZO+O;%T kB ԛ|Fm覺ey@b4h!e jhW#ѨA ޅfK7~Ǯ <; |(XS#t̔R7ㆲFO88ۯ&\F.ۦm(:;\t "Ozvz]? MQHQ1#Tu"YG;k瘻ё\l Sܠ5 vߤלU"X[Ĕ>fP`miv9}VKh"4'2$ieh->}GMUe j#+ biA6`dži/1U@G璞6@jX%KrrɗAT
°pehF" c)@KZa(]+G}Jw۵ʡ^(x8ӱַc@,*/7ǮG |S] ~q7KcP"uœ8ǰ}ҊM
_獑_b>@igW^k 8MҍeKWh3N+ R>`H"!=^gHs
,O ] F V֧ ̬C m5h0n+zN("yQ5f_ҧ20\L#ܚ&=?(x|Fad NGW4R +/bxiЇ<` ygy@"7Z9RЀ6Bx1c=rGoߤ9?Eڸ'aUAOl| Eab ^AKbNJ'skH׀?C{R%L)doP.쯐$WGPDf""b;oɔFA#&E|m-y0 Mʩ2kc:{B1!kPpSBUŶ3Iz L+/ ʾW˸Zhl{⻈ӀCDLHe+k@z}mytD+ިth*x KL[:7! '0ƷFx"/ m2ԚL[48\[ "}xb}oL`)A%:ZיpkKL}W-jEK: ә f[ޛ>?;q^#!g1md] f@J* 7><7$}.@s>D}a1 A`Dqu2
wm7B㡪G763R{S6TiU~~ϒyͦ4K|DXL~uRs-ݡ31Ժ:y]! t&-oCZI^|z.gM0k38g fxTQܷHGzXp*N+T9njZfmg`IGJ*h98 QScsz{㚎l4UZ=z| fc ~QQ>ָm/܃ rHh ryta{ )O:2عJتUW`2_}vweEog <&Hղ9Y!}^D"JR}qUY%^ /P*ݣ]'q,uSCϓZA.0{9O
+y l(U0iI;IZB.G aܦ%YRs KPv̶B,,r08&R'-\/zNy#U lxD.~[CG(:5ν0c G + C
2NcH!
Td@t~F$ς1:ێC bf t@r鋄C9i!`6 [&̉I߀-C"rB/(ވbxPCnO1wN lL+l)? aBf{̓HI3Xش6W`H4Au};fΆ_ͽ' ҥmvgC;;0<=&RBϸJ]EλRŇ 蛷*gq߯+'}r7ESZ_11s-?GeبqRIs$RvCt +hT%& TV ']e ~5&835
5: 14 i:\,*eϓ'_4>۱0@YKxR{z <$[eA{턨AKEtxT,IDz?" ;,RTsM[гUE.ii/ٌm-kHr?ob5BE#sx[TϜE{^X;[}x te_%DHةPcq5wq\ |ehJO{TDh0<4G\֔G`$uq ^ӥ8iu5 i 5+}aVm@OyLYM{ P, Ŵx=}tzzmuA `+`9:Gr3ỴA[ev55h^4Xl9&~Y8_(@ieyaUL! sa%#q9:>
f )yWWh[p־?|6}A " Mo
| ֏Mq#X' HUxMEwy妇ԘJ)~hKOɏS̄^ު=$o3P3LDk2U8ѧCS$fYst짎,.2Ni7'ml>!YQLaJ/\&,Džy5C&^?\T71OsTgWh e}=BJ"|n#WD*4Mn=oW} ,W'DVtK\;#I]0D JUǗƴ ݸxeC8b q6(V2>7|
WjCoQHm 4 bFx\{Sr]vOӳKTle` {J)Zz^
OBTG;V|;&&',/8^4vU
endstream
endobj
219 0 obj
<<
/Length1 2581
/Length2 19500
/Length3 0
/Length 20986
/Filter /FlateDecode
>>
stream
xڌP \'Ӹqww NpMpww ngIޢ>k9MA b`twe`ad)Y)(ԭ\mSh] y2sdƮ ;{-
`ef3/@ uS98z9[YX+ڔE ;=@hhjl Ps0z5#/
=
t: (Ɍni\ lL.nf@g 9@MF X zk`ad/ݿO&acSS;Gc{/+{ -$)J07mhl :onlekl2W I1(blbe;E4*K؛9]]'n 4;x[ٛN͑I
(#o
`ffa @OSK^)Y~A8:8AI ́>.@o"x+hae $Aw1f @e`o_eҐPWג'DE <> l V 77,VQ{s ?
@C Pq}ffSIבmo@nRSK6hf\A kggfVnvW+j { Ehljjϰ#dV@e߷
@ej9\@/8RrpAa_3 `bwp ;8('I ` Lb7I0Iq1$
` L2;I"b@(AX P,JEܠX w?]yW@5 w?]뿈TbG
z[b4 {j ? <
i_,3Amr[=/"#^@,@ 2Ҳ :b/XA:UTP @6/{7;߷V*8
zR Aov #2tGvP m\I r wo*4 (? r]@݂*bkb(?@W83YXٚpP qhAx:y '^7 Ws39 tm 4_w0 v_#7.0CN BS|+uuGFx礵6-IC^;$DH ! f
d+x,E7r>Gg@=jN9ihy&s80 h.EoI'}%IaϾXj5qWJzSPEy̩>ƬӦ
WL''ϩD(سz^ 2]l$AmUtN}sXfR{:hi` !x
EccaM[J8 'lB[V|hVFeu{'4Շ ;eeo"ՕY ab !·ܰ&x·d#b_?_kU^u^003Y *&~Q?ɋ7ix*F S>8ͧ/ X6/A`5aIj+vYQX7+\|^ߕo\[v!jW X4kxˌ͘-|`X=FsR ?bYp C?H[FO]M uCc9~bu 6Tw/h(ߠfSga"Lcڧ$_4 W7GtQ $g!SV?,ygVӷseiB
-)1 )NƓq'|%n6
P|B]VH3ds:SѪ bg4^!tJLv~YB:H;~*t έE.|c=*,)Rbm -tfkGHBC_E wˌ`nx28RB;e=y{
W'uT&,x'\l5 1I h͉7}n%~I v C#>-G
(
~ɡ2^ ^A'u q|ѢqrN9=>*a3=eUMēv|dՂXG:P
OGLc3TĶ8Fab]p#L+ c")FHFn MY
S$xdk)%>7ӱC%[IӖzpK9Dĺ2m)Rh ]-g P~3?% `.3Wljp~ g× iiq\2wxAix/_)҅0٩ 1
}rޡLq X'i<{qB`æƼxZVKEmϘ- $Qn%Jڇ=b1[CCQ&' ᶩ
:pmk1lnoBs)φ2&~vj2!if4eivʙ@=FzI1ՐPH(c=HdE5I=Jhx ߹;;
A wGna5:
QnZFBx(*܄<,_!pke}ALغqND7~C+zz,aPO5"W鵑iN8`&g&v ;{KlC,'w_3NY]a'т[ e+)jci-[
',z.5 .q6NkTܓIU#Zu&R/}( z="$l.@ZՂQDhײa ũhɲKFƕAވ,ApV!ϪVot䮈6b%5NZ K)/!NtֶDqBlEKK^IiZʀn)#JlŻFUe4=\>q>`ed>hMn xEke #663 l\;L} 8qA/}Yk>ܼL2_raa~4rF
Aᆧ#Օ:<,ZD4Ttyt@69J)vi+:DhY$0 D1aV,W:/Cݙd|J5JN j2EA@[IU`4DU]{Lx[ F =XũMoAfo6~74Ȟ:l)6aց뇰dV 28Z! pOx.C_zfǺ}͛IVJ< &jm.[
$%+c&ĭW+ Zx,",a Ar=Jb X"[< gZcОA
o_$jz]Qs~ĊMNL*("" Hj$5Н$:ljWfg{ק3|9lqGr>Jq#5U& (ZiM=/ru7i#jn Ez; 6a9b$!\RLx-/]c,6!XFU Qd
轠4h
_V
B)*|є- Cg>!< (*Y RV!wY i |>|@<+=&H=1S훩>ڰ)SmNQً ;jzPlȟ1QEp_h. ,+j }BZ k+Mn@ o1
a>v8c C49r K= F A!-+T6q6";*3+1 .#~'B"bR膭4aKv ft{
6Ev6fT^ss ō𢿝{ڏ!ЏHH<ꎺqCF!Zq966ȗs:v#5c ШY
-م9
z1Cy 9x_e6vSF: *Ì&fh_3{egQDu0w,GܹAG>q*B/˥ S Ї r-V$+|s&S/$]y]ygO܀"N5( uo
v Fb&u;oХZ%4d 5OtyY>|.v:7"0;20; n m
X]ނ+ u]&|)}kysΥq\#TŽ4ד_8μb
s\ E !dn*
ĿDsZol4i5v9Fut;`YXǹ;gxp+ӫv3dV#g>.0:r-F1F)V"V`%^$-pa~JuEۓU0M^
-E_3&+Mbf %(soij: ιrWꦥ_Sn̟7j Bm2 2_cgccE
%mLoveC4#@橠]g _?ܻ[
6e%ǔ*P[S(W}l.(e" 4%9;|j-'H#9jtp,c,2"5:vMǭS=)Ө`aP9itI--!eQoqcXs=uP MwgsT)uxB y)3m=.mlXE2( B qUG [X%Qy*WNϐ-"Ȓ$W2A B(iꭨm=)nIrI$3** [`F] rLb䠖zĜB5'\ ;):SO[
\!˱ 7
Μpfc=x9m1gFWHIF8_oY,(S+@@2]uץ8 JnC6`Qٟ4 f{x*{kXNg Z C7-L?eНmG^丄i9o
s ~sz/4- Y@B8 )E.Ϝ8XPZ{sm+|ڗoaM`-WpQðϚ8 :F
hLȯԪ,GFk7^ \XtG;Sg3!Q4#Z u#噹?c"
QG$+,$[k7tُ}~ׄqLzq>T-L&inCU8f$&(r|
k0Q/4R`$O,Va^W asZ|2{`=u+2 S V
=XhOsjG3bCDfsm~\E z/oOCP!BK
xr ` q7BѷWZ\sa2SrZ p5 %S)m? oJ-7)Oa ) ~ s\ 6'/dG [h(ϤY"e3ے XDUIS~QZꊻk E~ur~WYH[s?MD=|.W7 (N}'SVjy ̧&܆/nþ>Oi,_G ߰5bJնhSrm CA}&yQX:[KgŕG L1e I`-I█Yo½l`.&]2$bq AFOTyw\ł~B 0cה@+aZ%ڢi۵5^&3"9{R'~o[蒘e
F`7[IVSi3fsx, 7w֟rmH>' [;j 'ke G+9JS gϸh:5L@fw V?E, ɢ ܚ`=%|+h1kJv 60=~l z?yR]X01zqA=TG?? C0 ~;ˇm TtIGu pV?UY@ BVY#ňdaq,b}(>:ɺl*&fR3~ P#җ;v}b6/f1QO'\i Vg咝qcP10iVopzs;h6:`_KL
]
K|| ډ/e6̎)s (h} dl W k/죒R 8)%MlA
D@N'[olJ"mZ?n%)YnRx@}XV6h :-t˺2KO -r s!\\8eF2C|6ZC<ٷ@khsՓgBYvS3Ζ5G"O( ]E[ Inz9G:
)ͦybjfyݸWR?S*2]Kϱb6cN-
^־\mok Xmc 7ʦG 2\=1UAZm
Nalr/ڙG=z&:cFC Rcgm]4gE]$j|8G0.n>G
s ~Ueyu YxfٸG{!J
, }4QOa}Cj FLdaA,X^a } %?)Wu>NeD;[ܘ
7nJ=Q\E:*i,ö|lhh;?d/ +~ϵf5DDq0|HrM?!C hO$W$9dmעEgceShz92F5;FCXw:zj:J)=T2O j}%y4p,QX> {/x{k NPY0'|[w+!Ge_aϭvV [^m@WvqNL 9
!w`$FT[DM[l͝n;~` S
WGItVvї>N
+L=GL~sAoQ}ċc$=
+URdkoj # nV>m8gй7rXw/+$Ve,f!9pȕ
["dT !<%$_I
?3R@AzV^B LΥP(7٩nX B҉%g|i)
6im ]R][ ,"d?{b!̕8 $w" 0p((FwӫǏΗ.zw*[`NVLJҖKH g#4~ %!,i`RiX)Q WfC e\V86Wi1I lL>Ϋ,OakdT O v*Nq>J!` (tv#s6,i=PP5mE2p@+xBa{ɗtvNԴ %;rZT;[Ϟ]:L 8FHv(K)W^Sx#hEyb2l|dkW@R Xbi >ʋgI7 7N.8CcE\v& ͚73[R2%yU2К ~bav
U Iv =ᙕhQQ5FFk@ uo2Ldg j7${i J͉~.Uc[G)KC CU(.?shZGT0\w=?mAG?; ѝ_#A,':-B^roV$yљ(4a8 4nkNN0&Ƭ)/[3}fn'vAG ~0%r9i.p,赉dMJゑ
uGXDn^X%:oR7L ϶X|rC`Ә4
7MZStCs8W6u2Q2@] Y]RRSZ2pUR|o<c(%)& f9xH5
/:"9Fvѓ&,M l^ 4#B^bL8.[Z*
*A3w h{ R $RoY ÔnF (]O2}dy Ԛ2β[vdlWk4b:Xgθ6hn̞8}*m$ࢠá܌|
dd-'IzUNS֞*O nïO}jDOhM%5dD$#C}g("K:Ph5#kcn-U%m/L\ PG ?z=u;'P: ;Qq0Zr ?qFxڳL e{L�amS5x 3> D0Pb'ʓzR4dڵer;*qdE\$Cm>@V^x՞}*IZ qpf&^18Hm /\tWҗƶq0@N #V>8۵X8 /
ngtD=HKǤ"]ysPGط/w
ږOÚbdTmvm\[ w]]| jErnvnzTg_ \W9muuEYO.D7A~R?*~5)Y !Vb85om. 53Un n8 YC~
faj1j NkZd3ڌ B+:iG' ӘZP6 ibn+=sHO-06=#7՞U6ı:b*cj߿܋j);p
' D4
@n;Q5#EM:B XĜ3eWk
mx+nX~[cwln{^[
/W)qs~-49ǦΦ?*T w :91W g `rp/B+-CvMO IDٱ4V5 LU(|}{`AOlб0JǍz"ojkp;\I&,B}K¢R#TϖY6bU@-Yh5M? >Ǵsi2hK 4.ź$ec?M.џC Cp@9OgS 86JoJB*A 6A396|KLKMeFpw"sFY(
{פyru~Q}i=0z6mۘEv)Sr:N3oMJC3X7u& F3;S{-?'= WW2,vɫmϰ[q9kU=[ӑ{CUaRGRq9b msgMM`^SM2oӒ
[-D;С~mw_le1y:͛Ak> M! &V " YW &*7nGٌ uUӪ%M @/Jf EF *\2LItCхpDS9s -`]7F4ece &_*OAmc_إ|uXC?zDd],YnMѰoCE_ b!k%fٟw@hJ=JXIXuktRtq盱 qۼ1='k)^dTSv((ӫO5;, \qsWaw ;J& ^\c',ϑSQ<ꂌ~};q GP #w U٫]T_`2m롇6&xlPL C2ʜH]p6UOׅm6$Fcp@Fw"Ex-9a$^ForHhgO- כ7ܓN$ ʂ .y3cʬ3@p^LJ:dAƱ~Oz8|ڀvRw+Iq Gݣץ"HX\Vʩ j|5G)Gqp8 ˥MG MWNhκpHV) Zop?m rh[b3ӕ{AwM]*LAc(bYFѲ)4ɥۅ\3wH$)5 ' Kd}^u82 9Xz5|^qK`uŀs8`_ ,Ri̍tM_*|K_xNS&DF뼌u[53[Aןyd4aGihe Er%BOs)~[1*ڃ
rB1'hKpa6$!7 >V
TV Ϥ*R:cXvx{[πi&Q ή!ZɜjIY _[XvgLM ;۠-AR=ޙ]a} b7 x9|ԓs'Eӈ<\O|!Nyj\g6 )N2m%1T!2!R]/K\'һEdUgaBJJ]QQa&[cA,m*94;C)c/Lmn nc_JqFQ4˭bC(rkCfmڋ q$~Hw>n=#Xm[8)IS"he \;{/L˘t.9zOsJNб$J sǟZ,LƺaNN@Mt TxI`L%Y:--C՝
S!IW-Ļ83ud$D_;6V'毝 e~ܣ3. ɺyE];}־xS
蕵=3UV[3 SQ&rZX p%f##3M'Րf6*VC+͈EFثl2&"fY1ࢽ7bXXnHW/b( J I-D)=?wi2YՆw1Mq?-HEңLf~).T-Ys
IqAuy(Ռj-@=۔c*Cְ IuiGl˺t2^+c!xC Y5 9F1VXO'彎H{V1:irږ;F-07omDG,ZL(R0
pEN Mw˰dIqPB۶mӝ}C犤²pAF R$3` #iX/{ST5hz?Xu Q MvWA/c7&P y}n0[[Dg$q\ 4-_ #
@
{!A/ ސ0ɰ s!uL]n5Fq|)y EZ^G[Gx%-*? t8M8(DMlG8 lX]
P[dK?SE w]^BR?@StR2cm+jr?T
;$zSe|ؓh2y ΐuq /IէHl[kaEA_J's+dx /-
{3UXvk3'
z`kka$S0882O3ɱpv*ĉw
8%Ğdcb^Y7iN- .XM?I TI-uʨx/yS j04zcXE\>5[f}1Y¤%L^A&,h `z.Nԧ1 PJד uS?6l57ˀẏ@SrX+93YP^KprJnuؚyj{Mp:;/8tW9lc;g;H
Aj
eȥF'n;߶f ԏ'W"_8W]{F+uoBk,(]
,qb7$tLo#|%:Au_?4mI9Smkk4TxG]H!0O,rܗט1ru: ycu,Yw_J&q_|7<=Mz?_u:u{GDhem/ %[?7Ow[SnYJʹ/a%YL+aV|8\WS*O)Y BVJ WSp*6"gv]̒8ȧe|ҧ>,uR)EyLܷD
l7݇+2Nm "1 Sli!;W̜iWz!ulȫ"?Njb!E1հ=pč?9fԞg AٖdZ<%^'Vc4մ:ˁ?X`8
T
EAƸ._Ԁk츖0 в/pQU6[Ӵ ]*a*s{X@xI-Z ;-# 5}* ec7,X:q+$bV{ʦ$36OlQ~ܓ If@>暎܊} E広'Uo͂F[ T\zyCꈜ\13g+_ɫzb0d)M;Ln;{$^L (:Gt6@i}h D/,ɻ`Iz*2>F ,kv(tׇ}Ĭsܹ 2,N)IW L)iC?+}GĀs9*YKE!jcq>OLx(ҲJܼ9, bUD> L
b|ǡ佔7*XM4G܊ _إvo3_hx9n%68.u^+W'G'W?D3W 9'Tt=cas3E%=lȌ=<,)F7; r,!9+eX3;> Fw^SM @_+B:Ԭ& jF&WdMipk2ֶD\SV#d']jӀ~s rip/ kj^wҞ"vԙdE"
\b94}xˑtcyr^Y%BބMd]awBAѲf3:E0)?J/廻1fĝbli'4'Dxh/B7~YzbWC'ȾH[B,:K8=Z?=u
7 |3&rgr)|t4 "-j.JZgus)$0mG)u6M5<~|e߄ ͇ȩ-y@* E]O
̤ eW`N
yɑ)`HI w?H8:.oU3ԭ~.B7=?HY] &P߽e8:|`~SݸZz`ENWU49KuTB̭;F6t9~b; #h_^Sta`=TcY
h|s*@GJ*V4CK,T x gp*?B[V RU~v]{Y3 /
+c3u1[=>4FN]WYO)f2F (b66}? +/.4,?4ِ^j.:}JJZ+M wГXy;r%ȴ2TYOSvT 7JcZ9.>̔}AĺE n &x"Ƽd2 ݶCXN-߁%q/Cz[
T& {^ mҡ4XӹAH$2CY£HNl'=6C)(7 -Ӈ HXK )кFw_o!ܮWܷ v*B49RE~63x'Ћ;ʭHgY94Mt ~* |ul\yM& nZ|_^湇%.Vo&G`L ^.r5gmAY_[}j3rV@^
9iLQZLw
}| $Q1Jx+`ܘh7PEQwbMbU JoU=
vSdc}EW.;(敧Lh]J;(gӱW'&U6dQi ډ'a4_Eǧ
p Si|YPr + ŌD6!G`(G Й#ŗCEZXL &_A6*9 qʺ`8D&
\
d_uZ
0N LZ|+ǽ$9{)^rT} .86 4j6%Voíǁ64F=
^_sdM8RP~QxgJMpM3o ONZND ` o QmV:Xv%9l 7^֗x) X\O 'aMujOe'n 3༞/KR\
ɠF e4%gKwsiȞ\g,ѴȜ &Ѣݸ.^ .M-_MsdġiSA?|ra4뻜_"&,eV_ÿ?Iy_sCxi7K>7Jͯۘp5AM_p1 ]`[&IluBƞ\ M}":)bQ e.z4 dcI4JvFݲ滤SSx2:Qie, ahv,=9ZXsn;4o)/&2F
;SԺmc.<]2$連19yN[mO+ˀ M@R>4кf . Qh
_CF-3ѪSm$pdвP{!Vm;o}&fGoJ. `oSuJEcKLV^>a5*ٞ塢}@}FZPt ?7)VBe!Am eC4I=x"nU6P쌒NjPA3v}>v*t]AJTZ+upo$]DR? ? 솪8@/U{ 㔎R{YߋwADpĿr>I& Dٿ YIա? ]u҇/`
k^C%$P}>
stream
xڍTk6LwHP]9 "ݡt ҍ yyk}z֚kkh59`y8̍( Q @
ˌŠvqaEq l * Pryτ@ ( "E XT9JpAs{X? f+ p#bTAnv`LJ@P6
vW
fQ;77'a..OOON+'V qh]. `kj G8 :v?vm' x0@!V`C;xX PwU
tEJYY @0o`*n^n u?ă<@(r@^J zh\\ Nn~ye9 ssU,l\Nl 0k_MX;q `Eٿ(&l`7 ?`/+;_uܿ 8 6M}!6+ psq`
rXm!0?6@ qL e
A>_.5Y ]5? 퓖{|8xy <@ ?@O, _U UB}إ35,R? `G&@~ W yw(9B 4U3`kz@s WyZfeG,캿
5ஐ_
dY9< ? οYM3 |??a^5 B l.XNTe \:#! $q?h _e_!LῊrwqyz?]{gV"!!WRO<9GŦSY8|]'Td\H%
t,m1K.|kEo~r<^kbknw7 *LJ //_: 7!~Tbuv'V)Y RLdnI`4Ce 9 +w/){x6,W>Fk'`MV"S.s'N)t?kNl;r'Z8me/2_C
:Ś 5)VHF _^ (MB,p+MQ `1 e{iϽln/INڐooUcJ"Ia!? uQ#M 9KE9Ik +< ѫYy Ս1bH['chu*
S.}m ƟqID\_i \$w1n^zmMQ)z&;&f
~7#HpHyz _Э6UT~ |̮S3Ccهa9cBf ZB,%?C(;c\ԲFވ~wTbZ\e.NU!2ᔳ?=pAvh=\ӎMK<j'iӤ #4ve+Z:G*nV~"GjbCݦw n <{ޔ yyzoҋ}W&'Jr0_cC| WO=ڊ tT*Z1L>9OqyH:Ar6GQY3+c >fI$4q
xrk8mYX/ w-=:a;8g ye-A4%VB4Lr3QWOڱDBtHUy 1 zYla 5!?`9Ec)B} m'7{B#b+3W4 Ȋdn w[Rm w3%qԕu2Q=1w/0u&[86iU$0yhaNHkgsgtT! ~|A@KRgNJofQ~6tV9!|;oR+^BB'+^S
K: ʟ45|N
u bȊqf:N6[.ƞz$Jzgí[IkQ9c6ݽC2 S+VR Vt;\jQ.JrBi#u6,UȺ26r5@>wuUQ#ZNv-hx{ 7W?]̿ԕZI+f 9`9؞]Bh=Zp`P@V4; Oय़^t9NwRI#Rl[N=)خ\H}\yd٢w<4
Z?W04 #tOͤ}6^Y?i% d@[c2́njC99Q`PG
*Y+L{3 "n~Fo/;.E qS0yqt-XL'i 0kur,X=T@ue+?bEB>Ʉ 5~
A AMWwW[={a5XimBUo"]u o CMD?F,hvo2ӂN[d}лy\hKE| "{S6|fbsaj-~ynf(`BB(PLE#eV Z:pzٚi@ ԟf 2x 7*~*J7&mp?{eeɸ5G2A3 J)@B} ; ?^`1ՔF@NiG8jҝ^ 9r z-)a]Xt3Z{CnQC
J $9uSKCSRYڴEF<|71 FiӄrjԗBkgf]bokLĉ]H'er`\2
f&饕ׄ,Wnʟ
ؐS{YXQo%*CRz< Fv _%J,
OBP^v!U+=(?8ݯ_-6LFE|G(doU9I
Icu]>{kYmuHgC)pF}JZ@%miIOV59q3MyQu$ :O %pF̹qNٝV #~\>wdTCǺ/ĥ˃ =WQ߯ 2lɢ~ʃB3"3ҩnv C.HĘG=3}b@ k@T➾H[=o͛0z# 9`H&.gzڵ}fk*V MR F'6nL-xMFiM'Y*Z^TW5~}e+~qf}O`Tb>F#geŲLn7) Q@
T Ŋ&#$`\wv g#XStl [Du)kD#^Z'!3]
7ESv(5wEXuG!?f>eznxjr ?(dg!'v{7Ԛ]AU8KvaVj: ~z 9tq|j>ZWSϞW%Q$xPoWi)~vw?}P69/xƍTEᱳ6NĎ [*[ o^g%= Ѿ8-g*& 誥JK]N9T cZٻ m=aPQtà2ľH]բPf]sE3JT/`۠MAPOF_uc[B/GQ:Ž Opz'nسY,E@SL1Ղ!hց{X+Kvi qflPDKn1kL},Y<4L{Լ+FRݑ5g4;3CoN"E>,V{Y Q6?!C汚(-L+ M*F͝x]V\°_foo՞4tՋJR.< SH}n܊`}zFh"D422%kQiS&_x]BϬJ wKP堠>P #>&<]8pX^d2(Q1 -AӫM7рm( Sޫ < Pp.=\ì>$. ꈌ+ b' "9Hq\sS8=Tz,UKbhyrAΫFn+0 2ڨ%i1dwÅQa?je7ݪLk+Y0ɘdos2^1|z^}B0EoȀ&JTs\LEdgMPyݐ=o̎ZE y/XS?Pc-W
gZ6 ,zᑻbv/}E4ʐ[~/ KWYI\]^ud#rS DŽ IیjEvԯ`O}"1 Ar8/[CdP,FO:ع=[C
EkЁ_v3#7g cse~ {d[SɔTTطi\<&Ao
Α OP{ B.t +6A{e+7{eI̞%ipR~K Y D/F VG?87@agzZ'д%q}B^FEI)=h>h9?ӊ#Jw`g/xҬTjU;dm%bgDLIF/4HJ0Pr{MkB]mg W<={o lrhƟ_a͌DL2܆j>tƋ~1x,fV;\(dh!HMځeI>j\)yĬϛa-L$|!2
ĩ:z! b
ŠLբD
J]`R~OSY|BEʐ
͏jQ4
XVCT=Oᱲ|ʂ"LucEwi$a}K/:uwXbwFϼ6>|mX o+ 4*Vj#ՓkQzT<>r2ȱFkx}`gbjAJC=
Pə?δ*A~?]J}ԩ5}̟.cȶ5 h挴hQndH,CDEU1 /B+i^Vg4 9WtrIOgFv,'F@kJ:I"m -*"Y\פ w_#ic m3Qp
5m q|._3LDK 2(]c4)`l=: Ȫ9)!'Z!eL/\.,E8(Gb; q;o̩Õw`8zݸKJR0MA>ы_U{=WOD0St HCj5a8) W~) 'k3OxV6e =4PoQULzw]8MpЦT9 Fx$T;7Go;AD,}@YH8 ~_eQ Aom\UH^Rse@g%:Q_+sgW66) wA+2>xZ^kАz]JVFIpy4%mi+Z)-f{XX/)jFRY)
0WōD;;zI.i f˓A @ϓ g KW?L#86aXAA2(~?t}q~1Q+90Z2 ] ӝ! Hq"j"H?h l:DIiSU,-,QXdD Cb6M"\xun&cox3PF҃({j^:I`M"P
V[̃2Q҂)-
4g}=F|MX
7%gf:!]Ï]LL| ᣠO
;7mRl^>*VXCif@^P al։ǭ[y7(PsgKJ7Ϧ~r#m$۩y*5 tLlT!~ N㞅g3'ѺYe bW܄{ iy| DU ,,8y %
endstream
endobj
223 0 obj
<<
/Length1 1357
/Length2 5946
/Length3 0
/Length 6879
/Filter /FlateDecode
>>
stream
xڍVT۲I HG~)JOBޫT)B PHQ MQK'"MJQ
HE=w{kV|3g|^>oj!qA`8ID 46W )D$,ly#XAX?$pM,#ڴ`81
{P *UP@ iD_ @R kb|Hw -( UTT X$ a8x"
X`H. )Dxp>J`p`` 'J H`C`v
1)0`caX@4x# 1 ij }#菃 P)+;cP>0t0!U #)\N]9¼0xX
s!:. 訛0bu"}p~R~H_ !^6UB!8?ЯXxcBc?k7$W >`+4 rDA |D+e7e& naH7 ?w (pEq hF'E!DAȯϿWDnb .X@Hw440A ^RZ P4@\3) Wcn@O[WM_/ae!2Mp{ Ni;c, oߨ/0;/W j#Wc+ >FԀ:ߗA!\M8 [70!=($*#7 Gj_꒖`X,,D 1q'D"~3 K18b@l/ p`A)/# X,QVO<_F p~˛*y% J'M Tiϳfi=L Ev&bRC͇AoWhx%-ՖB}CEzד_b0g
"M[4[z.oH{T^U}dpQNJ$ V0K)A8(lL!nVhȧ~\B\v|;,.5V
8>K
Fe>8膏(\ F&_zM~~}Z m]/ZB7b2ұD
2#NN
_KK'
{ 0<ϭs1 {{2_e_w+| ߌ^U8
:#--p Yږ2cU# ѸH8Wᢐpr,spp^- Q:Ly5p͖%'ՈNzRЪEq))飷Y[ȋhxu/%9߱{VgBO.֧VMy J9uњiNH> ~[ҥ
y\iƈ< +PvQ)ݵV Wrg6r+Q j-ZO
mSuѦ=
EaNB_kŌ5ಳp [. j-j2ٿr y?g3畦1Y"s6|4~qjhiY!}SU,)+ׂڃkjӯGXf ="6O*{OVTbjL¸4LAیyy>HJʙ s<;ߍDyfV5~`lsVU ʫ)"%M`ku u't}ZGm6p5pQa5+
t4BmIw?57>pܘEa8㗺 vnY ںM~:YW9OFU&5ZV(S7
;~Ig, H! ,B{=X@
6_XPUmKog~|J2/:H{}{R \Ń(q^D:2wQ97K.VS U+ (k MzVؙ<1 lP=!f!*7B6G
qa Q<:Y] yO+QW9z\≀v%F]%f8& O :d 6M5=S'u E[!.òi6s>-p;bW.c<-rHrB*lsH:'n9ŗj>Űmܽp-$+!{ϷLsrvJ.j#ɽ]џ_.A[h *YH^ęう8vD# <+s- 1{S%ޟt#P ՔgWQO"Ǭ$ hȘu=9U}Nc ;eMZ=d2[剦ܮ)j+(ȨfoG]nqJYތ$? )/(\c#/}H^VD_TnQϭgeW>h|gjƭILdfDHY3e Le-9SKs$ \UX
٫;>}O43uF})b t]aTnHpi/mC 7)Ӟ
`N ;xub ȃ]e^S2o={M5nC6ژyZ=Wo1WGF[1ϧ
c4M*be>ހFˢ\Fa#;A(T9e1^zmKeKrוu5#_ 0o=@qe,BvrPSʕe:Y(sMge B \k>4su_f&Ukz9 ;Z\o[mipTlw^-]Y廦Z$-}l,q9 +m̐x|Y(Q= ˳sVzm 2wAM2kMƈGm\M+ԫ/=Ż滺 SRo-9 61^+
Fv0x z`t9 R]i^rf Vy\S G"-c&f: ƁG +2{R:N|6y:a5 ɘUFC.Q.Hn`{{> -QMPTڪ7t 0 > 3Z4rƾF68JPSDxO_)P y7a)jN6v'"ldh z!ծ/l[:G&ԪlhFG>A*^M;ժ
7ZCHщk]1]z:҆4~z_w&n0!-(",adg#Llm'
[qto_7/NNsUaj{ [F>ʰ~,jn(WWX"&^ॠNadΓb\d\.4Itzy4p'-̉pus&k@)
^$}kK"/ NEBc}~ӗ^ӈ/|OOC>PîuԸ_CpT+F2 )Em 9Qq(>Χ։'uHyg< 'R q?K947-D.rZGf5f5#ǵzW
҂1Zbk-* ҮfnZURCyz.,vj O j2+ӖZwWQ`g?Lf12Χ}Q5Ѵc%\&~tU- 7cKKUA:_wզj uuxK 1V[/1"kMSɁO]1{CwJwa'%oM- kٜq+DQ}*EF_O{ǎ.F0
^|R-hxv#Ua
عyFbgCRn!!%VXpK} >n)-)``uGܟ{vaIc̐Kȑ.!cuQ%[ 2HytI) M *kO?mfRtV
&:wFR2VGL6za˴48S]e'''MJnfzBesc. 7)$]c @q+)/sBcE&B.>}jX@{KtV.v}${P7mTeK=>iO b0Ib21"1M;0bI< :7tg9%f.Ht [
lz+G3iv_Y =
ݔq3EKe f]Z4}F6+5Gh7L VNr+*PZ4ҦC8u}}y4+3кx9.p9s>ǩ>۔2wN '(tw
|cꂝI$eW5.0)<ۅ"mxҵ &6
N*Q
i>+ù;8 X)|8j~I9)É>%w|TwﲿU ǖ! &jHЅ/`^wعbԇHxpWu@gw7U5շ\\"F +-8ku
>;o3{|2~)h~?_G1d Mj}gVV eiv0 !<,/qNC LudSV6bb^d; fTrÐƌ䔀S=rS)A]Jat'@jY5v?YaY+S%cNҽ՞eW;Z?$,ZL< M3(2Ŕ [3ti5 [>3Jh2̝W\ ֫Q6ZRlA=G'[S.
f"X+ʝ$@"_}$ gswVh7oUݶT`rZLBIi5b<1 BƳMCݴ%DT٬،4C|ޖsKuntdمW#5,ݧ+o1 D4pĔ:}i05j0u gMi8o50] n ,g S
z2?٬P?p XXϳ[&F\857Yw<)>gWBu4 gS,"8 AV+BÊ); AŖ$|?ɽ+V%V~b CSE"^Ε/jQJ{ Թ>yXVUȭh;dtW?.$AKY?xɁŤVq^_~<7AV/=U"Jc3rB {7`B{p?CkGW7 IsO
gŋ
{sM>
`Sqf]sUE'Z xS o<`9jޏ1fޡPvySO\X4d{!"D]#MR6grh0A ɢvl(
8`Oe V}oJM }oDksKOQ2N1Wx]Rdg`\30@B @QMl/:oW1
endstream
endobj
225 0 obj
<<
/Length1 1408
/Length2 6296
/Length3 0
/Length 7257
/Filter /FlateDecode
>>
stream
xڍtTk/]H!H 000tttH piR@.Fy{ZY?xvX[[fRA@ >qsQ ? 3 1`T KD 4`P Ł"|| ~>> yKw
@
\`^p=y 6kv PLL;@ [[B{FkK@f
!=,c=cx W dU.@ 0 mP_%ظ9C.n ,? @. =z^Π02? g3Yl BZ +- 5`CA >$ _OfHnWFVGVVwdea na 7 'A EW|@ma ?Ev? 5} X0$cA n'g i_Q | Z_CkxG
6}s]5@6`7֪ ,; D"d
FXI?q_ CA0W
/ r +U J5v ,pK/<䈑\Coxy0,` 0 rqCv F ȇ Y qs@ \A& ڃ@ kYk52 ۣS/ع}mnD)YV2) I6Τ|6c5'Z$kxc*&IB< mzVdk:xÄ?= >²n\OJ
z:2Q{ה>vBt3 ZMv'W d&yU~I{;eu\M܊ 5DƞP鰖U!ѡOqМV1bJ%w$%Wi ͌uG)v 9iێL7ӧiDdjBv1gDV S
P>i(yjBJI'V2*62#NfIfNtpJ&DgFW2vXL\ȭ "5$&0ZP&i܀d$L2v$AF Px
B«mjQvo}TfXmElTs'o$[ 挓'rOt(e% f}Xs[
8y?$+ h{)=e@Dz{fTC};=NMclw~]KIİ]XZ'h:5g8SPr)3$<ʠ]0њ/q)F,<Ϙ'`w\AGQ}wgpGqSRṢAdj5Sk{v6t3aQ
Q o|6ݰ B 8%J@R V۫(ʟ҅=ܤt_el7h7:}[K&whS@-xߠ9m T{4v6ObH- U*ĺr_ԓ ͇ט\pݝ f+4zrq{ RGwvʉ U=r ox(gF1.q5Y-2N]=P-Hɺ$3g4?tRnΆg~vΝ(}[T"Ϻ̺3 Nʽ87}\|ڊ=uk"̮?+5 tf+$ie#v54fq͉U
bBj6ZLo?߃B],_%^s^G0tf({7 G6_CIW̬wF}Ǖ%B:Rq'ow"SXT2 1PNq/:Q| (s |X 4C~j&VK`\B!>wj=E=8b~ Ht21~OY;ou389JL O=[ ʭNyB<~pTz"W[ ϯ%s~AZG4$w 7b%8s[b^tE(Ax
s\2L/tl
Q [gȫV %UE 8dGwm r2_pm1Ku`PͮtOV=y6 U |nz, y;"={v +3v9[(vܐ}6*]b4~)\T Oxm *B0ɋw>"K-b]紬+97Oc@T (R>\W
6y @+E{=8e(Y)_˶%x,wPB"4dt /{͖,Ѳ
> X EyK2[<99#Tj^ (N{Q3,ra<1ՆGg3Sx|UD3v?h,< >:2 0%eMJvI*oͰjN` [pL7
]v5lZ 8$gdǂ$gNY.Pn1KBHz&9c ^[FKiO2'Ђ θwJ
EƸ*j6J4
=K-Q}(:C8 #e05Y}ŷˑPٶڕ"Ϙ(;5;%5^K~ OaWкxw _Q8n
EÈ'Zk }w: 'G
öBK 2Wꧡh*,6)u1@ WU;O`kχ{ȑ6Y ,۔*IWxbnLTwDKRE8Y fW]ư0}Q:*z i#.g`u nƾ1=!Q( R ;܅LhưHD%͈; տ^:01Eڐb nF 1s,E`d)GOyFR>\d*Cn0tU,;
qs@N}~2Q3} 9&hFlku;`%+->5q2\6!%X}[OwFE*moA틜T τnE ӯV,dFp@j%9=i|$`cMaV:xh"j!c&CܰGy¯=j9ˉi|ncl-A}BNվ t'>OOq& (њtqrVo8 ;-ї PR7wGVd?$-sp"曖8Rbepꤧ^Z ;c?M*큣V
N=L= XDW-epS-ޔQ(tbwMnGi+N=%2<|ǪPEk||#OF!kq$lh֒.H~i9Gdh REI7c,dR C, |@֗p#}]) 3 cʤmORٖ駢teU+p;-y/[s
s>}bѪ`^~s''!اm}7s{p\՚젎W5]A[D.h/R\"M9c|,br>k#&,?Rͪ u^|%+F l1Kw8)eHI)Wkkq(2[L`xE 8mǟV$FFMu j
ҕ `0Nco 2ȟETF6gRN4y >*qxS! ەXƭ
d>7&=Q'Ϟ6lbͲubIzsWW4_۽Bqgjqo*)(OfCXi&_'Q46 v)_*
8Rz'QMc8"V&:tz8@ \r{^OOi#N'4-G98.-/Fpa DHGD ?[8}0,W[ TEx~~pe͕Ro KPAsuXi6ޝ yW79ktwyDX]%/ٜuZKt =#v&J8OM) 9Ҿ'{ ؈X[9 xYw"zip QM ^"NC͓PQN=),7Ga>]ier.EdWCԵۮ}¡;+Ͼ"{_,SzPe~L1 ==F xyC5j&iP (E8NPlϳ5MCwZbe9,O֥
OۺW MDI'aӜ z h }TZE_u9V-;1rgwQ~0WJ~ GKPxs6˷Bt p`
R" Dd:}CT)aZl0v'FMN69z;:&3[=4 )v?tی:k㳲_ßY~A
CuZ*zߗRl(_GrV>
stream
xڌwuTZ. 1twtww0tt 4Hw(!w<.u\UYd93 $455 ll,ll HԚ? $jm-I_2@3w0M ,r{8 9<ll 66
$?YAK 6؇3sr߽ t_hf A.p% <9 . wjܿ' ;;Ŀ. wܿ_}/"'`f/` aVwW\(w/п6<j[/_9`>]v_G+7:s_o h8 뼯#bޝƣuAƅ U߾+}ꨍp4dDT<=x <&V8&l4 K[esl3> J ȞWQ KcjDpH\TH+
Uq C: wKمFf 4O=#7aUuarF ן
tph j |?"n.Y̲2>Z˳ (arbԮ/,n8$l
z="}/JwEM5
v&"gj5^)wu4|F(s*Ěih8fj0>cXU}C&[|?i8YPR>.ajZ-d !%b~S07#=T|\z>QYM+9FSՁ 8k/}hEU16#dLamN̪T{iu39*- 70ih2:pGoD․tk'K/ TV:2K 2s FY|簡 <-t T1&YH=5
}ߐS-X+W&Yz.Z:oԆq @9PUo˛tWӮ/ x1aQ_MF[.\o>ӚkA`Pwp&JQ]Nx)2ja'M:Z d`JkrbܜDǎIm^"P*6SehǢ1Yul!gES1SuOHE, 8&"^dRJ9qWyԍgs{,s-\04X] _"5/S>阝۫DN *+Ie ȒzjT 5g~b7^ LSk^4i?@EZÄ
g$Z5$B qY3+M\bn+tPlb bnŅfUnELMcvj&&2=O zAwG p,MO7t}ϴ/{C5Zï2Q测@+*% M4!RWU1H_pK nC*ي斚\/Ԍt||s!T gMH[90bZ
!R~aL̿r9G?TtI.߀m15M.zHjM>.FY(iAjtTN!BHO6qY|R ]9Dx<>Ag IaFZ+57gmg#ͦ-3דg%φ'7t+<4kNFKFHCf
{[C j y`yHÝɲ{!is[")bSzzf# Og[>HPc[! |KeJdLNy $e?4I/V;쒰w.&}n*S[g2 ui?<\ٹhg'ߙso ߵ 1ȚpWtA3,zKКWI]% Or%aː.Tfs܄
V2"r-̭`GU:a,z^PӵPrG-lMɋÙ zwd2rFHʤ"S+:mf'D
Jg~Jn~df{7 ,k2d z3
i!9ߩ}CCxTc12}QFP^ܓݟL2{vRK5yevz |^ّ[wB=ølqt ͗"- < ɇ5 G'=RwϾY zzd'u(1-^Xkshs7u\XZ=61~mz!^j#?Љ쁎|9T 5w N2O9#y{߽
p%+YMeW)Zqk3y"՞V$&ai!UnGj0{
=tՋJD} ,۬F 2.96~,kZoȾ^aG2cE%@r0{ѳK>-,~^NgF d ҉g>L{Yv6yuuLoz9zRrEj]0K.LϨȌ ejQ,LDbZ6$f*ꮿңwHPP 1u>J& Ӏ. [wwroԱ#aMu1'H
>ի8Kǔ~L0= p^~8-gYe. Aa5l$Bb;wJavvJoDx뺏@$9ga>Ye!,-oTʥ]τ+?>79*P@!mwIH)8fdEwKƽ#~{ yۣg l|+jb5eLXӒ3;̷R an+|K(r%\(~R#sׄPh9?ÅhژbdE=5Kea^5WhE;(4p6ؐn%4NxO[Lmͧm0 䭟Sjϥv!qźik+n)+5f]z3a0GFgtxR k Ǥ`+/.
u
(ʬʯKQm{
N~" 5
(-#L1 >Hd (Rn ?~!8( 5 m11 ~ zi9^uRP }át_>K7ݝɅHM3L s-"iEr|/A5ŗ߭=]cON TzcHaVf|0=݊`?
A8IQ 8 Q&5dFzzayUDZl%N dQ02Č叼ɲ5 -|R7nCl ei ~PrrI2aG:6ФЃӷ]v(.hFrU F1@~Ts`D0Z "Sa.kHwoH~?>O+(ў![.Zcye;q\bs5j ,hnwSަ@o>gJn H85]д
-Z=WkqH `7O57(֥HkX1tޔݘF$oT8o7'i*Qkb%A'
]UҊPt̖j Ll0 v vQ4S 5g؍bZN>?n<2ZQ&1>_%ٱj_DfI3vНx~TPOA8.+Dzlպ@uʠ\2&l
&FB(FD .FxnJc/%ꌑA-kڼ*BcI m6"ʯrʥ˒{CfZw+KOG"+qP x3- D?E`X*淖Ӟ D,e(>W ^m{ DZ<%Jcs_/Y V5+:9lԣĆWz| vRk tb0-Nm
,fCan_y/vj֯Q[ݲ3ڸO!Н394q0>r(_<NXh"bǮYzs Xi'dxQ8 2#_TT6WVg> g c%li } P=)=]n[I è"8Qߵ{=7Gvvp
4j0b O3)_ys&Z|z\G.,caJ"K>=2NpHw+ٛ۽'e""O@c.:ydYI֗ K|1KB( l5"#$`O n R(a(axEpkIJ/RvŢzH`D* Ik=uo0|U/aja.
Z
_O=ut0WD4HY(@4#%n]#(ό &i^L;o9"SF(~:ɴܾM B^dx)~^=ֺ|_I3, ud41ſf|ӗm,c8.+6 {>~>jl8 2(f?k⫣tXXˊB]ȩ,
V
R<˟}tK?$><@ * Lm wG -f2{ ܰ |aǫKuZ^}䇚 _A и_PJoyBm|֍4R~~~wS`r#Tr|त*{%l( ľS
3Z4[7=:=5o
[G*c(Jb < LbǑzdL~>ڔp,gJNf{L/| UjÈ_؎[W_a d
ʚ;NQ+'` myhqcb|X߉KRV( ;uo_JŞZM/stpG hk0Vǂ@z@!T{{9Q mЊH|bjs{J/v<+J؋Ts( nN,ڝ- d{~(#֢ wnD/Ĥ jhn!SjIktbˋzo} & 콎
5r 0xQHVôhdTd]Ͽ[cB[ӉcO;P"d4[|=v/061C4۶do~`)? IS]O5$PʍuR@+2e_mID=
C_Kq3˻55pđ_|U4 d,N [KvdL!F)fsz|
)vRڣ|cGg
8,etw`1I-u Yp
v3Aa@EI!q/ o4@b}Y=YG:IOֶ89 LX<5f'JyRzėf Yu
$ 2 jZ$bg^LMLQ쏪kvVr)?1Vu~ͲC`\kxO-48DIjN)pf"ˣjXABA ki/n 'mRsva$j8)5sI{9
ሜ5 UXaBW=9] 偹Z"^7\I6 ^XHe)t_bL6g/nwm)iUf)"IΪ8R#(k)
!B2"\zv,ۓ[N{偾v/- Q0Òk N="TRdNB֯kȡ= "o~{3 ˭p )gC6H3&gS{Uةp[tf{,[i[yIÉj=t:WAQDi
[Sǔ;j+f <t,GK5yQ\OYLݗ8s+3QH?:%.,٘5hW~.I ٫\ + @Z 6X1F@ қgc^hzt.:RblY 7 ߝÖܝhrsR?m'2!Eו]jGQ:ҟcs3iGgjaɝvaqHeaf{, i4 yOũ(~IٸK=v.bJ?6N/T.S\ѓ>XŌC9Csn`{
EqgpΑĭܐ` c!e92ɊOoX erTE0u4}u>OOx|]
Gz3,::/>X! @ tY7ۓv;oE >-KZhU1²G83)ck毃hl,bhJ#@g%(Լ#5U~;QB|13)dq%5u]mI/j
X $>,C5 6/1 )ҵ.{@7n4p$},ZT
Mdsկ߇ ; 6\Ywp' 6eZ'@ Ǝ
,4kY|encB*"i;>;#4j$wr݇vB轑=]^y
hF+;.2l4K}wWU[}ݗw<@^3Sѵţ2)KJ!lU1c5`p_5a7j¤:JkM?Zpʟ^6PnDmcq_WsAP WiD }57(.Vu=KFVw fxVXsH֘ۖsגR?1qN+^"]'O猷Ap\B~tdI[?b_d$qu5O $M
h]Jsڳ mM>B:pWU&C{ Wy8?KL8KI yg BTsOjYdže |r*
ƻQ7{7Ul8J%Y:Xpۉ O\f_ a( ur>A
ƷJq @U3xE4\X@45odNv.iįo
{$Ep5υ· :+KGZ^ JT
u]䲦҅
p%ٞ(ǭY8DB#JBg!~
(t)XW2PKu|Mf23oG.~/6Q|pC@~Q;.*[Q Nv. c b[50U|#59g@ _gNW5@fmW%kv+d`W \Pퟝ[2{SJZ$%d24~ޝg34լLƪ\m͇Ǔջ*JX' cq+DFg
f+g9WE8x1?2l?$*zLhb{
+-{l7#|1vEbPN4a&r&Ð0j-3UP?~ eH Q15sNu (7kdUpp80E>ɁFmfA^QeK~H盾亞b{NG
Jlo u)d@54 ~h9+[i2{Y|ɱ -hm :T
`4k >KJP4i!]1YP|y(V㛣1 ;>7a6jG*RyL; W i<4,Z)I
߶ Y`Q]%W
]ֿH DudQkROJU lhv+7s7Y֍td[06'('
}ΏyގY3S[\]5Q _9}ywRaH K$-hkޚqN7L2.I2.
mcjEd"zቬ& EnCu^617
Dc
VaȆE%zƽpxb$_[v6gc%p
,¼W-= oT}VQ09g G[ލ
>CTApҴ!1ɛIj;܂; @1
ɝ~g
N:Cr X94\V]f=u_Dđ`2AO ?_r²/O⢿P i_=U4cR~.B+ UaSE/ru%{a
&*T7CQu3X}<Ua I
t1 {G*ZHXJ:G_ O(DgQv0摤=MEyrϲ&?P=%p*e8
1+Φg6mT iE
{{h{= A#x m>d4ʳ^'X[@hjtXl I"*hdZE"֭_q\ֶD#$9 p7Ēh.s)3Q-3Q
ץ%?^뒇
b G˗r s;m_3n
c4os OzaE 27?O"i\ˎrոk, z$56o(6Q/ ~0҈Dbcv ߄RB:kD8U͉N%>aU{)@ q'zs3oiQGdl\#ە6Jrݻ |.0}(B({f~DgJTj sL^ʏj
h8`f* +c/haމH fv^CO=ޕE9L>' f4_E)O-6bexLYJ _F"D|#@7"B$=c)>x]pW&I݇B47p+J}CŔ_ATe|p%ŀҸ|z ( Rǰ @t:tPwz%)dë/ C5{4H =NwF,Q+>QBgǜW$4b@k;s5g+~#_F(s)ObS] _ׅ;u5є-̀([֨`鰮OҭUV.wk oiޱ
=qCd0kpY9X'[>Hk)z *W+n֪Qһw0AOeSvڻHiA_g L+~&0}N$a5T`RW\3$a wzstSV<*I)XO w
:8cOGG71r+5uvq-VqCkܘ}V^ w~xWFY,6V[I/seQ(%|M4L/s=M. Z
Nǚ BRϏ rrށ}Ř>{yLMln /#xB_,s3Lļmu4 f>WU9A 5K #S|RTm tkfY瘸yLzО,.fC]x5Ho`bsZnRj%8 yc
dQ,PD~Q\iSNDtyg DW R$!sŶ~ђC~l1mg ;%}x;jEyPUh7ʬpic) FlĆR
bl* 2a
iQCuM[;
eVȐhwTBY&Ay0Npw\"-̂k@D+lG0/3/Rw~u15Xeq^]׀uA4xXC
Lx7`;*
Ĭ!Uw= r/}alxe}Sf#s@Je,X){c0;!oKRCM^tbᕚUBc`=Rv,7:˙$,C#Tѱ_E#'"۾}'n# ;⽛;?۸q7W_wʃlidխ\I[$<0`H"=樋ZeRJ#
e
=%6Xhd717|R+&PS j:>W U4Z 0{ >:#2M;o5]FŜץ`RGEND]0"K9 s8Kk/>+>
stream
xڍ8m6nS5 jP{n$A$%6J=(E]tP*Em}q|ߑH~y# &J#T @ *z&V@a1)
sC=<f ACz3Ȳ+]w/ szQN2P.].9陲!5dR6 @EAe5r_dz~X%K*hMpo>thegf-YJs^5qt<ہ'1?q&ظ ϰKCZQja-\ДrTw+KtR ۷
GʅȥK?|?j5~Q}ǾNA2}.m$˝-e}KfTm ]㰆 [D eXX3nQ Gˣ3͉̽PV *9BZ^T Iy*[ɦ0A,pw4E;6-q
>ƏTmBڃܒ0u oC,
)!=ٴxJcuo0*qnPm_JdAfMkR3=H;Mw{hË. Izz|ndvPݚ8stgQIcKc|άf"/Nrj(-yM4 stW`.Ű}6lW N1 +Щ4~̪u3̋;?) 32ݣ߳VyG˨fn= \($^M:ūf ,FhZ(֬43_&Ui nl|ݖ5E!E&-^n<ZYU-[ )]|oc<#Ub:EA9T!M7nhEsPWa3C 1OHocV B__%N^,0r*dkGޒ]2zl}w {#YQÎJ*m7<{CgКyb~hzFS[Ϩ4:Հ'ZXb4Y
cgTN_')U]> ɮdg*l
:XqR9[Zs|>\,ZxN[' u[3@J8GCZg7l^2 6X 7/l"ٛ96(%͞VWyrXM7^HqV-N6Єv$n_zc+ d+Q/F6]#N`:"
R,ǵa_UA(ƙOsaX{. g_X+4aûA9vG[
s$8Bxj xO^2/
RI& {\]V#ɍWI ' ' dJ3ֽH( 4idU tI}ꏹ*
l aЏ%>[cFJ
?7J|vwaUݷ C?n$#֗t$'z G&qru5Kft> A o=/ Ao4꾀b-s+
IP19Dc%5 wUx7}&]-堆/n
wM؋,pP(]^}uhJ'tGμZqh& U`j;o[{| _5 ϾXMؙ9fA/D^jBs
Uxd")J ߮ XDԓYǽj>}9N.`qɣaWB-H N_z} ЇӺ}VvQ/M[Wbެ-(Dû5
mLC}="
cyBN]X>n0jy$al&C`ݷn nJvC~<
k2w BlYe.0yo-[ }4Fe)X5a[rV~w7S*acbN+ZksLZ&x`Xw>\5hKC㠓KvҪs *(tJP &v78#NRW3d 3d kܬEXX
@2_#T /О,Lm\̓ޝz]؇Go~^ކI8nd+^ܨL9b'%ߜg=QZT^ukr垥{MqI߀
ި>k_ ~ցԣ&+H:/ uh,`(ٞ *l 9\NaK},;
\W &w+-6 Hg ЦXc6n RpKb>{fgoӡ߇4d?٦U4NHκ%{K;Kp%c"/(ZܔyhNɃ_}ih{D@8Yki0Ez璳y(Pvy8y~J40YMK~\_=e;JX pʌ:X*h[ϒ'/ qk8.WXT
?tD2=
x@h-϶(ɻ54V'z\; 9\%MTYm=+!dHށ<
PK . ][ɦd6_tG ۩;RYd( ^>9Dj}Ύs\)B b7 >e4M}Х>%p=Dҍء2%Y8_Fv; `&1:AӸ;/SlZa.@HBĝ'd騵v7}e.Z}w e FyB˾GNU|s {j{6w:M)l>Oa(
桰 ۞c dR/y?꤫v*PGz2w5NVUl3PQYzkcjL6Qwj '𤦢,>^٩![xT_UǴUTJA;a`mqv#zuJK )~o27?+MiJ7>bLmZm h 4ʡ#'~ʮ^>x`N: lmġ. ;8 Mpx\`PѦOka;&Dhߔi
ٙP0j\ W+ #|l[ B/T:DziK> n<^>
yo7$0BLڂN67KR$E ~2Kʍ2KDA-VfOfvS,o^5ąxjxB Up ړv< \/OK/mN
uCqv
S9%hrc/
{ 14{}i9`9CaBvxP:`& a:ᙪ)_v}v+> C;#bE#
|Ք%5=Lq
fa9V mn N%4-HPp^5#xM!6ʞX G0Fd~6ia-2+-,|m@}}7nܮgNB2};`³57mh'w?]sm P-G?^&+ GFQYgc։qƫ*$RL3Nq8t}*zvKtrZRJmY#2/XOdH4.udj(?p>)ЋN[H"013۔|2u8w+)
&W֬ż{I]$<(EHїll׆O$w&^K rNU{0hQAEY7Mцp/D;.{ Yb,aARFBK6!
뚰)ŕ:w߸Ҿ]-";u:1G[Yg3W C]x]aX̡]ڴM 5gK*6wgN=,,sMkCBJ)#@1y(:e϶BʨҾ@Gߒ Pƹ}jq:K#lSgePlrNy\7
9еgw&*Cd,kӪd\
=qMXadQar oύO]L-K"RǢ_#'8eyWkj ~K`ʼnc4w}\E%xY9~a
aK|ȷ n^K`M
E!ݫh\ngE ;R^; @/J7W
rk_`{o,/~x8&o"e
~6' 1Vz?R1 IB{?f]i]Wa MW mjKX~HIb]UI1ryu'/` w@r8*@J74v2
V{=D|=ƒLxg&vd|ޞg)`t:[y
BuЕԹLW} nrx~LH
9C)v
U C7G\ގkIzh<
K\badps41.9L9NlɖR'&+gz:+CZ vI&#<7Sޯ*["g
?!{CW.
L Yt0,D^ϙ<M!+QO5G;c'% s6b:m^62{T&X4(*qpߕ 5 uJ0̟L5GT ^3o`7r |nP݁d(MR;&(@9bֆ*AG a(#yAO;foox*Y?>Ԙ3_ 3pǺ9!6$G4{:^
#/Vv/]bIK"|3n; "+O?; a]m"8#0 Yz5eo/ w |zS&
In3y w-CGSDPAMa3OQLLԝ>j eC6UsW8٥]`S+l>a;& y+}8%3XxH}ĥe6* P旽J0,r X: N5qYX-H0vOCr 65,4gߘ]B|fOgOPw ug(%I~[= l_7 "Yj%*^h doŻ^u߁
0 lf n[lOosH{x+g3|_*sg~ɖ0E1yE"bM K/o` 1pN;#e& > JGA
endstream
endobj
231 0 obj
<<
/Length1 1374
/Length2 6003
/Length3 0
/Length 6951
/Filter /FlateDecode
>>
stream
xڍuTݶ-AzWޑB IPHB H& i"EQ"{rιyoFH\s_
M
4
',
UM,e`8 ps"qΈa 9EQr A@qQ@ wg8PTJNTZ 1r@5u(v pmEee~]$ Cq03
C"pSp\DD<==AP,Wz"q@c@ށ LpM ?q A g$ +Qp h4pE (
7_DHb( vq({ 4pB@(
uƢP( 9l; A ,"h𧬎]\( ?5$ ȟuB=QvHpwW3 G`Y)) Y
97vEN
'uEC v @ qwNs0 FY/ Bx>~ F9{~Et5̬L
+,
JJEEe$Ҳ@B VeL O;hj@, D_,/wCο|Gt 7>oC-L#];⍠NjYXTGb5^!s#?q_VsFh,_W/w
?UG_>B17 }E#~+(Bq ~F ub@wg χO As`&Zv4ᅀ'0PWG,ޑ~` )|w9CsM>@ ?!/g8-v{M8~X:5jz
b0V첕Xp@Ş{ݺFcҚ*Is76>'A 42 {/@etvVH2g$2?ҩH LN`z3G6YA.?|QNi?~5kp)Fy=2x2WKM */ ;/agXW"Y`WSq=)e%mxm\j 6ǪA&kV;{^%)z
n.0ؕN7רΓ~68bIIu(CKg5fѐĞZߊmu|W
w rG{dɋT ;m' 39u5Ƨbm=،E܋Xw<J1 ńVI:;i
[#TH| z\x |
px-ƳTM ֭ !BX&?]ՁتWYÿnޚ
߇qf㦏 h^BܷP/rP;#:YD<|?\I .춷b`5 [ W?~p ,Zֺyv~sh0!& :=D'bnfS63C m =/+-]2$|
# %$1R HKȂ7UZ#ΦNݾSE ~q x,
$e:vNK fBxU<- emlRe&sպ Rod%fdH0)'3l.s Vq`溣$-Y%YJ =/ IxƢ_.cijx?bp/yj8M @O
v!L)!M =hsu>RW)x.md kׯwVyD9FǾOu=̡)~^Z _L*i]@1
y b d0( Y7 KaZ}< X91cWfQʩ: ݽLe-G6SXT Qx̷P/D-RCm4֖.ۖƷ_x7ek%9?tg L,ܯ}!"dy9" l (cʸW ˾6v*(g\r(/jR{gME
HC;u4C(b]f GN5~KF=5 +Y$"l)[VGbo㞵z?Z3 m3)uőԍH;Z@COϞ-,Ft-S o/%yZF %]Ř\;0 i)=n/&2|aOZfCx1T+Q&EV@ /'x^9La^wi "R(/ y(m>0>ޝuHb(jpf- e|Bǫjé*g4\=S3;j gAVtd
E3rG եIq6xv7?<:ƲQt˘
"O ỸjEUer3m/<, ļa"hJi\G0gQ ʬGvmjJG4?d-qEFCѪ ~!ێ-.b2#{Es=uPWt&Va^ ¤½ iҡw[R :qjwI <El})nد0 .r
[Ϋ dNtZC^'{]a*]Dde K!]auTڄԧ1u߽Į 7I%[ |ѓ}OMy W
TO2Y7I93 2䥽Σhi|nqV^ΒcD@d=aCO >?Dn #ywkdXkp`lN;%`گG̾ 1m閵 fOSJ??2^>xGY#Üpآ Sޝ]U 9\a7vZ\s 0'>[6-?xk咉bTB~Mjď8 O.oi=/;+0=ɿ8 =<ϊy5@{>oVzh( Ho@b`bӐeE2 rډWjҹcuyva\'^
},~l0PmGAOL'_ln(T-1#
&}DdQKB5-oEK-ky t!ߢ$ꮓ[yZoIA,)] H)-e%扑ƾ´c0C'
5 tIXBQF9*>\z/5͢SP}g/+M6 QKu8T1=xyQs G/0g*EeF˩5VhM fV?QST}![Q>pj֘p>$>4x89CnWᵟ rF}~3 B+s(u"sԵ{ksDaP6j5ݥoj^ "Rs؝s/Y^Bck pcaKKޫ 1YSҀ=MK๔qN&P4E06%X1!駪m=?Su4dէS?%,>
KX4kJqH870'+')g_G(- ,M>F2(v,?2 ^`Nk,Y}\cVB9^h:c hM̖&;JPUye>-op q,?@XD >WuS tpK; M ȁ(flc}5"YpGgc6I4jxh"k 7t
3My|XV7lWix\6: XCK>ڬHJ2M~/ُc B1A#,EƙJ*w
#| C$Z 8]!߹%emT} J 2GWv.e/P@*KwKX ڎ/Spw6i#R\̏k4 hB
$vzqY 5ߦꜼ ݘ9\lF'[D6lԳ%."9}")W:N۶cusH+} (V>چ햋Q~ { w y !8X3*/ZبU ]N- :Ϸcy}UdZ0,'$BlDOK͔CZr<
ו\AOl 4OGǤJ8}B}*;Z Ne1(ၥ'~6Jm0ۙmc5C31XCZiQTX] UНARMOؼJ? M zeQѠR5RWZλ&!Ăa3 ZD錤`b乨
~k6{z~
endstream
endobj
233 0 obj
<<
/Length1 1658
/Length2 9777
/Length3 0
/Length 10845
/Filter /FlateDecode
>>
stream
xڍPk.Cq";w'@whqw)N+NpvgΙ$ϵ|Z3x"n 1@, IeMyN ;;+;;'2
& 926 a! 4s}I*C 7{ WO . C @ d Pf(@@dI3% >@gljtxhafЀX^^QÃl- ԁ.@gw%w 3?" 4m@.+4 V f@ d-
y%#̀`pxlfaqp4{ +=* 0[64w_ * #0{\,A.. =2fi$vuA]h2w/ YV۰tsd܀RؼY]< ia;#/%oK~>GK@? puvoÝ 0_@;</ ϓ ,!`{:b6MmU qiZ ` ppsx~ :[A 2YR0</_ t忣_+qKOGo ⅹn/[ y6@Kʻl8, ܬA.2 O[߬[{A`[
t/KfarP/e;4b{8yxff^/pl%/2XK~ +3) lf"~ H f?wl@ `rl< 6 % %ÿ%9_9__ut/||[9;>miu@ 9Pm]hMk1NnK$-i"]`I 5C&;]|v; iт
mV̾"w.Jn;\`Kk/ oζg3eiT]Wڜ 5 F#sgWIԊOtS;N"ٿM_*g>8@sT<M%V4 LlHl
7w Ϫ`z%>Q)f7, dLrc,Rc+n*__"Bq!-pe9QnK
oyF{5Jw3gn'MR6#;?Z:ezW-si
ͷF*J/ n ? iϜ!T(e*X _1%fV6hflIU~\*6wԊϴ0q (& RdR gJJA{DjI郔T|I<)b<Z+ 5Bd}`.fOF{Er0(z@g+ 7?>y$s;exD@Ah=Y7tm7&Tx!h?rB4WCm (MPj5J
Jec|ڳڑt b#n]l(U2Bu~5Sɫ] d/Z.]XJ4КQ Z
q!2^Լcc N[D_* .-mG"*9kVy0=fn;fY%c kĖ&3&Kx'] ű` |YP[1Ċ|Qg)D DZa궟`uYr>Ej"P-K'ӁS @^ꦄv{K!i#h!D&eqo]9+i-0
TbĤ(LI-6߄ZŢlCw)w5XQ Ft 00 pe|$4ѯVڰ`&ǂ#(ɛR7U(R<܈!JG7 B*cr
S Ww P|g ^|w:f냺Sp{6Eď;tR z#c!`Ɯ;3/
pP$ v_L{z)`("&e:ʦ3Lu3(9{]7&8#sjOru #\\Bv ʌӖ <:JYV8 r =}PF" m?gUSuN >Q) 'k>0 ʭl
Dӿ ;<@EEf[7ȕQsP: @]p
/-5*!
o[ݸ['x ڎJ2[L? ؞{Y<-yf.<Ԍ$ޟ"轖i3csM=ʗU\W
N>W}Vk% }7$v=o>..n O]z!;Z&
Wol.߬jP>d/ 6R/Ȧslц{dh7Dɰ*5
#:5D?Z~XڦmKkIm6CUݲBI:n@+*_ %=M
n<ęZsuy0J]F;CZLAe-={t%;ˣ Oȴ ƮQZf}{*t66.?'zyw #jG9;r ي}gW|xj6Vs. ?=?fAJJ`5[\;b}HW>F_עJҙԳ 罵JރUJe{DD^)-dKENaNƣ ȹ().oyV3 T:'6_].= ;+pHb.i˻`Ib m(L*$ qaO4hnlRf QG7@Uz7&@vW[<xpthٱ K :L"t SKFDe1HH]ցJݯzyRo.Z1wbH)#< 'E^h] Kր5ldb6t+$[xh^雥X̰_ئp~*-%Θs b6fwԒY\SLҸP:E\
%RΟm/,?D\ΚGqz*Ra+`B4(
=9دJ2p} |kSC+T[`-:%
MrPv%; nIWp> 0 xs㻫P
}m2pb@A<c5j}Tg6!}Fˤbi }u7.(I~2%O t 1R%<=2qbŏX.+V^4 |dntݧNx}߿+*I"?.*f^Z$YQ뽶nUeRڄ~ Jbׇhxk'p˙y3ӐϞR]*X!2~k5-',-$9S78$%n:{^WA(qQX3f`ζDa 1v̲r_#-sY*lգzGBI@
ID2H
Is( {KJ ﱧ$mH;@엵جT9Z'T;6q VKDm|A剡̊#L @(M]ycnrSs5tMIgjTR_:]'pt6/^7X#R sd[aAU<=OrDg494S{Rݨe@P@ꃰl+GU'yXbC2~S>dq ^HY 9al.'AS:eݦ=|wৗw, Δ*C}q )"B
" ZuH]SN?ǼmwKZi!I$^3z Nس(9% tyG=V7> GNiu o79 I1ڌ=GG TNg^K }Y?%Y>vX蠣5\j˪+Z4LuҀFcz5\taznWܚ>6
%Zxpx=qR&0ϖ
>߄j
pfmWDY0O.hؙͅ w/'?xV uMtNhSvqz%jm ڙ˓~9!) ߥqmk ]9xCI*Vzd6G\cTg "H SJ۷ -[(Nh,aKB9Ljytwr)aFAmT0q p &1"m
~RMzp
).42̳++ ܆ k?ۼRW ;!,Tӷ3D/'W،#h¡ԫ
&
W9FcϴÆeeA Wj(qi`@xaFGc>6~un`=6gk(VY=Q ,qșM{j~Vv8v41u\uk
xFM|
f2sݭy$iSF̱i=veg"6: pݛKM9Ӌ>ڮm-9Ukf# +3Ǽv9 t#QcNMr
)v Y|魑 `GJd_[G=kŏ] ^K&O
H54aMXv>~%WDG8ƓmdkX\Cgb]hl6= "WVJ rԎ@ j0R{NHPkF É&U~e8WsbtU"}ѵO_v3l'z1WOV (ԹEai*mw ##Vغ], *
(tE 6 ÷KM YG{[n9K`ŋI u%%Nt;7?qh`$l"Lu8q;}۾-:OrJNe RFS br1
rmrbB ?AbƳV[,EwrIᒶš;3ݼK~Dv:|h~VG-z^
"W?L6bJ_nK6 Л_;Abbl,hƄ'*uf$TuE9gAhA#>;DM$D9)ryEy |&ķ):Z
yEH+
sɴ-k>|XEnk
Q)&>۲HKԡ){{_`O⾥Ն(:gbǝ< 52eKy1SX9N(M=W Ep2;M@ڧoyueBq,ފӨ6 8żOoѮ;$<ވeaf³WmR "H8'ͶiX^mUj pm* ̤ Qx$s Κ /xoGm`(N ͥU1+eA301qw C_7u,.!%jZ˨⯥0?wA2&2$
#ᕻ, m% c&+Kc>ejOp;۰?-=G[\f`{t;u4ʬ K@9
Rc5{)tF)`ֿ,.{
zb:/@-Y/.ࡰzt} +x Q]toe8هcJA'E|#Ox;M7?2Q:0K1 j%kpAqqhw;%Bnۇ#$`Rg+Dns:hfoiOM|'^
{P -*k6:
Ud238#TGehrF KUob<6w-ȹ~rBS \.f|:kW1tztӠW
.)3i#&UROok8kwT߳L_N96#'s^FPx8>N5:(]L ij>paFX;} "f#Kntޙ^XTh"O<]'l%6s~ؔ3>{b[/hfx\3®6K tGF\r={ &D}>9"2FzhVqՂlj
ZoI(+DҌ@E2qrZ thT"qux ];7*`c֘,qj'>cOX SVV$tqo݈|aq=QGf) C~q`J jSXuIk鬅kjȸeey'Q9( y0>F2j H
>
^Y}Ii -&CeRu*yxe\Z"KyYCWc>V&ZےM
覟L>?
n2>V~vG10=H xe^asr93%t,+xQvy<<|i֕ 28VxIq $\}ӖQ'<?J A3Ֆ\Z`mBoK( QW Q`:FvO L|aU'(= Ƽ51v(sg
j音J@M_2
X8c s }ʠ%' 2BCl Kx[29U0`pr={Ef +řdksֽG8]5 E [N:duLl&5~ bE2=ΨV Y]E7U ̛Φ$hL) #0\P[#k k{NzS5]
G{Y6GgB+LMԌqiH8=LSF˛]$nn4m@nx P!g3]q ]w tvT-%p,lj$2;`PMlC.Gԏ" y[$W*!vFDzP@5b` ԒkiW2)ѯYԍ!41͉e}T 59ze0[(˽
$
m-Rf9-24 nƜ }vbrŷ ܳcуZap=;
dShj[3;=MNE& BYl0kfB&p 8 <ٌN#å1 FX+BSeHG|Q n`4e7VQt'̦B!6/[0uRt"3h2-\XG=C(֤
endstream
endobj
235 0 obj
<<
/Length1 1790
/Length2 4877
/Length3 0
/Length 5979
/Filter /FlateDecode
>>
stream
xڍtTY5 SRPB$HH "4AQ*ґfH 4uFgVJ){9"'M̥4x{. GJC-# JC 2 XD 'Q8m8hd, 0e2@ J; ʀ6
<Eh]< G4rqi # pF080(gʉ80#0(RI$e0]L ϝ1$4`"n($p2p՚4H@c?
x;(@ሔ2"
$r 4tap? Jm ܻ':8t8hIv_a\ɨڿ|(7"DQV@(|p z@Szr 6P>E oB$AQ wv
r)Ox 6 Ͽ!8o O ֿwTW55 e <,&p̯* # b)wn kw6+!k 'uxܾilʦkTLn y_d q6 9
gIsDC*k;Y ґszi.Iz*yrKbGh s+* 6OP0qEmቀ؆O%1|G].Ϟ^ E&kuw '<ʐuX) %/M귿ȑܥfa64ˠ&CޫORG&oTHwt&t|˪jYD[K e}lN f`LZbj]~,
"T<3*J aIշL[*! ܺmȄ>?Wm©31䛱Cr 1._ 9_>^wC˽>/-Gt 9 vT :È 0%Z`yrJ oaڱC'
YK>Ys1EoZ`K-"w~fo gC: =c[p^=Flz1o\ 5)ߕGDp X\EV7s'bKt,K0IƗ×#(F(cw>p _9*s48qHWK]H{$ɘԡ iA iuR`hVKçd_[)KѼ- Dǔ
ji֦DٿUR=GXTNU%4>_^P)iOshq[ܻEb =/G*"Kya#BDHHV
ZZM(uEq#e!TE5ύϱm &⠿paԣaĔC >||RkAW4o4{NN]l++Kw
s\*zs{ Cm=k'U}O +I^R3x%-E@v7tyD,6oNx]赺т -z+F2.͟Xs#E
@^; \/~;⭻yp3%'<>!%hJ
|ީ$kᄔzjh| h\ yzZ3^ՑzOc3N.7)|nM^Bq
>) NG@/@}9 [8-8yzȥ} %Sq\/'#46na%ƴ=Oe䎕ǣJNj:zm$vht_RZNx; s#+⼝\LhO)gFÖ3Ws o-GHەuϝszo]6mMJ ;[tb\,]Y62aly .ewqzܵ'sqtGYLյ ƽԒ3Of,%`t5$1d
Or
[vE^=>6w K;cNwsm&BO_oYH{h|qVL㱻d?+5AxrAqDK4".hHTDn Sޡ -׳7b>@'n~aLZTSY.|BеS1&&aQ
.h _>`YHkF"fY[Ha0╸A@FsnBEL/%TZsj7
'<VxLHp)KtҔyq\+GTWYzk){Y@
}ze*1in?;3%`8c X^~5Ҭցs)p7W2DmlP5BoNQ>
thxa@òهwOSp5TH'~H5*t7rc̹
V] AX:-?xQ` [|ISʹeLS[ 81L]?i-g^)Ut&vF x^wNY9YzY: %XRpZ i _!ưp,||%kaNT _y~ד-YcZe᎘ Gwc͞-HIGrhkT c6wSz-0 sN
=oC&ޗÞ02-|% 0z\r81/7cT C(M//CǽTmͺ:(.M`s:6^. v8a[aO5~Άosjb(s>2[XϘ~U ^q'u2둃 r MJ ;tPVd[/v,cAB 5a_ {2W j6%(l=<ڤ'pz M籡f3'7劥 qwnf5orڧbN:oj^N>*Ų J{2lj>c\,>;{4k0q4@^Us5ϏIM}Q %fמ|UsPYҰ_ykD& rW ,:cqv+\Q)C>@ɰ=l7WwT?5((خNzh $}Aec$hV3%*fX_ {xT f1dMT]yd3{d8b/˒gtv|䗛оHB%>NS>ht>8Ir序]=Gp l|+ Mٵϕo-b6a"~u+&Qѥ I#kTuDLuT rۋe#E4tVdT%-2] ,TYɗ=z >~4HyBx(O^ܔĤ?,:klԫtPӿY>F Vx'9'$f{%_6B|Zkە5+ɢ7|=C?
>٭}-G_vO5yk%^qI"癫V&Qqr$wV JY=BBG:_d n@qr
¾&^i2LoqG7X<:o=$)T:b57hE^;}9uՂLE*fgwҞ\fr+ؘؚm2Ӫ%Z/3 )_x104\g
*:g*k ZYU˾P\]+s%r? Mu
niD5W {U ^G>? ^MSr+:W9X\|x-zs>pdܷK(I*lǭZ#C+&^՝X#ʜ] ˙>涋HrٴW;yS+o"[JJo 1M*ۮs۾ރ -bD+us^z9֧f o-bZ
(c.U F
endstream
endobj
153 0 obj
<<
/Type /ObjStm
/N 100
/First 875
/Length 3412
/Filter /FlateDecode
>>
stream
x[nG}W. 0H ;1@dH4=zlEh_ŞU5khgǃmb5Zy HOof/gd Gb6st휝GTe QOfJ N OL ?chn-RȭClHQ_m@os>?9t}09MN߽y߾.9ړ) G_. Yw2zw,Wor %5Sj C(7wc:=Uthg mt>Yuь!(4bh'C@I0JgRlC "ȏͿ\
rg$#%KG*E9zX- _3 }yx+o|;ImӵHoiG)j VƱorԷ9rцVw-xϺYt%czVWIQVQ;$]@\ߦ$2nv X$}4O:a!3%X(\7ڑH&#ad I1q`;8 B?570xFD*3#Nd(ܣbՑ~&8 m>3( 0 X/:m * 1ge1S[胒Q*.Z %2V(| sGl3ڞ ]Џ}熏UV,֙,C fS,
XbchbrK6ϸň@YBF=O I >|`Mxnk;9$X8Ɇc%rtA ,b-c/1K$(* DP%x5z+&Jf FJn]?S:tyۊ̰ ^FXv<m kϲQMd_§{:n} *nsU}[3`5K .Wԅl~UeugN ]
eSti u )v3ҞJWh)YzXձ;_~}ߪyOsBT_ՒIٕ
W.e.qyBEztRhkx2t'Pɹ(*ޤ8V
*dH)k9K+V8/հOƤ0I d!`(BCJEQ 1IN0[ڐXeaJ?QRp_x=3-ם II%JkK*ZjF't(.Kj6!29tuJLr6T)m[RԩNrD;I6UZtuiW]*: -BeL# n{~efۨRI%o#UXV[MqDzWKײZIrNˌU;!,*嵢n闶BWW,
Mi'5_}tfXGhA,7+L|'[^#S u
M7']}P7<86?(O cʽ;/?ǃ ӣϗIg< LXF'ӝcyoA/8Ma2^yxQO`2MBUퟀƨ\?- o^=?huAO
fS ]TV]mT{PœM:^g{Y}{NOs>p 6NWh Nțu}}n\ٰfNT4[m/m1{v|oߠVl] >y{o ܉fWT}NzTwzجV٣{?@6FxnU>a>
9Tk=Ocq&`Λ;, &!p$&ڮ~Qr`, &>wfحvO?y6'W6Uj>~ɇT`,xe䃍&anصߘ{t9kJ~+7~qM5vkp( ūJzJ[pp7nV^T]m/~P
*]UOjq5yz{h .ToA3T~ ˯)CksΓl*c({
rVde Ckol~ ުMv헿@g* =LCkafwۋ &ɡuة
nXϯ /wf'}v_?;t|tCP'>,)kM)o'1R
endstream
endobj
264 0 obj
<<
/Author()/Title()/Subject()/Creator(LaTeX with hyperref)/Producer(pdfTeX-1.40.19)/Keywords()
/CreationDate (D:20211205213914-05'00')
/ModDate (D:20211205213914-05'00')
/Trapped /False
/PTEX.Fullbanner (This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2019/dev/Debian) kpathsea version 6.3.1/dev)
>>
endobj
238 0 obj
<<
/Type /ObjStm
/N 37
/First 303
/Length 1293
/Filter /FlateDecode
>>
stream
xڕXK7 W 67-&>NEN2]صlfV3a3H~' )@Ga0F(B9'CD+ qU^ Fp B@ 395@(
ht$sR*HTjTR@8g, AzZhC%KmSCZ68
(r!Q\()L/ǡ.` xG.6olv8nlvOOQYTWbC_6oV!Ñ~8/#*k6{[,9j1m~:FmXV9Ӑ@5 "4=\ot$
`5oRةq,c A^B(t D[ƎtdִV!3eRkfSWoezUi1Krǁrm@yoJ 1r8A;k%0\p-1~?Ow*}Ko' <2<8 %_ U9
endstream
endobj
265 0 obj
<<
/Type /XRef
/Index [0 266]
/Size 266
/W [1 3 1]
/Root 263 0 R
/Info 264 0 R
/ID [ ]
/Length 638
/Filter /FlateDecode
>>
stream
x IsMaĀ "BBLf2Y/X `N?Q6ʚcf/zjg e5 {5R-KyzJjX ȵt jGOoܖ
#뤧Wkofl@5k;@+ up 7d;.k>ݭᷞWmp9rx{ 6@8!#(}n-1$ `gYp
`\pk}&Og֭sÁyp0 N0`هak8/]`!0Vc$$I@$J0 a* otFƳɭz.."AJ) R$HXTRi@@mv{fӉtb:1NL'Ӊtb:1N>DwVf1狪UU wj}hPU;USjU3 ݣf=xTռ˚*RcÂ_
z****ux|Yϯ?!X
endstream
endobj
startxref
180137
%%EOF