lbfgsb3c/0000755000176200001440000000000013627435412011746 5ustar liggesuserslbfgsb3c/NAMESPACE0000644000176200001440000000031113625774746013176 0ustar liggesusers# Generated by roxygen2: do not edit by hand export(lbfgsb3) export(lbfgsb3c) export(lbfgsb3f) export(lbfgsb3x) importFrom(Rcpp,evalCpp) importFrom(methods,is) useDynLib(lbfgsb3c, .registration=TRUE) lbfgsb3c/LICENSE.note0000644000176200001440000000304613511470416013715 0ustar liggesusersThe repository for the original Fortran code by Nocedal and Morales states: Condition for Use: This software is freely available, but we expect that all publications describing work using this software , or all commercial products using it, quote at least one of the references given below. This software is released under the "New BSD License" (aka "Modified BSD License" or "3-clause license"). References R. H. Byrd, P. Lu and J. Nocedal. A Limited Memory Algorithm for Bound Constrained Optimization, (1995), SIAM Journal on Scientific and Statistical Computing , 16, 5, pp. 1190-1208. C. Zhu, R. H. Byrd and J. Nocedal. L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization (1997), ACM Transactions on Mathematical Software, Vol 23, Num. 4, pp. 550 - 560. J.L. Morales and J. Nocedal. L-BFGS-B: Remark on Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization (2011), ACM Transactions on Mathematical Software, Vol 38, Num. 1, pp. 7:1 - 7:4. Minor modifications to this code were made by John C Nash to allow it to work within R. This code can be used under the same BSD 3-clause license. The wrapper code and other materials in the package are released under GPL-2 license. All have been written by John C Nash. Help of Berend Hasselman is gratefully acknowledged. Minor modifications to the code were also made by Matthew Fidler to allow direct C calling and some additional optimiation options. These were relaesed under GPL-2.lbfgsb3c/README.md0000644000176200001440000000143713511470416013225 0ustar liggesusers[![Travis-CI Build Status](https://travis-ci.org/nlmixrdevelopment/lbfgsb3c.svg?branch=master)](https://travis-ci.org/nlmixrdevelopment/lbfgsb3c) [![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/nlmixrdevelopment/lbfgsb3c?branch=master&svg=true)](https://ci.appveyor.com/project/nlmixrdevelopment/lbfgsb3c) # libfgsb3c interface from C This is the fork of the libfgsb3 from cran with the following differences: - The return type has changed is is very similar to what `optim` returns - Allows a direct C/C++ interface through a R registered function, similar to C interface to `optim` with 2 additional arguments. - Allows adjustment of tolerances for minimization success. - Added `xtolAtol` and `xtolRtol` minimization success criterion. - Added `maxit` termination lbfgsb3c/man/0000755000176200001440000000000013625551253012521 5ustar liggesuserslbfgsb3c/man/lbfgsb3c.Rd0000644000176200001440000001620613625774410014504 0ustar liggesusers% Generated by roxygen2: do not edit by hand % Please edit documentation in R/lbfgsb3.R \name{lbfgsb3c} \alias{lbfgsb3c} \alias{lbfgsb3} \alias{lbfgsb3f} \alias{lbfgsb3x} \title{Interfacing wrapper for the Nocedal - Morales LBFGSB3 (Fortran) limited memory BFGS solver.} \usage{ lbfgsb3c( par, fn, gr = NULL, lower = -Inf, upper = Inf, control = list(), ..., rho = NULL ) lbfgsb3( par, fn, gr = NULL, lower = -Inf, upper = Inf, control = list(), ..., rho = NULL ) lbfgsb3f( par, fn, gr = NULL, lower = -Inf, upper = Inf, control = list(), ..., rho = NULL ) lbfgsb3x( par, fn, gr = NULL, lower = -Inf, upper = Inf, control = list(), ..., rho = NULL ) } \arguments{ \item{par}{A parameter vector which gives the initial guesses to the parameters that will minimize \code{fn}. This can be named, for example, we could use par=c(b1=1, b2=2.345, b3=0.123)} \item{fn}{A function that evaluates the objective function to be minimized. This can be a R function or a Rcpp function pointer.} \item{gr}{If present, a function that evaluates the gradient vector for the objective function at the given parameters computing the elements of the sum of squares function at the set of parameters \code{start}. This can be a R function or a Rcpp function pointer.} \item{lower}{Lower bounds on the parameters. If a single number, this will be applied to all parameters. Default -Inf.} \item{upper}{Upper bounds on the parameters. If a single number, this will be applied to all parameters. Default Inf.} \item{control}{An optional list of control settings. See below in details.} \item{...}{Any data needed for computation of the objective function and gradient.} \item{rho}{An Environment to use for function evaluation. If present the arguments in ... are ignored. Otherwise the ... are converted to an environment for evaluation.} } \value{ A list of the following items \itemize{ \item{par} The best set of parameters found. \item{value} The value of fn corresponding to par. \item{counts} A two-element integer vector giving the number of calls to fn and gr respectively. This excludes any calls to fn to compute a finite-difference approximation to the gradient. \item{convergence} An integer code. 0 indicates successful completion } } \description{ Interfacing wrapper for the Nocedal - Morales LBFGSB3 (Fortran) limited memory BFGS solver. } \details{ See the notes below for a general appreciation of this package. The control list can contain: \itemize{ \item{trace} If positive, tracing information on the progress of the optimization is produced. Higher values may produce more tracing information: for method "L-BFGS-B" there are six levels of tracing. (To understand exactly what these do see the source code: higher levels give more detail.) \item{factr} controls the convergence of the "L-BFGS-B" method. Convergence occurs when the reduction in the objective is within this factor of the machine tolerance. Default is 1e7, that is a tolerance of about 1e-8. \item{pgtol} helps control the convergence of the "L-BFGS-B" method. It is a tolerance on the projected gradient in the current search direction. This defaults to zero, when the check is suppressed. \item{abstol} helps control the convergence of the "L-BFGS-B" method. It is an absolute tolerance difference in x values. This defaults to zero, when the check is suppressed. \item{reltol} helps control the convergence of the "L-BFGS-B" method. It is an relative tolerance difference in x values. This defaults to zero, when the check is suppressed. \item{lmm} is an integer giving the number of BFGS updates retained in the "L-BFGS-B" method, It defaults to 5. \item{maxit} maximum number of iterations. \item{iprint} Provided only for compatibility with older codes. This control is no longer active.) \item{info} a boolean to indicate if more optimization information is captured and output in a $info list } } \note{ This package is a wrapper to the Fortran code released by Nocedal and Morales. This poses several difficulties for an R package. While the \code{.Fortran()} tool exists for the interfacing, we must be very careful to align the arguments with those of the Fortran subroutine, especially in type and storage. A more annoying task for interfacing the Fortran code is that Fortran WRITE or PRINT statements must all be replaced with calls to special R-friendly output routines. Unfortunately, the Fortran is full of output statements. Worse, we may wish to be able to suppress such output, and there are thus many modifications to be made. This means that an update of the original code cannot be simply plugged into the R package \code{src} directory. Finally, and likely because L-BFGS-B has a long history, the Fortran code is far from well-structured. For example, the number of function and gradient evaluations used is returned as the 34'th element of an integer vector. There does not appear to be an easy way to stop the program after some maximum number of such evaluations have been performed. On the other hand, the version of L-BFGS-B in \code{optim()} is a \code{C} translation of a now-lost Fortran code. It does not implement the improvements Nocedal and Morales published in 2011. Hence, despite its deficiencies, this wrapper has been prepared. In addition to the above reasons for the original lbfgsb3 package, this additional package allows C calling of L-BFGS-B 3.0 by a program as well as adjustments to the tolerances that were not present in the original CRAN package. Also adjustments were made to have outputs conform with R's optim routine. } \examples{ # Rosenbrock's banana function n=3; p=100 fr = function(x) { f=1.0 for(i in 2:n) { f=f+p*(x[i]-x[i-1]**2)**2+(1.0-x[i])**2 } f } grr = function(x) { g = double(n) g[1]=-4.0*p*(x[2]-x[1]**2)*x[1] if(n>2) { for(i in 2:(n-1)) { g[i]=2.0*p*(x[i]-x[i-1]**2)-4.0*p*(x[i+1]-x[i]**2)*x[i]-2.0*(1.0-x[i]) } } g[n]=2.0*p*(x[n]-x[n-1]**2)-2.0*(1.0-x[n]) g } x = c(a=1.02, b=1.02, c=1.02) (opc <- lbfgsb3c(x,fr, grr)) (op <- lbfgsb3(x,fr, grr, control=list(trace=1))) (opx <- lbfgsb3x(x,fr, grr)) (opf <- lbfgsb3f(x,fr, grr)) } \references{ Morales, J. L.; Nocedal, J. (2011). "Remark on 'algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization' ". ACM Transactions on Mathematical Software 38: 1. Byrd, R. H.; Lu, P.; Nocedal, J.; Zhu, C. (1995). "A Limited Memory Algorithm for Bound Constrained Optimization". SIAM J. Sci. Comput. 16 (5): 1190-1208. Zhu, C.; Byrd, Richard H.; Lu, Peihuang; Nocedal, Jorge (1997). "L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization". ACM Transactions on Mathematical Software 23 (4): 550-560. } \seealso{ Packages \code{\link{optim}} and \code{optimx}. } \author{ Matthew Fidler (move to C and add more options for adjustments), John C Nash (of the wrapper and edits to Fortran code to allow R output) Ciyou Zhu, Richard Byrd, Jorge Nocedal, Jose Luis Morales (original Fortran packages) } \keyword{nonlinear} \keyword{optimization} \keyword{parameter} lbfgsb3c/DESCRIPTION0000644000176200001440000000347313627435412013463 0ustar liggesusersPackage: lbfgsb3c Type: Package Title: Limited Memory BFGS Minimizer with Bounds on Parameters with optim() 'C' Interface Version: 2020-3.2 Authors@R: c(person(given = c("Matthew", "L"), family = "Fidler", role = c("aut", "cre"), email = "matthew.fidler@gmail.com"), person(given = c("John", "C"), family = "Nash", role = c("aut"), email = "nashjc@uottawa.ca"), person(given = "Ciyou", family = "Zhu", role = "aut"), person(given = "Richard", family = "Byrd", role = "aut"), person(given = "Jorge", family = "Nocedal", role = "aut"), person(given = c("Jose", "Luis"), family = "Morales", role = "aut")) Maintainer: Matthew L Fidler Description: Interfacing to Nocedal et al. L-BFGS-B.3.0 (See ) limited memory BFGS minimizer with bounds on parameters. This is a fork of 'lbfgsb3'. This registers a 'R' compatible 'C' interface to L-BFGS-B.3.0 that uses the same function types and optimization as the optim() function (see writing 'R' extensions and source for details). This package also adds more stopping criteria as well as allowing the adjustment of more tolerances. License: GPL-2 Depends: R (>= 3.0.2) Imports: Rcpp (>= 0.12.3), numDeriv, methods Suggests: optimx, microbenchmark, testthat, knitr, pkgbuild, rmarkdown, RcppArmadillo LinkingTo: Rcpp (>= 0.12.3), RcppArmadillo Encoding: UTF-8 VignetteBuilder: knitr NeedsCompilation: yes RoxygenNote: 7.0.2 Author: Matthew L Fidler [aut, cre], John C Nash [aut], Ciyou Zhu [aut], Richard Byrd [aut], Jorge Nocedal [aut], Jose Luis Morales [aut] Packaged: 2020-03-02 16:38:23 UTC; matt Repository: CRAN Date/Publication: 2020-03-03 11:10:02 UTC lbfgsb3c/build/0000755000176200001440000000000013627233177013051 5ustar liggesuserslbfgsb3c/build/vignette.rds0000644000176200001440000000031113627233177015403 0ustar liggesusersb```b`f@&0rH$''妠 ts 2sRq.HICB t0XXT%igwI-HK î?"5lP5,n _@ ,s\ܠL t7`~΢r=xA$Gs=ʕXVr7Klbfgsb3c/tests/0000755000176200001440000000000013513627745013116 5ustar liggesuserslbfgsb3c/tests/testthat/0000755000176200001440000000000013627435412014750 5ustar liggesuserslbfgsb3c/tests/testthat/test-genrose.R0000644000176200001440000000225713511470416017513 0ustar liggesusersgenrose.f <- function(x, gs = NULL) { # objective function ## One generalization of the Rosenbrock banana valley # function (n parameters) n <- length(x) if (is.null(gs)) { gs = 100 } fval <- 1 + sum(gs * (x[1:(n - 1)]^2 - x[2:n])^2 + (x[2:n] - 1)^2) return(fval) } genrose.g <- function(x, gs = NULL) { # vectorized gradient for genrose.f # Ravi Varadhan 2009-04-03 n <- length(x) if (is.null(gs)) { gs = 100 } gg <- as.vector(rep(0, n)) tn <- 2:n tn1 <- tn - 1 z1 <- x[tn] - x[tn1]^2 z2 <- 1 - x[tn] gg[tn] <- 2 * (gs * z1 - z2) gg[tn1] <- gg[tn1] - 4 * gs * x[tn1] * z1 gg } context("Unconstrained Genrose test with gradient") nn <- 100 xx <- rep(3, nn) lo <- -Inf up <- Inf test_that("100u", { ans100u <- lbfgsb3c(xx, genrose.f, genrose.g, gs = 10) expect_equal(1, ans100u$value) }) context("Unconstrained Genrose test without gradient") ans100un <- lbfgsb3c(xx, genrose.f, gr = NULL, gs = 10) test_that("100u", { ans100un <- lbfgsb3c(xx, genrose.f, gs = 10) expect_equal(1, ans100un$value) }) lbfgsb3c/tests/testthat/test-bounds.R0000644000176200001440000000104713511470416017337 0ustar liggesuserscontext("Bounds") test_that("bounds test", { bt.f<-function(x){ sum(x*x) } bt.g<-function(x){ gg<-2.0*x } n<-4 lower<-rep(0,n) upper<-lower ## to get arrays set ## bdmsk<-rep(1,n) ## bdmsk[(trunc(n/2)+1)]<-0 for (i in 1:n) { lower[i]<-1.0*(i-1)*(n-1)/n upper[i]<-1.0*i*(n+1)/n } xx<-0.5*(lower+upper) abt <- lbfgsb3c(xx, bt.f, bt.g, lower=lower, upper=upper, control=list(trace=0, iprint=0)) expect_equal(abt$par, c(0, 0.75, 1.5, 2.25)) }) lbfgsb3c/tests/testthat/test-cyq.R0000644000176200001440000000605113511470416016641 0ustar liggesusers# Ref: Fletcher, R. (1965) Function minimization without # calculating derivatives -- a review, # Computer J., 8, 33-41. # Note we do not have all components here e.g., .jsd, .h ## context("Fletcher's Chebyquad problem") cyq.f <- function(x) { rv <- cyq.res(x) f <- sum(rv * rv) } cyq.res <- function(x) { # Fletcher's chebyquad function m = n -- residuals n <- length(x) res <- rep(0, n) # initialize for (i in 1:n) { #loop over resids rr <- 0 for (k in 1:n) { z7 <- 1 z2 <- 2 * x[k] - 1 z8 <- z2 j <- 1 while (j < i) { z6 <- z7 z7 <- z8 z8 <- 2 * z2 * z7 - z6 # recurrence to compute Chebyshev polynomial j <- j + 1 } # end recurrence loop rr <- rr + z8 } # end loop on k rr <- rr/n if (2 * trunc(i/2) == i) { rr <- rr + 1/(i * i - 1) } res[i] <- rr } # end loop on i res } cyq.jac <- function(x) { # Chebyquad Jacobian matrix n <- length(x) cj <- matrix(0, n, n) for (i in 1:n) { # loop over rows for (k in 1:n) { # loop over columns (parameters) z5 <- 0 cj[i, k] <- 2 z8 <- 2 * x[k] - 1 z2 <- z8 z7 <- 1 j <- 1 while (j < i) { # recurrence loop z4 <- z5 z5 <- cj[i, k] cj[i, k] <- 4 * z8 + 2 * z2 * z5 - z4 z6 <- z7 z7 <- z8 z8 <- 2 * z2 * z7 - z6 j <- j + 1 } # end recurrence loop cj[i, k] <- cj[i, k]/n } # end loop on k } # end loop on i cj } cyq.g <- function(x) { cj <- cyq.jac(x) rv <- cyq.res(x) gg <- as.vector(2 * rv %*% cj) } # nn <- c(2, 3, 5, 8, 10, 20, 30) nn <- c(2, 3, 5, 8) for (n in nn) { str <- paste0("Chebyquad in ", n, " parameters"); context(str) test_that(str, { lower <- rep(-10, n) upper <- rep(10, n) bdmsk <- rep(1, n) # free all parameters x0 <- 1:n x0 <- x0/(n + 1) # Initial value suggested by Fletcher ans <- lbfgsb3c(x0, cyq.f, cyq.g, lower, upper, control = list()) if (n == 2){ expect_equal(c(0.211, 0.789), round(ans$par, 3)) expect_equal(0, round(ans$value, 3)) } else if (n == 3){ expect_equal(c(0.146, 0.5, 0.854), round(ans$par, 3)) expect_equal(0, round(ans$value, 3)) } else if (n == 5){ expect_equal(c(0.084, 0.313, 0.5, 0.687, 0.916), round(ans$par, 3)) expect_equal(0, round(ans$value, 3)) } else if (n == 8){ expect_equal(c(0.043, 0.193, 0.266, 0.5, 0.5, 0.734, 0.807, 0.957), round(ans$par, 3)) expect_equal(0.004, round(ans$value, 3)) } }) } lbfgsb3c/tests/testthat.R0000644000176200001440000000007413511470416015067 0ustar liggesuserslibrary(testthat) library(lbfgsb3c) test_check("lbfgsb3c") lbfgsb3c/src/0000755000176200001440000000000013627233177012541 5ustar liggesuserslbfgsb3c/src/blas.f0000644000176200001440000001425013511470416013622 0ustar liggesusersc c L-BFGS-B is released under the “New BSD License” (aka “Modified BSD License” c or “3-clause license”) c Please read attached file License.txt c double precision function dnrm2(n,x,incx) integer n,incx double precision x(n) c ********** c c Function dnrm2 c c Given a vector x of length n, this function calculates the c Euclidean norm of x with stride incx. c c The function statement is c c double precision function dnrm2(n,x,incx) c c where c c n is a positive integer input variable. c c x is an input array of length n. c c incx is a positive integer variable that specifies the c stride of the vector. c c Subprograms called c c FORTRAN-supplied ... abs, max, sqrt c c MINPACK-2 Project. February 1991. c Argonne National Laboratory. c Brett M. Averick. c c ********** integer i double precision scale dnrm2 = 0.0d0 scale = 0.0d0 do 10 i = 1, n, incx scale = max(scale, abs(x(i))) 10 continue if (scale .eq. 0.0d0) return do 20 i = 1, n, incx dnrm2 = dnrm2 + (x(i)/scale)**2 20 continue dnrm2 = scale*sqrt(dnrm2) return end c====================== The end of dnrm2 =============================== subroutine daxpy(n,da,dx,incx,dy,incy) c c constant times a vector plus a vector. c uses unrolled loops for increments equal to one. c jack dongarra, linpack, 3/11/78. c double precision dx(*),dy(*),da integer i,incx,incy,ix,iy,m,mp1,n c if(n.le.0)return if (da .eq. 0.0d0) return if(incx.eq.1.and.incy.eq.1)go to 20 c c code for unequal increments or equal increments c not equal to 1 c ix = 1 iy = 1 if(incx.lt.0)ix = (-n+1)*incx + 1 if(incy.lt.0)iy = (-n+1)*incy + 1 do 10 i = 1,n dy(iy) = dy(iy) + da*dx(ix) ix = ix + incx iy = iy + incy 10 continue return c c code for both increments equal to 1 c c c clean-up loop c 20 m = mod(n,4) if( m .eq. 0 ) go to 40 do 30 i = 1,m dy(i) = dy(i) + da*dx(i) 30 continue if( n .lt. 4 ) return 40 mp1 = m + 1 do 50 i = mp1,n,4 dy(i) = dy(i) + da*dx(i) dy(i + 1) = dy(i + 1) + da*dx(i + 1) dy(i + 2) = dy(i + 2) + da*dx(i + 2) dy(i + 3) = dy(i + 3) + da*dx(i + 3) 50 continue return end c====================== The end of daxpy =============================== subroutine dcopy(n,dx,incx,dy,incy) c c copies a vector, x, to a vector, y. c uses unrolled loops for increments equal to one. c jack dongarra, linpack, 3/11/78. c double precision dx(*),dy(*) integer i,incx,incy,ix,iy,m,mp1,n c if(n.le.0)return if(incx.eq.1.and.incy.eq.1)go to 20 c c code for unequal increments or equal increments c not equal to 1 c ix = 1 iy = 1 if(incx.lt.0)ix = (-n+1)*incx + 1 if(incy.lt.0)iy = (-n+1)*incy + 1 do 10 i = 1,n dy(iy) = dx(ix) ix = ix + incx iy = iy + incy 10 continue return c c code for both increments equal to 1 c c c clean-up loop c 20 m = mod(n,7) if( m .eq. 0 ) go to 40 do 30 i = 1,m dy(i) = dx(i) 30 continue if( n .lt. 7 ) return 40 mp1 = m + 1 do 50 i = mp1,n,7 dy(i) = dx(i) dy(i + 1) = dx(i + 1) dy(i + 2) = dx(i + 2) dy(i + 3) = dx(i + 3) dy(i + 4) = dx(i + 4) dy(i + 5) = dx(i + 5) dy(i + 6) = dx(i + 6) 50 continue return end c====================== The end of dcopy =============================== double precision function ddot(n,dx,incx,dy,incy) c c forms the dot product of two vectors. c uses unrolled loops for increments equal to one. c jack dongarra, linpack, 3/11/78. c double precision dx(*),dy(*),dtemp integer i,incx,incy,ix,iy,m,mp1,n c ddot = 0.0d0 dtemp = 0.0d0 if(n.le.0)return if(incx.eq.1.and.incy.eq.1)go to 20 c c code for unequal increments or equal increments c not equal to 1 c ix = 1 iy = 1 if(incx.lt.0)ix = (-n+1)*incx + 1 if(incy.lt.0)iy = (-n+1)*incy + 1 do 10 i = 1,n dtemp = dtemp + dx(ix)*dy(iy) ix = ix + incx iy = iy + incy 10 continue ddot = dtemp return c c code for both increments equal to 1 c c c clean-up loop c 20 m = mod(n,5) if( m .eq. 0 ) go to 40 do 30 i = 1,m dtemp = dtemp + dx(i)*dy(i) 30 continue if( n .lt. 5 ) go to 60 40 mp1 = m + 1 do 50 i = mp1,n,5 dtemp = dtemp + dx(i)*dy(i) + dx(i + 1)*dy(i + 1) + * dx(i + 2)*dy(i + 2) + dx(i + 3)*dy(i + 3) + dx(i + 4)*dy(i + 4) 50 continue 60 ddot = dtemp return end c====================== The end of ddot ================================ subroutine dscal(n,da,dx,incx) c c scales a vector by a constant. c uses unrolled loops for increment equal to one. c jack dongarra, linpack, 3/11/78. c modified 3/93 to return if incx .le. 0. c double precision da,dx(*) integer i,incx,m,mp1,n,nincx c if( n.le.0 .or. incx.le.0 )return if(incx.eq.1)go to 20 c c code for increment not equal to 1 c nincx = n*incx do 10 i = 1,nincx,incx dx(i) = da*dx(i) 10 continue return c c code for increment equal to 1 c c c clean-up loop c 20 m = mod(n,5) if( m .eq. 0 ) go to 40 do 30 i = 1,m dx(i) = da*dx(i) 30 continue if( n .lt. 5 ) return 40 mp1 = m + 1 do 50 i = mp1,n,5 dx(i) = da*dx(i) dx(i + 1) = da*dx(i + 1) dx(i + 2) = da*dx(i + 2) dx(i + 3) = da*dx(i + 3) dx(i + 4) = da*dx(i + 4) 50 continue return end c====================== The end of dscal =============================== lbfgsb3c/src/init.c0000644000176200001440000000161113511470416013636 0ustar liggesusers#include "../inst/include/lbfgsb3c.h" #include #include #include // for NULL #include void lbfgsb3C_(int n, int lmm, double *x, double *lower, double *upper, int *nbd, double *Fmin, optimfn fn, optimgr gr, int *fail, void *ex, double factr, double pgtol, int *fncount, int *grcount, int maxit, char *msg, int trace, int iprint, double atol, double rtol, double *g); SEXP _lbfgsb3c_lbfgsb3cpp(SEXP, SEXP, SEXP, SEXP, SEXP, SEXP, SEXP); void R_init_lbfgsb3c(DllInfo *info){ R_CallMethodDef callMethods[] = { {"_lbfgsb3c_lbfgsb3cpp", (DL_FUNC) &_lbfgsb3c_lbfgsb3cpp, 7}, {NULL, NULL, 0} }; // C callable to assign environments. R_RegisterCCallable("lbfgsb3c", "lbfgsb3C_", (DL_FUNC) lbfgsb3C_); R_registerRoutines(info, NULL, callMethods, NULL, NULL); R_useDynamicSymbols(info, FALSE); } lbfgsb3c/src/lbfgsb.f0000755000176200001440000042346313625551177014167 0ustar liggesusersc c L-BFGS-B is released under the "New BSD License" (aka "Modified BSD License" c or "3-clause license") c Please read attached file License.txt c c=========== L-BFGS-B (version 3.0. April 25, 2011 =================== c c This is a modified version of L-BFGS-B. Minor changes in the updated c code appear preceded by a line comment as follows c c c-jlm-jn c c Major changes are described in the accompanying paper: c c Jorge Nocedal and Jose Luis Morales, Remark on "Algorithm 778: c L-BFGS-B: Fortran Subroutines for Large-Scale Bound Constrained c Optimization" (2011). To appear in ACM Transactions on c Mathematical Software, c c The paper describes an improvement and a correction to Algorithm 778. c It is shown that the performance of the algorithm can be improved c significantly by making a relatively simple modication to the subspace c minimization phase. The correction concerns an error caused by the use c of routine dpmeps to estimate machine precision. c c The total work space **wa** required by the new version is c c 2*m*n + 11m*m + 5*n + 8*m c c the old version required c c 2*m*n + 12m*m + 4*n + 12*m c c c J. Nocedal Department of Electrical Engineering and c Computer Science. c Northwestern University. Evanston, IL. USA c c c J.L Morales Departamento de Matematicas, c Instituto Tecnologico Autonomo de Mexico c Mexico D.F. Mexico. c c March 2011 c c============================================================================= c JN 20150118 change name setulb to lbfgsb3 c MLF 20180818 changed name back lbfgsb3 in c wrapper subroutine setulb(n, m, x, l, u, nbd, f, g, factr, pgtol, wa, + iwa, itask, iprint, icsave, lsavei, isave, dsave) c Berend noted earlier format beyond column 72 integer lsavei(4) integer n, m, iprint, itask, icsave, + nbd(n), iwa(3*n), isave(44) double precision f, factr, pgtol, x(n), l(n), u(n), g(n), c c-jlm-jn + wa(2*m*n + 5*n + 11*m*m + 8*m), dsave(29) logical lsave(4) c ************ c c Subroutine setulb c c This subroutine partitions the working arrays wa and iwa, and c then uses the limited memory BFGS method to solve the bound c constrained optimization problem by calling mainlb. c (The direct method will be used in the subspace minimization.) c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c x is a double precision array of dimension n. c On entry x is an approximation to the solution. c On exit x is the current approximation. c c l is a double precision array of dimension n. c On entry l is the lower bound on x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound on x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c f is a double precision variable. c On first entry f is unspecified. c On final exit f is the value of the function at x. c c g is a double precision array of dimension n. c On first entry g is unspecified. c On final exit g is the value of the gradient at x. c c factr is a double precision variable. c On entry factr >= 0 is specified by the user. The iteration c will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision, which is automatically c generated by the code. Typical values for factr: 1.d+12 for c low accuracy; 1.d+7 for moderate accuracy; 1.d+1 for extremely c high accuracy. c On exit factr is unchanged. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c On exit pgtol is unchanged. c c wa is a double precision working array of length c (2mmax + 5)nmax + 12mmax^2 + 12mmax. c c iwa is an integer working array of length 3nmax. c c itask is an integer indicating c the current job when entering and quitting this subroutine. c c iprint is an integer variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c icsave is a working integer c c lsave is a logical working array of dimension 4. c On exit with 'task' = NEW_X, the following information is c available: c If lsave(1) = .true. then the initial X has been replaced by c its projection in the feasible set; c If lsave(2) = .true. then the problem is constrained; c If lsave(3) = .true. then each variable has upper and lower c bounds; c c isave is an integer working array of dimension 44. c On exit with 'task' = NEW_X, the following information is c available: c isave(22) = the total number of intervals explored in the c search of Cauchy points; c isave(26) = the total number of skipped BFGS updates before c the current iteration; c isave(30) = the number of current iteration; c isave(31) = the total number of BFGS updates prior the current c iteration; c isave(33) = the number of intervals explored in the search of c Cauchy point in the current iteration; c isave(34) = the total number of function and gradient c evaluations; c isave(36) = the number of function value or gradient c evaluations in the current iteration; c if isave(37) = 0 then the subspace argmin is within the box; c if isave(37) = 1 then the subspace argmin is beyond the box; c isave(38) = the number of free variables in the current c iteration; c isave(39) = the number of active constraints in the current c iteration; c n + 1 - isave(40) = the number of variables leaving the set of c active constraints in the current iteration; c isave(41) = the number of variables entering the set of active c constraints in the current iteration. c c dsave is a double precision working array of dimension 29. c On exit with 'task' = NEW_X, the following information is c available: c dsave(1) = current 'theta' in the BFGS matrix; c dsave(2) = f(x) in the previous iteration; c dsave(3) = factr*epsmch; c dsave(4) = 2-norm of the line search direction vector; c dsave(5) = the machine precision epsmch generated by the code; c dsave(7) = the accumulated time spent on searching for c Cauchy points; c dsave(8) = the accumulated time spent on c subspace minimization; c dsave(9) = the accumulated time spent on line search; c dsave(11) = the slope of the line search function at c the current point of line search; c dsave(12) = the maximum relative step length imposed in c line search; c dsave(13) = the infinity norm of the projected gradient; c dsave(14) = the relative step length in the line search; c dsave(15) = the slope of the line search function at c the starting point of the line search; c dsave(16) = the square of the 2-norm of the line search c direction vector. c c Subprograms called: c c L-BFGS-B Library ... mainlb. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: a c limited memory FORTRAN code for solving bound constrained c optimization problems'', Tech. Report, NAM-11, EECS Department, c Northwestern University, 1994. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ c-jlm-jn integer lws,lr,lz,lt,ld,lxp,lwa, + lwy,lsy,lss,lwt,lwn,lsnd if (lsavei(1) .eq. 1) then lsave(1) = .true. else lsave(1) = .false. endif if (lsavei(2) .eq. 1) then lsave(2) = .true. else lsave(2) = .false. endif if (lsavei(3) .eq. 1) then lsave(3) = .true. else lsave(3) = .false. endif if (lsavei(4) .eq. 1) then lsave(4) = .true. else lsave(4) = .false. endif cj integer ia(10) cj ia(1) = itask cj call intpr(' The incoming task no. is ', -1, itask, 1) cw write(6,*) ' The incoming task no. is ', itask if ((itask .lt. 1) .or. (itask .gt. 26)) then cx call intpr("TASK NOT IN VALID RANGE", -1, 0,0) itask = -999 return endif c if (task .eq. 'START') then if (itask .eq. 2) then isave(1) = m*n isave(2) = m**2 isave(3) = 4*m**2 isave(4) = 1 ! ws m*n isave(5) = isave(4) + isave(1) ! wy m*n isave(6) = isave(5) + isave(1) ! wsy m**2 isave(7) = isave(6) + isave(2) ! wss m**2 isave(8) = isave(7) + isave(2) ! wt m**2 isave(9) = isave(8) + isave(2) ! wn 4*m**2 isave(10) = isave(9) + isave(3) ! wsnd 4*m**2 isave(11) = isave(10) + isave(3) ! wz n isave(12) = isave(11) + n ! wr n isave(13) = isave(12) + n ! wd n isave(14) = isave(13) + n ! wt n isave(15) = isave(14) + n ! wxp n isave(16) = isave(15) + n ! wa 8*m endif lws = isave(4) lwy = isave(5) lsy = isave(6) lss = isave(7) lwt = isave(8) lwn = isave(9) lsnd = isave(10) lz = isave(11) lr = isave(12) ld = isave(13) lt = isave(14) lxp = isave(15) lwa = isave(16) call mainlb(n,m,x,l,u,nbd,f,g,factr,pgtol, + wa(lws),wa(lwy),wa(lsy),wa(lss), wa(lwt), + wa(lwn),wa(lsnd),wa(lz),wa(lr),wa(ld),wa(lt),wa(lxp), + wa(lwa), + iwa(1),iwa(n+1),iwa(2*n+1),itask,iprint, + icsave,lsave,isave(22),dsave) return end c======================= The end of setulb ============================= c mainlb here subroutine mainlb(n, m, x, l, u, nbd, f, g, factr, pgtol, ws, wy, + sy, ss, wt, wn, snd, z, r, d, t, xp, wa, + index, iwhere, indx2, itask, + iprint, icsave, lsave, isave, dsave) implicit none logical lsave(4) integer n, m, iprint, itask, icsave, nbd(n), index(n), + iwhere(n), indx2(n), isave(23) double precision f, factr, pgtol, + x(n), l(n), u(n), g(n), z(n), r(n), d(n), t(n), c-jlm-jn + xp(n), + wa(8*m), + ws(n, m), wy(n, m), sy(m, m), ss(m, m), + wt(m, m), wn(2*m, 2*m), snd(2*m, 2*m), dsave(29) c ************ c c Subroutine mainlb c c This subroutine solves bound constrained optimization problems by c using the compact formula of the limited memory BFGS updates. c c n is an integer variable. c On entry n is the number of variables. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric c corrections allowed in the limited memory matrix. c On exit m is unchanged. c c x is a double precision array of dimension n. c On entry x is an approximation to the solution. c On exit x is the current approximation. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c f is a double precision variable. c On first entry f is unspecified. c On final exit f is the value of the function at x. c c g is a double precision array of dimension n. c On first entry g is unspecified. c On final exit g is the value of the gradient at x. c c factr is a double precision variable. c On entry factr >= 0 is specified by the user. The iteration c will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision, which is automatically c generated by the code. c On exit factr is unchanged. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c On exit pgtol is unchanged. c c ws, wy, sy, and wt are double precision working arrays used to c store the following information defining the limited memory c BFGS matrix: c ws, of dimension n x m, stores S, the matrix of s-vectors; c wy, of dimension n x m, stores Y, the matrix of y-vectors; c sy, of dimension m x m, stores S'Y; c ss, of dimension m x m, stores S'S; c yy, of dimension m x m, stores Y'Y; c wt, of dimension m x m, stores the Cholesky factorization c of (theta*S'S+LD^(-1)L'); see eq. c (2.26) in [3]. c c wn is a double precision working array of dimension 2m x 2m c used to store the LEL^T factorization of the indefinite matrix c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c where E = [-I 0] c [ 0 I] c c snd is a double precision working array of dimension 2m x 2m c used to store the lower triangular part of c N = [Y' ZZ'Y L_a'+R_z'] c [L_a +R_z S'AA'S ] c c z(n),r(n),d(n),t(n), xp(n),wa(8*m) are double precision working arrays. c z is used at different times to store the Cauchy point and c the Newton point. c xp is used to safeguard the projected Newton direction c c sg(m),sgo(m),yg(m),ygo(m) are double precision working arrays. c c index is an integer working array of dimension n. c In subroutine freev, index is used to store the free and fixed c variables at the Generalized Cauchy Point (GCP). c c iwhere is an integer working array of dimension n used to record c the status of the vector x for GCP computation. c iwhere(i)=0 or -3 if x(i) is free and has bounds, c 1 if x(i) is fixed at l(i), and l(i) .ne. u(i) c 2 if x(i) is fixed at u(i), and u(i) .ne. l(i) c 3 if x(i) is always fixed, i.e., u(i)=x(i)=l(i) c -1 if x(i) is always free, i.e., no bounds on it. c c indx2 is an integer working array of dimension n. c Within subroutine cauchy, indx2 corresponds to the array iorder. c In subroutine freev, a list of variables entering and leaving c the free set is stored in indx2, and it is passed on to c subroutine formk with this information. c c task is a working string of characters of length 60 indicating c the current job when entering and leaving this subroutine. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c icsave is a working integer c c lsave is a logical working array of dimension 4. c c isave is an integer working array of dimension 23. c c dsave is a double precision working array of dimension 29. c c c Subprograms called c c L-BFGS-B Library ... cauchy, subsm, lnsrlb, formk, c c errclb, prn1lb, prn2lb, prn3lb, active, projgr, c c freev, cmprlb, matupd, formt. c c Minpack2 Library ... timer c c Linpack Library ... dcopy, ddot. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c [3] R. Byrd, J. Nocedal and R. Schnabel "Representations of c Quasi-Newton Matrices and their use in Limited Memory Methods'', c Mathematical Programming 63 (1994), no. 4, pp. 129-156. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ logical prjctd,cnstnd,boxed,updatd,wrk integer i,k,nintol,iback,nskip, + head,col,iter,itail,iupdat, + nseg,nfgv,info,ifun, + iword,nfree,nact,ileave,nenter cj itmp for use in R output integer itmp double precision theta,fold,ddot,dr,rr,tol, + xstep,sbgnrm,ddum,dnorm,dtd,epsmch, + cpu1,cpu2,sbtime,lnscht,time1,time2, + gd,gdold,stp,stpmx,time double precision one,zero parameter (one=1.0d0,zero=0.0d0) if (itask .eq. 2) then epsmch = epsilon(one) cj Arbitrarily zero these. time1 = 0.0 c Initialize counters and scalars when task='START'. c for the limited memory BFGS matrices: col = 0 head = 1 theta = one iupdat = 0 updatd = .false. iback = 0 itail = 0 iword = 0 nact = 0 ileave = 0 nenter = 0 fold = zero dnorm = zero cpu1 = zero gd = zero stpmx = zero sbgnrm = zero stp = zero gdold = zero dtd = zero c for operation counts: iter = 0 nfgv = 0 nseg = 0 nintol = 0 nskip = 0 nfree = n ifun = 0 c for stopping tolerance: tol = factr*epsmch c for measuring running time: cy cachyt = 0 sbtime = 0 lnscht = 0 c 'word' records the status of subspace solutions. c$$$ word = '---' c 'info' records the termination information. info = 0 cw itfile = 8 cw leave itfile to avoid messing up subroutines cw if (iprint .ge. 1) then c open a summary file 'iterate.dat' cw open (8, file = 'iterate.dat', status = 'unknown') cw endif c Check the input arguments for errors. call errclb(n,m,factr,l,u,nbd,itask,info,k) c ERROR return if ((itask .ge. 9) .and. (itask .le. 19)) then call prn3lb(n,x,f,itask,iprint,info,k) return endif call prn1lb(n,m,l,u,x,iprint,epsmch) c Initialize iwhere & project x onto the feasible set. call active(n,l,u,nbd,x,iwhere,iprint,prjctd,cnstnd,boxed) c The end of the initialization. else c restore local variables. prjctd = lsave(1) cnstnd = lsave(2) boxed = lsave(3) updatd = lsave(4) nintol = isave(1) cy itfile = isave(3) iback = isave(4) nskip = isave(5) head = isave(6) col = isave(7) itail = isave(8) iter = isave(9) iupdat = isave(10) nseg = isave(12) nfgv = isave(13) info = isave(14) ifun = isave(15) iword = isave(16) nfree = isave(17) nact = isave(18) ileave = isave(19) nenter = isave(20) theta = dsave(1) fold = dsave(2) tol = dsave(3) dnorm = dsave(4) epsmch = dsave(5) cpu1 = dsave(6) cy cachyt = dsave(7) sbtime = dsave(8) lnscht = dsave(9) time1 = dsave(10) gd = dsave(11) stpmx = dsave(12) sbgnrm = dsave(13) stp = dsave(14) gdold = dsave(15) dtd = dsave(16) c After returning from the driver go to the point where execution c is to resume. c if (task(1:5) .eq. 'FG_LN') goto 666 if (itask .eq. 20) goto 666 c if (task(1:5) .eq. 'NEW_X') goto 777 if (itask .eq. 1) goto 777 c if (task(1:5) .eq. 'FG_ST') goto 111 if (itask .eq. 21) goto 111 c if (task(1:4) .eq. 'STOP') then c if (itask .eq. 3) then c if (task(7:9) .eq. 'CPU') then c NOTE: Cannot happen in R driver. c restore the previous iterate. c call dcopy(n,t,1,x,1) c call dcopy(n,r,1,g,1) c f = fold c endif c goto 999 c endif endif c Compute f0 and g0. c task = 'FG_START' itask = 21 c return to the driver to calculate f and g; reenter at 111. goto 1000 111 continue nfgv = 1 c Compute the infinity norm of the (-) projected gradient. call projgr(n,l,u,nbd,x,g,sbgnrm) cx if (iprint .ge. 1) then cx call intpr("At iterate", -1, iter, 1) cx call dblepr(" f=",-1, f, 1) cx call dblepr(" |proj g|= ",-1, sbgnrm, 1) cx endif if (sbgnrm .le. pgtol) then c terminate the algorithm. c task = 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL' itask = 7 goto 999 endif c ----------------- the beginning of the loop -------------------------- 222 continue if (iprint .ge. 99) then itmp = iter + 1 cx call intpr('ITERATION ', -1, itmp, 1) endif iword = -1 c if (.not. cnstnd .and. col .gt. 0) then c skip the search for GCP. call dcopy(n,x,1,z,1) wrk = updatd nseg = 0 goto 333 endif cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Compute the Generalized Cauchy Point (GCP). c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc cj call timer(cpu1) cpu1 = 0.0d0 call cauchy(n,x,l,u,nbd,g,indx2,iwhere,t,d,z, + m,wy,ws,sy,wt,theta,col,head, + wa(1),wa(2*m+1),wa(4*m+1),wa(6*m+1),nseg, + iprint, sbgnrm, info, epsmch) if (info .ne. 0) then c singular triangular system detected; refresh the lbfgs memory. cx call intpr(' Singular triangular system detected;', -1,0,0) cx call intpr( cx + ' refresh the lbfgs memory and restart the iteration.', cx + -1,0,0) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. cpu2 = 0.0d0 cy cachyt = cachyt + cpu2 - cpu1 goto 222 endif cpu2 = 0.0d0 cy cachyt = cachyt + cpu2 - cpu1 nintol = nintol + nseg c Count the entering and leaving variables for iter > 0; c find the index set of free and active variables at the GCP. call freev(n,nfree,index,nenter,ileave,indx2, + iwhere,wrk,updatd,cnstnd,iprint,iter) nact = n - nfree 333 continue c If there are no free variables or B=theta*I, then c skip the subspace minimization. if (nfree .eq. 0 .or. col .eq. 0) goto 555 cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Subspace minimization. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc cpu1 = 0.0d0 c Form the LEL^T factorization of the indefinite c matrix K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] if (wrk) call formk(n,nfree,index,nenter,ileave,indx2,iupdat, + updatd,wn,snd,m,ws,wy,sy,theta,col,head,info) if (info .ne. 0) then c nonpositive definiteness in Cholesky factorization; c refresh the lbfgs memory and restart the iteration. cw if(iprint .ge. 1) write (6, 1006) if(iprint .ge. 1) then cx call intpr( cx + ' Nonpositive definiteness in Cholesky factorization in formk;', cx + -1, 0, 0) cx call intpr( cx + ' refresh the lbfgs memory and restart the iteration.', cx + -1, 0, 0) endif info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. cj call timer(cpu2) cpu2 = 0.0d0 sbtime = sbtime + cpu2 - cpu1 goto 222 endif c compute r=-Z'B(xcp-xk)-Z'g (using wa(2m+1)=W'(xcp-x) c from 'cauchy'). call cmprlb(n,m,x,g,ws,wy,sy,wt,z,r,wa,index, + theta,col,head,nfree,cnstnd,info) if (info .ne. 0) goto 444 c-jlm-jn call the direct method. call subsm( n, m, nfree, index, l, u, nbd, z, r, xp, ws, wy, + theta, x, g, col, head, iword, wa, wn, iprint, info) 444 continue if (info .ne. 0) then c singular triangular system detected; c refresh the lbfgs memory and restart the iteration. cw if(iprint .ge. 1) write (6, 1005) cx call intpr(' Singular triangular system detected;', -1,0,0) cx call intpr( cx + ' refresh the lbfgs memory and restart the iteration.', cx + -1,0,0) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. cj call timer(cpu2) cpu2 = 0.0d0 sbtime = sbtime + cpu2 - cpu1 goto 222 endif cj call timer(cpu2) cpu2 = 0.0d0 sbtime = sbtime + cpu2 - cpu1 555 continue cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Line search and optimality tests. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c Generate the search direction d:=z-x. do 40 i = 1, n d(i) = z(i) - x(i) 40 continue cj call timer(cpu1) cpu1 = 0.0d0 666 continue call lnsrlb(n,l,u,nbd,x,f,fold,gd,gdold,g,d,r,t,z,stp,dnorm, + dtd,xstep,stpmx,iter,ifun,iback,nfgv,info,itask, + boxed,cnstnd,icsave,isave(22),dsave(17)) if (info .ne. 0 .or. iback .ge. 20) then c restore the previous iterate. call dcopy(n,t,1,x,1) call dcopy(n,r,1,g,1) f = fold if (col .eq. 0) then c abnormal termination. if (info .eq. 0) then info = -9 c restore the actual number of f and g evaluations etc. nfgv = nfgv - 1 ifun = ifun - 1 iback = iback - 1 endif c task = 'ABNORMAL_TERMINATION_IN_LNSRCH' itask = 5 iter = iter + 1 goto 999 else c refresh the lbfgs memory and restart the iteration. cw if(iprint .ge. 1) write (6, 1008) if(iprint .ge. 1) then cx call intpr( cx + ' Bad direction in the line search;', cx + -1, 0, 0) cx call intpr( cx + ' refresh the lbfgs memory and restart the iteration.', cx + -1, 0, 0) endif if (info .eq. 0) nfgv = nfgv - 1 info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. c task = 'RESTART_FROM_LNSRCH' itask = 22 cj call timer(cpu2) cpu2 = 0.0d0 lnscht = lnscht + cpu2 - cpu1 goto 222 endif c else if (task(1:5) .eq. 'FG_LN') then else if (itask .eq. 20) then c return to the driver for calculating f and g; reenter at 666. goto 1000 else c calculate and print out the quantities related to the new X. cj call timer(cpu2) cpu2 = 0.0d0 lnscht = lnscht + cpu2 - cpu1 iter = iter + 1 c Compute the infinity norm of the projected (-)gradient. call projgr(n,l,u,nbd,x,g,sbgnrm) c Print iteration information. call prn2lb(f,iprint,iter, + sbgnrm,iback,xstep) goto 1000 endif 777 continue c Test for termination. if (sbgnrm .le. pgtol) then c terminate the algorithm. c task = 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL' itask = 7 goto 999 endif ddum = max(abs(fold), abs(f), one) if ((fold - f) .le. tol*ddum) then c terminate the algorithm. c task = 'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH' itask = 8 if (iback .ge. 10) info = -5 c i.e., to issue a warning if iback>10 in the line search. goto 999 endif c Compute d=newx-oldx, r=newg-oldg, rr=y'y and dr=y's. do 42 i = 1, n r(i) = g(i) - r(i) 42 continue rr = ddot(n,r,1,r,1) if (stp .eq. one) then dr = gd - gdold ddum = -gdold else dr = (gd - gdold)*stp call dscal(n,stp,d,1) ddum = -gdold*stp endif if (dr .le. epsmch*ddum) then c skip the L-BFGS update. nskip = nskip + 1 updatd = .false. cw if (iprint .ge. 1) write (6,1004) dr, ddum cx if (iprint .ge. 1) then cx call dblepr(' ys =',-1, dr, 1) cx call dblepr(' BFGS update skipped for +gs=',-1, ddum, 1) cx endif cw 1004 format (' ys=',1p,e10.3,' -gs=',1p,e10.3,' BFGS update SKIPPED') goto 888 endif cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Update the L-BFGS matrix. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc updatd = .true. iupdat = iupdat + 1 c Update matrices WS and WY and form the middle matrix in B. call matupd(n,m,ws,wy,sy,ss,d,r,itail, + iupdat,col,head,theta,rr,dr,stp,dtd) c Form the upper half of the pds T = theta*SS + L*D^(-1)*L'; c Store T in the upper triangular of the array wt; c Cholesky factorize T to J*J' with c J' stored in the upper triangular of wt. call formt(m,wt,sy,ss,col,theta,info) if (info .ne. 0) then c nonpositive definiteness in Cholesky factorization; c refresh the lbfgs memory and restart the iteration. cw if(iprint .ge. 1) write (6, 1007) if (iprint .ge. 1) then cx call intpr( cx + ' Nonpositive definiteness in Cholesky factorization in formt;', cx + -1, 0, 0) cx call intpr( cx + ' refresh the lbfgs memory and restart the iteration.', cx + -1, 0, 0) endif info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. goto 222 endif c Now the inverse of the middle matrix in B is c [ D^(1/2) O ] [ -D^(1/2) D^(-1/2)*L' ] c [ -L*D^(-1/2) J ] [ 0 J' ] 888 continue c -------------------- the end of the loop ----------------------------- goto 222 999 continue cj call timer(time2) time2 = 0.0d0 time = time2 - time1 call prn3lb(n,x,f,itask,iprint,info,k) 1000 continue c Save local variables. lsave(1) = prjctd lsave(2) = cnstnd lsave(3) = boxed lsave(4) = updatd isave(1) = nintol cy isave(3) = itfile isave(4) = iback isave(5) = nskip isave(6) = head isave(7) = col isave(8) = itail isave(9) = iter isave(10) = iupdat isave(12) = nseg isave(13) = nfgv isave(14) = info isave(15) = ifun isave(16) = iword isave(17) = nfree isave(18) = nact isave(19) = ileave isave(20) = nenter dsave(1) = theta dsave(2) = fold dsave(3) = tol dsave(4) = dnorm dsave(5) = epsmch dsave(6) = cpu1 cy dsave(7) = cachyt dsave(8) = sbtime dsave(9) = lnscht dsave(10) = time1 dsave(11) = gd dsave(12) = stpmx dsave(13) = sbgnrm dsave(14) = stp dsave(15) = gdold dsave(16) = dtd cw 1001 format (//,'ITERATION ',i5) cw 1002 format cw + (/,'At iterate',i5,4x,'f= ',1p,d12.5,4x,'|proj g|= ',1p,d12.5) cw 1003 format (2(1x,i4),5x,'-',5x,'-',3x,'-',5x,'-',5x,'-',8x,'-',3x, cw + 1p,2(1x,d10.3)) cw 1004 format (' ys=',1p,e10.3,' -gs=',1p,e10.3,' BFGS update SKIPPED') cw 1005 format (/, cw +' Singular triangular system detected;',/, cw +' refresh the lbfgs memory and restart the iteration.') cw 1006 format (/, cw +' Nonpositive definiteness in Cholesky factorization in formk;',/, cw +' refresh the lbfgs memory and restart the iteration.') cw 1007 format (/, cw +' Nonpositive definiteness in Cholesky factorization in formt;',/, cw +' refresh the lbfgs memory and restart the iteration.') cw 1008 format (/, cw +' Bad direction in the line search;',/, cw +' refresh the lbfgs memory and restart the iteration.') return end c======================= The end of mainlb ============================= subroutine active(n, l, u, nbd, x, iwhere, iprint, + prjctd, cnstnd, boxed) logical prjctd, cnstnd, boxed integer n, iprint, nbd(n), iwhere(n) double precision x(n), l(n), u(n) c ************ c c Subroutine active c c This subroutine initializes iwhere and projects the initial x to c the feasible set if necessary. c c iwhere is an integer array of dimension n. c On entry iwhere is unspecified. c On exit iwhere(i)=-1 if x(i) has no bounds c 3 if l(i)=u(i) c 0 otherwise. c In cauchy, iwhere is given finer gradations. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer nbdd,i double precision zero parameter (zero=0.0d0) c Initialize nbdd, prjctd, cnstnd and boxed. nbdd = 0 prjctd = .false. cnstnd = .false. boxed = .true. c Project the initial x to the easible set if necessary. do 10 i = 1, n if (nbd(i) .gt. 0) then if (nbd(i) .le. 2 .and. x(i) .le. l(i)) then if (x(i) .lt. l(i)) then prjctd = .true. x(i) = l(i) endif nbdd = nbdd + 1 else if (nbd(i) .ge. 2 .and. x(i) .ge. u(i)) then if (x(i) .gt. u(i)) then prjctd = .true. x(i) = u(i) endif nbdd = nbdd + 1 endif endif 10 continue c Initialize iwhere and assign values to cnstnd and boxed. do 20 i = 1, n if (nbd(i) .ne. 2) boxed = .false. if (nbd(i) .eq. 0) then c this variable is always free iwhere(i) = -1 c otherwise set x(i)=mid(x(i), u(i), l(i)). else cnstnd = .true. if (nbd(i) .eq. 2 .and. u(i) - l(i) .le. zero) then c this variable is always fixed iwhere(i) = 3 else iwhere(i) = 0 endif endif 20 continue cx if (iprint .ge. 0) then cx if (prjctd) then cx call intpr('initial X infeasible. Restart with projection.', cx + -1, 0, 0) cx endif cx if (.not. cnstnd) then cx call intpr('This problem is unconstrained.', -1, 0,0) cx endif cx endif cw if (iprint .gt. 0) write (6,1001) nbdd cx if (iprint .gt. 0) then cx call intpr(' Variables exactly at bounds for X0 ',-1, cx + nbdd, 1) cx endif cw 1001 format (/,'At X0 ',i9,' variables are exactly at the bounds') return end c======================= The end of active ============================= subroutine bmv(m, sy, wt, col, v, p, info) integer m, col, info double precision sy(m, m), wt(m, m), v(2*col), p(2*col) c ************ c c Subroutine bmv c c This subroutine computes the product of the 2m x 2m middle matrix c in the compact L-BFGS formula of B and a 2m vector v; c it returns the product in p. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c sy is a double precision array of dimension m x m. c On entry sy specifies the matrix S'Y. c On exit sy is unchanged. c c wt is a double precision array of dimension m x m. c On entry wt specifies the upper triangular matrix J' which is c the Cholesky factor of (thetaS'S+LD^(-1)L'). c On exit wt is unchanged. c c col is an integer variable. c On entry col specifies the number of s-vectors (or y-vectors) c stored in the compact L-BFGS formula. c On exit col is unchanged. c c v is a double precision array of dimension 2col. c On entry v specifies vector v. c On exit v is unchanged. c c p is a double precision array of dimension 2col. c On entry p is unspecified. c On exit p is the product Mv. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return, c = nonzero for abnormal return when the system c to be solved by dtrsl is singular. c c Subprograms called: c c Linpack ... dtrsl. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,k,i2 double precision sum if (col .eq. 0) return c PART I: solve [ D^(1/2) O ] [ p1 ] = [ v1 ] c [ -L*D^(-1/2) J ] [ p2 ] [ v2 ]. c solve Jp2=v2+LD^(-1)v1. p(col + 1) = v(col + 1) do 20 i = 2, col i2 = col + i sum = 0.0d0 do 10 k = 1, i - 1 sum = sum + sy(i,k)*v(k)/sy(k,k) 10 continue p(i2) = v(i2) + sum 20 continue c Solve the triangular system call dtrsl(wt,m,col,p(col+1),11,info) if (info .ne. 0) return c solve D^(1/2)p1=v1. do 30 i = 1, col p(i) = v(i)/sqrt(sy(i,i)) 30 continue c PART II: solve [ -D^(1/2) D^(-1/2)*L' ] [ p1 ] = [ p1 ] c [ 0 J' ] [ p2 ] [ p2 ]. c solve J^Tp2=p2. call dtrsl(wt,m,col,p(col+1),01,info) if (info .ne. 0) return c compute p1=-D^(-1/2)(p1-D^(-1/2)L'p2) c =-D^(-1/2)p1+D^(-1)L'p2. do 40 i = 1, col p(i) = -p(i)/sqrt(sy(i,i)) 40 continue do 60 i = 1, col sum = 0.d0 do 50 k = i + 1, col sum = sum + sy(k,i)*p(col+k)/sy(i,i) 50 continue p(i) = p(i) + sum 60 continue return end c======================== The end of bmv =============================== subroutine cauchy(n, x, l, u, nbd, g, iorder, iwhere, t, d, xcp, + m, wy, ws, sy, wt, theta, col, head, p, c, wbp, + v, nseg, iprint, sbgnrm, info, epsmch) implicit none integer n, m, head, col, nseg, iprint, info, + nbd(n), iorder(n), iwhere(n) double precision theta, epsmch, + x(n), l(n), u(n), g(n), t(n), d(n), xcp(n), + wy(n, col), ws(n, col), sy(m, m), + wt(m, m), p(2*m), c(2*m), wbp(2*m), v(2*m) c ************ c c Subroutine cauchy c c For given x, l, u, g (with sbgnrm > 0), and a limited memory c BFGS matrix B defined in terms of matrices WY, WS, WT, and c scalars head, col, and theta, this subroutine computes the c generalized Cauchy point (GCP), defined as the first local c minimizer of the quadratic c c Q(x + s) = g's + 1/2 s'Bs c c along the projected gradient direction P(x-tg,l,u). c The routine returns the GCP in xcp. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c x is a double precision array of dimension n. c On entry x is the starting point for the GCP computation. c On exit x is unchanged. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c g is a double precision array of dimension n. c On entry g is the gradient of f(x). g must be a nonzero vector. c On exit g is unchanged. c c iorder is an integer working array of dimension n. c iorder will be used to store the breakpoints in the piecewise c linear path and free variables encountered. On exit, c iorder(1),...,iorder(nleft) are indices of breakpoints c which have not been encountered; c iorder(nleft+1),...,iorder(nbreak) are indices of c encountered breakpoints; and c iorder(nfree),...,iorder(n) are indices of variables which c have no bound constraits along the search direction. c c iwhere is an integer array of dimension n. c On entry iwhere indicates only the permanently fixed (iwhere=3) c or free (iwhere= -1) components of x. c On exit iwhere records the status of the current x variables. c iwhere(i)=-3 if x(i) is free and has bounds, but is not moved c 0 if x(i) is free and has bounds, and is moved c 1 if x(i) is fixed at l(i), and l(i) .ne. u(i) c 2 if x(i) is fixed at u(i), and u(i) .ne. l(i) c 3 if x(i) is always fixed, i.e., u(i)=x(i)=l(i) c -1 if x(i) is always free, i.e., it has no bounds. c c t is a double precision working array of dimension n. c t will be used to store the break points. c c d is a double precision array of dimension n used to store c the Cauchy direction P(x-tg)-x. c c xcp is a double precision array of dimension n used to return the c GCP on exit. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c ws, wy, sy, and wt are double precision arrays. c On entry they store information that defines the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c sy(m,m) stores S'Y; c wt(m,m) stores the c Cholesky factorization of (theta*S'S+LD^(-1)L'). c On exit these arrays are unchanged. c c theta is a double precision variable. c On entry theta is the scaling factor specifying B_0 = theta I. c On exit theta is unchanged. c c col is an integer variable. c On entry col is the actual number of variable metric c corrections stored so far. c On exit col is unchanged. c c head is an integer variable. c On entry head is the location of the first s-vector (or y-vector) c in S (or Y). c On exit col is unchanged. c c p is a double precision working array of dimension 2m. c p will be used to store the vector p = W^(T)d. c c c is a double precision working array of dimension 2m. c c will be used to store the vector c = W^(T)(xcp-x). c c wbp is a double precision working array of dimension 2m. c wbp will be used to store the row of W corresponding c to a breakpoint. c c v is a double precision working array of dimension 2m. c c nseg is an integer variable. c On exit nseg records the number of quadratic segments explored c in searching for the GCP. c c sg and yg are double precision arrays of dimension m. c On entry sg and yg store S'g and Y'g correspondingly. c On exit they are unchanged. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c sbgnrm is a double precision variable. c On entry sbgnrm is the norm of the projected gradient at x. c On exit sbgnrm is unchanged. c c info is an integer variable. c On entry info is 0. c On exit info = 0 for normal return, c = nonzero for abnormal return when the the system c used in routine bmv is singular. c c Subprograms called: c c L-BFGS-B Library ... hpsolb, bmv. c c Linpack ... dscal dcopy, daxpy. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ logical xlower,xupper,bnded integer i,j,col2,nfree,nbreak,pointr, + ibp,nleft,ibkmin,iter integer nprt double precision f1,f2,dt,dtm,tsum,dibp,zibp,dibp2,bkmin, + tu,tl,wmc,wmp,wmw,ddot,tj,tj0,neggi,sbgnrm, + f2_org double precision one,zero parameter (one=1.0d0,zero=0.0d0) c Check the status of the variables, reset iwhere(i) if necessary; c compute the Cauchy direction d and the breakpoints t; initialize c the derivative f1 and the vector p = W'd (for theta = 1). if (sbgnrm .le. zero) then cx if (iprint .ge. 0) then cx call intpr('Subgnorm =0, GCP = X.', -1, 0, 0) cx endif cw if (iprint .ge. 0) write (6,*) 'Subgnorm = 0. GCP = X.' call dcopy(n,x,1,xcp,1) return endif bnded = .true. nfree = n + 1 nbreak = 0 ibkmin = 0 bkmin = zero col2 = 2*col f1 = zero cw if (iprint .ge. 99) write (6,3010) cx if (iprint .ge. 99) then cx call intpr('--- CAUCHY entered---',-1,0,0) cx endif c We set p to zero and build it up as we determine d. do 20 i = 1, col2 p(i) = zero 20 continue c In the following loop we determine for each variable its bound c status and its breakpoint, and update p accordingly. c Smallest breakpoint is identified. do 50 i = 1, n neggi = -g(i) if (iwhere(i) .ne. 3 .and. iwhere(i) .ne. -1) then c if x(i) is not a constant and has bounds, c compute the difference between x(i) and its bounds. if (nbd(i) .le. 2) tl = x(i) - l(i) if (nbd(i) .ge. 2) tu = u(i) - x(i) c If a variable is close enough to a bound c we treat it as at bound. xlower = nbd(i) .le. 2 .and. tl .le. zero xupper = nbd(i) .ge. 2 .and. tu .le. zero c reset iwhere(i). iwhere(i) = 0 if (xlower) then if (neggi .le. zero) iwhere(i) = 1 else if (xupper) then if (neggi .ge. zero) iwhere(i) = 2 else if (abs(neggi) .le. zero) iwhere(i) = -3 endif endif pointr = head if (iwhere(i) .ne. 0 .and. iwhere(i) .ne. -1) then d(i) = zero else d(i) = neggi f1 = f1 - neggi*neggi c calculate p := p - W'e_i* (g_i). do 40 j = 1, col p(j) = p(j) + wy(i,pointr)* neggi p(col + j) = p(col + j) + ws(i,pointr)*neggi pointr = mod(pointr,m) + 1 40 continue if (nbd(i) .le. 2 .and. nbd(i) .ne. 0 + .and. neggi .lt. zero) then c x(i) + d(i) is bounded; compute t(i). nbreak = nbreak + 1 iorder(nbreak) = i t(nbreak) = tl/(-neggi) if (nbreak .eq. 1 .or. t(nbreak) .lt. bkmin) then bkmin = t(nbreak) ibkmin = nbreak endif else if (nbd(i) .ge. 2 .and. neggi .gt. zero) then c x(i) + d(i) is bounded; compute t(i). nbreak = nbreak + 1 iorder(nbreak) = i t(nbreak) = tu/neggi if (nbreak .eq. 1 .or. t(nbreak) .lt. bkmin) then bkmin = t(nbreak) ibkmin = nbreak endif else c x(i) + d(i) is not bounded. nfree = nfree - 1 iorder(nfree) = i if (abs(neggi) .gt. zero) bnded = .false. endif endif 50 continue c The indices of the nonzero components of d are now stored c in iorder(1),...,iorder(nbreak) and iorder(nfree),...,iorder(n). c The smallest of the nbreak breakpoints is in t(ibkmin)=bkmin. if (theta .ne. one) then c complete the initialization of p for theta not= one. call dscal(col,theta,p(col+1),1) endif c Initialize GCP xcp = x. call dcopy(n,x,1,xcp,1) if (nbreak .eq. 0 .and. nfree .eq. n + 1) then c is a zero vector, return with the initial xcp as GCP. cw if (iprint .gt. 100) write (6,1010) (xcp(i), i = 1, n) cx if (iprint .gt. 100) then cx nprt = n cx if (nprt .gt. 5) nprt = 5 cx call dblepr('Cauchy X[1:5] = ',-1, xcp, nprt) cx endif return endif c Initialize c = W'(xcp - x) = 0. do 60 j = 1, col2 c(j) = zero 60 continue c Initialize derivative f2. f2 = -theta*f1 f2_org = f2 if (col .gt. 0) then call bmv(m,sy,wt,col,p,v,info) if (info .ne. 0) return f2 = f2 - ddot(col2,v,1,p,1) endif dtm = -f1/f2 tsum = zero nseg = 1 cx if (iprint .ge. 99) then cw + write (6,*) 'There are ',nbreak,' breakpoints ' cx call intpr('no. of breakpoints =',-1, nbreak, 1) cx endif c If there are no breakpoints, locate the GCP and return. if (nbreak .eq. 0) goto 888 nleft = nbreak iter = 1 tj = zero c------------------- the beginning of the loop ------------------------- 777 continue c Find the next smallest breakpoint; c compute dt = t(nleft) - t(nleft + 1). tj0 = tj if (iter .eq. 1) then c Since we already have the smallest breakpoint we need not do c heapsort yet. Often only one breakpoint is used and the c cost of heapsort is avoided. tj = bkmin ibp = iorder(ibkmin) else if (iter .eq. 2) then c Replace the already used smallest breakpoint with the c breakpoint numbered nbreak > nlast, before heapsort call. if (ibkmin .ne. nbreak) then t(ibkmin) = t(nbreak) iorder(ibkmin) = iorder(nbreak) endif c Update heap structure of breakpoints c (if iter=2, initialize heap). endif call hpsolb(nleft,t,iorder,iter-2) tj = t(nleft) ibp = iorder(nleft) endif dt = tj - tj0 cx if (dt .ne. zero .and. iprint .ge. 100) then cw write (6,4011) nseg,f1,f2 cx call intpr('Piece ',-1, nseg, 1) cx call dblepr('f1 at start point =',-1, f1, 1) cx call dblepr('f2 at start point =',-1, f1, 1) cw write (6,5010) dt cx call dblepr('Distance to the next break point = ', cx + -1, dt, 1) cx call dblepr('Distance to the stationary point = ', cx + -1, dtm, 1) cw write (6,6010) dtm cx endif c If a minimizer is within this interval, locate the GCP and return. if (dtm .lt. dt) goto 888 c Otherwise fix one variable and c reset the corresponding component of d to zero. tsum = tsum + dt nleft = nleft - 1 iter = iter + 1 dibp = d(ibp) d(ibp) = zero if (dibp .gt. zero) then zibp = u(ibp) - x(ibp) xcp(ibp) = u(ibp) iwhere(ibp) = 2 else zibp = l(ibp) - x(ibp) xcp(ibp) = l(ibp) iwhere(ibp) = 1 endif cw if (iprint .ge. 100) write (6,*) 'Variable ',ibp,' is fixed.' cx if (iprint .ge. 100) cx + call intpr('Variable fixed, index ',-1,ibp, 1) if (nleft .eq. 0 .and. nbreak .eq. n) then c all n variables are fixed, c return with xcp as GCP. dtm = dt goto 999 endif c Update the derivative information. nseg = nseg + 1 dibp2 = dibp**2 c Update f1 and f2. c temporarily set f1 and f2 for col=0. f1 = f1 + dt*f2 + dibp2 - theta*dibp*zibp f2 = f2 - theta*dibp2 if (col .gt. 0) then c update c = c + dt*p. call daxpy(col2,dt,p,1,c,1) c choose wbp, c the row of W corresponding to the breakpoint encountered. pointr = head do 70 j = 1,col wbp(j) = wy(ibp,pointr) wbp(col + j) = theta*ws(ibp,pointr) pointr = mod(pointr,m) + 1 70 continue c compute (wbp)Mc, (wbp)Mp, and (wbp)M(wbp)'. call bmv(m,sy,wt,col,wbp,v,info) if (info .ne. 0) return wmc = ddot(col2,c,1,v,1) wmp = ddot(col2,p,1,v,1) wmw = ddot(col2,wbp,1,v,1) c update p = p - dibp*wbp. call daxpy(col2,-dibp,wbp,1,p,1) c complete updating f1 and f2 while col > 0. f1 = f1 + dibp*wmc f2 = f2 + 2.0d0*dibp*wmp - dibp2*wmw endif f2 = max(epsmch*f2_org,f2) if (nleft .gt. 0) then dtm = -f1/f2 goto 777 c to repeat the loop for unsearched intervals. else if(bnded) then f1 = zero f2 = zero dtm = zero else dtm = -f1/f2 endif c------------------- the end of the loop ------------------------------- 888 continue cx if (iprint .ge. 99) then cw write (6,*) cw write (6,*) 'GCP found in this segment' cw write (6,4010) nseg,f1,f2 cx call intpr('Piece ',-1, nseg, 1) cx call dblepr('f1 at start point =',-1, f1, 1) cx call dblepr('f2 at start point =',-1, f1, 1) cw write (6,6010) dtm cx call dblepr('Distance to the stationary point = ', cx + -1, dtm, 1) cx endif if (dtm .le. zero) dtm = zero tsum = tsum + dtm c Move free variables (i.e., the ones w/o breakpoints) and c the variables whose breakpoints haven't been reached. call daxpy(n,tsum,d,1,xcp,1) 999 continue c Update c = c + dtm*p = W'(x^c - x) c which will be used in computing r = Z'(B(x^c - x) + g). if (col .gt. 0) call daxpy(col2,dtm,p,1,c,1) cx if (iprint .gt. 100) then cw write (6,1010) (xcp(i),i = 1,n) cx nprt = n cx if (nprt .gt. 5) nprt = 5 cx call dblepr('Cauchy X[1:5] = ',-1, xcp, nprt) cx endif cx if (iprint .ge. 99) then cw write (6,2010) cx call intpr('--- exit CAUCHY---',-1, 0,0) cx endif cw 1010 format ('Cauchy X = ',/,(4x,1p,6(1x,d11.4))) cx 2010 format (/,'---------------- exit CAUCHY----------------------',/) cw 3010 format (/,'---------------- CAUCHY entered-------------------') cw 4010 format ('Piece ',i3,' --f1, f2 at start point ',1p,2(1x,d11.4)) cw 4011 format (/,'Piece ',i3,' --f1, f2 at start point ', cw + 1p,2(1x,d11.4)) cw 5010 format ('Distance to the next break point = ',1p,d11.4) cw 6010 format ('Distance to the stationary point = ',1p,d11.4) return end c====================== The end of cauchy ============================== subroutine cmprlb(n, m, x, g, ws, wy, sy, wt, z, r, wa, index, + theta, col, head, nfree, cnstnd, info) logical cnstnd integer n, m, col, head, nfree, info, index(n) double precision theta, + x(n), g(n), z(n), r(n), wa(4*m), + ws(n, m), wy(n, m), sy(m, m), wt(m, m) c ************ c c Subroutine cmprlb c c This subroutine computes r=-Z'B(xcp-xk)-Z'g by using c wa(2m+1)=W'(xcp-x) from subroutine cauchy. c c Subprograms called: c c L-BFGS-B Library ... bmv. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,j,k,pointr double precision a1,a2 if (.not. cnstnd .and. col .gt. 0) then do 26 i = 1, n r(i) = -g(i) 26 continue else do 30 i = 1, nfree k = index(i) r(i) = -theta*(z(k) - x(k)) - g(k) 30 continue call bmv(m,sy,wt,col,wa(2*m+1),wa(1),info) if (info .ne. 0) then info = -8 return endif pointr = head do 34 j = 1, col a1 = wa(j) a2 = theta*wa(col + j) do 32 i = 1, nfree k = index(i) r(i) = r(i) + wy(k,pointr)*a1 + ws(k,pointr)*a2 32 continue pointr = mod(pointr,m) + 1 34 continue endif return end c======================= The end of cmprlb ============================= subroutine errclb(n, m, factr, l, u, nbd, itask, info, k) c character*255 task integer n, m, itask, info, k, nbd(n) double precision factr, l(n), u(n) c ************ c c Subroutine errclb c c This subroutine checks the validity of the input data. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i double precision zero parameter (zero=0.0d0) c Check the input arguments for errors. if (n .le. 0) then itask = 13 cx call intpr(" ERROR: N .LE. 0", -1, 0, 0) return endif c if (m .le. 0) task = 'ERROR: M .LE. 0' if (m .le. 0) then cx call intpr(" ERROR: M .LE. 0", -1, 0, 0) return endif c if (factr .lt. zero) task = 'ERROR: FACTR .LT. 0' if (factr .le. zero) then cx call intpr(' ERROR: FACTR .LT. 0', -1, 0, 0) return endif c Check the validity of the arrays nbd(i), u(i), and l(i). do 10 i = 1, n if (nbd(i) .lt. 0 .or. nbd(i) .gt. 3) then c return c task = 'ERROR: INVALID NBD' itask = 12 info = -6 k = i endif if (nbd(i) .eq. 2) then if (l(i) .gt. u(i)) then c return c task = 'ERROR: NO FEASIBLE SOLUTION' itask = 14 info = -7 k = i endif endif 10 continue return end c======================= The end of errclb ============================= subroutine formk(n, nsub, ind, nenter, ileave, indx2, iupdat, + updatd, wn, wn1, m, ws, wy, sy, theta, col, + head, info) integer n, nsub, m, col, head, nenter, ileave, iupdat, + info, ind(n), indx2(n) double precision theta, wn(2*m, 2*m), wn1(2*m, 2*m), + ws(n, m), wy(n, m), sy(m, m) logical updatd c ************ c c Subroutine formk c c This subroutine forms the LEL^T factorization of the indefinite c c matrix K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] c The matrix K can be shown to be equal to the matrix M^[-1]N c occurring in section 5.1 of [1], as well as to the matrix c Mbar^[-1] Nbar in section 5.3. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c nsub is an integer variable c On entry nsub is the number of subspace variables in free set. c On exit nsub is not changed. c c ind is an integer array of dimension nsub. c On entry ind specifies the indices of subspace variables. c On exit ind is unchanged. c c nenter is an integer variable. c On entry nenter is the number of variables entering the c free set. c On exit nenter is unchanged. c c ileave is an integer variable. c On entry indx2(ileave),...,indx2(n) are the variables leaving c the free set. c On exit ileave is unchanged. c c indx2 is an integer array of dimension n. c On entry indx2(1),...,indx2(nenter) are the variables entering c the free set, while indx2(ileave),...,indx2(n) are the c variables leaving the free set. c On exit indx2 is unchanged. c c iupdat is an integer variable. c On entry iupdat is the total number of BFGS updates made so far. c On exit iupdat is unchanged. c c updatd is a logical variable. c On entry 'updatd' is true if the L-BFGS matrix is updatd. c On exit 'updatd' is unchanged. c c wn is a double precision array of dimension 2m x 2m. c On entry wn is unspecified. c On exit the upper triangle of wn stores the LEL^T factorization c of the 2*col x 2*col indefinite matrix c [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c wn1 is a double precision array of dimension 2m x 2m. c On entry wn1 stores the lower triangular part of c [Y' ZZ'Y L_a'+R_z'] c [L_a+R_z S'AA'S ] c in the previous iteration. c On exit wn1 stores the corresponding updated matrices. c The purpose of wn1 is just to store these inner products c so they can be easily updated and inserted into wn. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c ws, wy, sy, and wtyy are double precision arrays; c theta is a double precision variable; c col is an integer variable; c head is an integer variable. c On entry they store the information defining the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c sy(m,m) stores S'Y; c wtyy(m,m) stores the Cholesky factorization c of (theta*S'S+LD^(-1)L') c theta is the scaling factor specifying B_0 = theta I; c col is the number of variable metric corrections stored; c head is the location of the 1st s- (or y-) vector in S (or Y). c On exit they are unchanged. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return; c = -1 when the 1st Cholesky factorization failed; c = -2 when the 2st Cholesky factorization failed. c c Subprograms called: c c Linpack ... dcopy, dpofa, dtrsl. c c c References: c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: a c limited memory FORTRAN code for solving bound constrained c optimization problems'', Tech. Report, NAM-11, EECS Department, c Northwestern University, 1994. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer m2,ipntr,jpntr,iy,is,jy,js,is1,js1,k1,i,k, + col2,pbegin,pend,dbegin,dend,upcl double precision ddot,temp1,temp2,temp3,temp4 double precision zero parameter (zero=0.0d0) c Form the lower triangular part of c WN1 = [Y' ZZ'Y L_a'+R_z'] c [L_a+R_z S'AA'S ] c where L_a is the strictly lower triangular part of S'AA'Y c R_z is the upper triangular part of S'ZZ'Y. if (updatd) then if (iupdat .gt. m) then c shift old part of WN1. do 10 jy = 1, m - 1 js = m + jy call dcopy(m-jy,wn1(jy+1,jy+1),1,wn1(jy,jy),1) call dcopy(m-jy,wn1(js+1,js+1),1,wn1(js,js),1) call dcopy(m-1,wn1(m+2,jy+1),1,wn1(m+1,jy),1) 10 continue endif c put new rows in blocks (1,1), (2,1) and (2,2). pbegin = 1 pend = nsub dbegin = nsub + 1 dend = n iy = col is = m + col ipntr = head + col - 1 if (ipntr .gt. m) ipntr = ipntr - m jpntr = head do 20 jy = 1, col js = m + jy temp1 = zero temp2 = zero temp3 = zero c compute element jy of row 'col' of Y'ZZ'Y do 15 k = pbegin, pend k1 = ind(k) temp1 = temp1 + wy(k1,ipntr)*wy(k1,jpntr) 15 continue c compute elements jy of row 'col' of L_a and S'AA'S do 16 k = dbegin, dend k1 = ind(k) temp2 = temp2 + ws(k1,ipntr)*ws(k1,jpntr) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 16 continue wn1(iy,jy) = temp1 wn1(is,js) = temp2 wn1(is,jy) = temp3 jpntr = mod(jpntr,m) + 1 20 continue c put new column in block (2,1). jy = col jpntr = head + col - 1 if (jpntr .gt. m) jpntr = jpntr - m ipntr = head do 30 i = 1, col is = m + i temp3 = zero c compute element i of column 'col' of R_z do 25 k = pbegin, pend k1 = ind(k) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 25 continue ipntr = mod(ipntr,m) + 1 wn1(is,jy) = temp3 30 continue upcl = col - 1 else upcl = col endif c modify the old parts in blocks (1,1) and (2,2) due to changes c in the set of free variables. ipntr = head do 45 iy = 1, upcl is = m + iy jpntr = head do 40 jy = 1, iy js = m + jy temp1 = zero temp2 = zero temp3 = zero temp4 = zero do 35 k = 1, nenter k1 = indx2(k) temp1 = temp1 + wy(k1,ipntr)*wy(k1,jpntr) temp2 = temp2 + ws(k1,ipntr)*ws(k1,jpntr) 35 continue do 36 k = ileave, n k1 = indx2(k) temp3 = temp3 + wy(k1,ipntr)*wy(k1,jpntr) temp4 = temp4 + ws(k1,ipntr)*ws(k1,jpntr) 36 continue wn1(iy,jy) = wn1(iy,jy) + temp1 - temp3 wn1(is,js) = wn1(is,js) - temp2 + temp4 jpntr = mod(jpntr,m) + 1 40 continue ipntr = mod(ipntr,m) + 1 45 continue c modify the old parts in block (2,1). ipntr = head do 60 is = m + 1, m + upcl jpntr = head do 55 jy = 1, upcl temp1 = zero temp3 = zero do 50 k = 1, nenter k1 = indx2(k) temp1 = temp1 + ws(k1,ipntr)*wy(k1,jpntr) 50 continue do 51 k = ileave, n k1 = indx2(k) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 51 continue if (is .le. jy + m) then wn1(is,jy) = wn1(is,jy) + temp1 - temp3 else wn1(is,jy) = wn1(is,jy) - temp1 + temp3 endif jpntr = mod(jpntr,m) + 1 55 continue ipntr = mod(ipntr,m) + 1 60 continue c Form the upper triangle of WN = [D+Y' ZZ'Y/theta -L_a'+R_z' ] c [-L_a +R_z S'AA'S*theta] m2 = 2*m do 70 iy = 1, col is = col + iy is1 = m + iy do 65 jy = 1, iy js = col + jy js1 = m + jy wn(jy,iy) = wn1(iy,jy)/theta wn(js,is) = wn1(is1,js1)*theta 65 continue do 66 jy = 1, iy - 1 wn(jy,is) = -wn1(is1,jy) 66 continue do 67 jy = iy, col wn(jy,is) = wn1(is1,jy) 67 continue wn(iy,iy) = wn(iy,iy) + sy(iy,iy) 70 continue c Form the upper triangle of WN= [ LL' L^-1(-L_a'+R_z')] c [(-L_a +R_z)L'^-1 S'AA'S*theta ] c first Cholesky factor (1,1) block of wn to get LL' c with L' stored in the upper triangle of wn. call dpofa(wn,m2,col,info) if (info .ne. 0) then info = -1 return endif c then form L^-1(-L_a'+R_z') in the (1,2) block. col2 = 2*col do 71 js = col+1 ,col2 call dtrsl(wn,m2,col,wn(1,js),11,info) 71 continue c Form S'AA'S*theta + (L^-1(-L_a'+R_z'))'L^-1(-L_a'+R_z') in the c upper triangle of (2,2) block of wn. do 72 is = col+1, col2 do 74 js = is, col2 wn(is,js) = wn(is,js) + ddot(col,wn(1,is),1,wn(1,js),1) 74 continue 72 continue c Cholesky factorization of (2,2) block of wn. call dpofa(wn(col+1,col+1),m2,col,info) if (info .ne. 0) then info = -2 return endif return end c======================= The end of formk ============================== subroutine formt(m, wt, sy, ss, col, theta, info) integer m, col, info double precision theta, wt(m, m), sy(m, m), ss(m, m) c ************ c c Subroutine formt c c This subroutine forms the upper half of the pos. def. and symm. c T = theta*SS + L*D^(-1)*L', stores T in the upper triangle c of the array wt, and performs the Cholesky factorization of T c to produce J*J', with J' stored in the upper triangle of wt. c c Subprograms called: c c Linpack ... dpofa. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,j,k,k1 double precision ddum double precision zero parameter (zero=0.0d0) c Form the upper half of T = theta*SS + L*D^(-1)*L', c store T in the upper triangle of the array wt. do 52 j = 1, col wt(1,j) = theta*ss(1,j) 52 continue do 55 i = 2, col do 54 j = i, col k1 = min(i,j) - 1 ddum = zero do 53 k = 1, k1 ddum = ddum + sy(i,k)*sy(j,k)/sy(k,k) 53 continue wt(i,j) = ddum + theta*ss(i,j) 54 continue 55 continue c Cholesky factorize T to J*J' with c J' stored in the upper triangle of wt. call dpofa(wt,m,col,info) if (info .ne. 0) then info = -3 endif return end c======================= The end of formt ============================== subroutine freev(n, nfree, index, nenter, ileave, indx2, + iwhere, wrk, updatd, cnstnd, iprint, iter) integer n, nfree, nenter, ileave, iprint, iter, + index(n), indx2(n), iwhere(n) logical wrk, updatd, cnstnd c ************ c c Subroutine freev c c This subroutine counts the entering and leaving variables when c iter > 0, and finds the index set of free and active variables c at the GCP. c c cnstnd is a logical variable indicating whether bounds are present c c index is an integer array of dimension n c for i=1,...,nfree, index(i) are the indices of free variables c for i=nfree+1,...,n, index(i) are the indices of bound variables c On entry after the first iteration, index gives c the free variables at the previous iteration. c On exit it gives the free variables based on the determination c in cauchy using the array iwhere. c c indx2 is an integer array of dimension n c On entry indx2 is unspecified. c On exit with iter>0, indx2 indicates which variables c have changed status since the previous iteration. c For i= 1,...,nenter, indx2(i) have changed from bound to free. c For i= ileave+1,...,n, indx2(i) have changed from free to bound. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer iact,i,k integer itmp nenter = 0 ileave = n + 1 if (iter .gt. 0 .and. cnstnd) then c count the entering and leaving variables. do 20 i = 1, nfree k = index(i) cjn write(6,*) ' k = index(i) ', k cjn write(6,*) ' index = ', i if (iwhere(k) .gt. 0) then ileave = ileave - 1 indx2(ileave) = k if (iprint .ge. 100) then cx call intpr( cx + 'Variable k leaves the set of free variables for k =', cx + -1, k, 1) endif cw if (iprint .ge. 100) write (6,*) cw + 'Variable ',k,' leaves the set of free variables' endif 20 continue do 22 i = 1 + nfree, n k = index(i) if (iwhere(k) .le. 0) then nenter = nenter + 1 indx2(nenter) = k cx if (iprint .ge. 100) then cx call intpr('Var entering free vars is k=',-1,k,1) cw write (6,*) cw + 'Variable ',k,' enters the set of free variables' cx endif endif 22 continue cx if (iprint .ge. 99) then cw write (6,*) cw + n+1-ileave,' variables leave; ',nenter,' variables enter' cx itmp = n+1-ileave cx call intpr(' no. variables leaving =',-1,itmp, 1) cx call intpr(' no. variables entering =',-1,nenter, 1) cx endif endif wrk = (ileave .lt. n+1) .or. (nenter .gt. 0) .or. updatd c Find the index set of free and active variables at the GCP. nfree = 0 iact = n + 1 do 24 i = 1, n if (iwhere(i) .le. 0) then nfree = nfree + 1 index(nfree) = i else iact = iact - 1 index(iact) = i endif 24 continue cx if (iprint .ge. 99) then cw write (6,*) cw + nfree,' variables are free at GCP ',iter + 1 cx call intpr(' no. variables free =',-1, nfree,1) cx itmp = iter + 1 cx call intpr(' at GCP ',-1,itmp,1) cx endif return end c======================= The end of freev ============================== subroutine hpsolb(n, t, iorder, iheap) integer iheap, n, iorder(n) double precision t(n) c ************ c c Subroutine hpsolb c c This subroutine sorts out the least element of t, and puts the c remaining elements of t in a heap. c c n is an integer variable. c On entry n is the dimension of the arrays t and iorder. c On exit n is unchanged. c c t is a double precision array of dimension n. c On entry t stores the elements to be sorted, c On exit t(n) stores the least elements of t, and t(1) to t(n-1) c stores the remaining elements in the form of a heap. c c iorder is an integer array of dimension n. c On entry iorder(i) is the index of t(i). c On exit iorder(i) is still the index of t(i), but iorder may be c permuted in accordance with t. c c iheap is an integer variable specifying the task. c On entry iheap should be set as follows: c iheap .eq. 0 if t(1) to t(n) is not in the form of a heap, c iheap .ne. 0 if otherwise. c On exit iheap is unchanged. c c c References: c Algorithm 232 of CACM (J. W. J. Williams): HEAPSORT. c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c ************ integer i,j,k,indxin,indxou double precision ddum,out if (iheap .eq. 0) then c Rearrange the elements t(1) to t(n) to form a heap. do 20 k = 2, n ddum = t(k) indxin = iorder(k) c Add ddum to the heap. i = k 10 continue if (i.gt.1) then j = i/2 if (ddum .lt. t(j)) then t(i) = t(j) iorder(i) = iorder(j) i = j goto 10 endif endif t(i) = ddum iorder(i) = indxin 20 continue endif c Assign to 'out' the value of t(1), the least member of the heap, c and rearrange the remaining members to form a heap as c elements 1 to n-1 of t. if (n .gt. 1) then i = 1 out = t(1) indxou = iorder(1) ddum = t(n) indxin = iorder(n) c Restore the heap 30 continue j = i+i if (j .le. n-1) then if (t(j+1) .lt. t(j)) j = j+1 if (t(j) .lt. ddum ) then t(i) = t(j) iorder(i) = iorder(j) i = j goto 30 endif endif t(i) = ddum iorder(i) = indxin c Put the least member in t(n). t(n) = out iorder(n) = indxou endif return end c====================== The end of hpsolb ============================== subroutine lnsrlb(n, l, u, nbd, x, f, fold, gd, gdold, g, d, r, t, + z, stp, dnorm, dtd, xstep, stpmx, iter, ifun, + iback, nfgv, info, itask, boxed, cnstnd, icsave, + isave, dsave) logical boxed, cnstnd integer n, iter, ifun, iback, nfgv, info, itask, icsave, + nbd(n), isave(2) double precision f, fold, gd, gdold, stp, dnorm, dtd, xstep, + stpmx, x(n), l(n), u(n), g(n), d(n), r(n), t(n), + z(n), dsave(13) c ********** c c Subroutine lnsrlb c c This subroutine calls subroutine dcsrch from the Minpack2 library c to perform the line search. Subroutine dscrch is safeguarded so c that all trial points lie within the feasible region. c c Subprograms called: c c Minpack2 Library ... dcsrch. c c Linpack ... dtrsl, ddot. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ********** integer i double precision ddot,a1,a2 double precision one,zero,big parameter (one=1.0d0,zero=0.0d0,big=1.0d+10) double precision ftol,gtol,xtol parameter (ftol=1.0d-3,gtol=0.9d0,xtol=0.1d0) c if (task(1:5) .eq. 'FG_LN') goto 556 if (itask .eq. 20) goto 556 dtd = ddot(n,d,1,d,1) dnorm = sqrt(dtd) c Determine the maximum step length. stpmx = big if (cnstnd) then if (iter .eq. 0) then stpmx = one else do 43 i = 1, n a1 = d(i) if (nbd(i) .ne. 0) then if (a1 .lt. zero .and. nbd(i) .le. 2) then a2 = l(i) - x(i) if (a2 .ge. zero) then stpmx = zero else if (a1*stpmx .lt. a2) then stpmx = a2/a1 endif else if (a1 .gt. zero .and. nbd(i) .ge. 2) then a2 = u(i) - x(i) if (a2 .le. zero) then stpmx = zero else if (a1*stpmx .gt. a2) then stpmx = a2/a1 endif endif endif 43 continue endif endif if (iter .eq. 0 .and. .not. boxed) then stp = min(one/dnorm, stpmx) else stp = one endif call dcopy(n,x,1,t,1) call dcopy(n,g,1,r,1) fold = f ifun = 0 iback = 0 c csave = 'START' icsave = 2 556 continue gd = ddot(n,g,1,d,1) if (ifun .eq. 0) then gdold=gd if (gd .ge. zero) then c the directional derivative >=0. c Line search is impossible. cw write(6,*)' ascent direction in projection gd = ', gd cx call dblepr(' ascent direction in projection gd = ', cx + -1, gd,1) info = -4 return endif endif call dcsrch(f,gd,stp,ftol,gtol,xtol,zero,stpmx,icsave,isave,dsave) xstep = stp*dnorm c if (csave(1:4) .ne. 'CONV' .and. csave(1:4) .ne. 'WARN') then c2345678901234567890123456789012345678901234567890123456789012345678901234567890 if ((icsave .lt. 6) .or. ((icsave .gt. 8) .and. + (icsave .lt. 23))) then c task = 'FG_LNSRCH' itask = 20 ifun = ifun + 1 nfgv = nfgv + 1 iback = ifun - 1 if (stp .eq. one) then call dcopy(n,z,1,x,1) else do 41 i = 1, n x(i) = stp*d(i) + t(i) 41 continue endif else c task = 'NEW_X' itask = 1 endif return end c======================= The end of lnsrlb ============================= subroutine matupd(n, m, ws, wy, sy, ss, d, r, itail, + iupdat, col, head, theta, rr, dr, stp, dtd) integer n, m, itail, iupdat, col, head double precision theta, rr, dr, stp, dtd, d(n), r(n), + ws(n, m), wy(n, m), sy(m, m), ss(m, m) c ************ c c Subroutine matupd c c This subroutine updates matrices WS and WY, and forms the c middle matrix in B. c c Subprograms called: c c Linpack ... dcopy, ddot. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer j,pointr double precision ddot double precision one parameter (one=1.0d0) c Set pointers for matrices WS and WY. if (iupdat .le. m) then col = iupdat itail = mod(head+iupdat-2,m) + 1 else itail = mod(itail,m) + 1 head = mod(head,m) + 1 endif c Update matrices WS and WY. call dcopy(n,d,1,ws(1,itail),1) call dcopy(n,r,1,wy(1,itail),1) c Set theta=yy/ys. theta = rr/dr c Form the middle matrix in B. c update the upper triangle of SS, c and the lower triangle of SY: if (iupdat .gt. m) then c move old information do 50 j = 1, col - 1 call dcopy(j,ss(2,j+1),1,ss(1,j),1) call dcopy(col-j,sy(j+1,j+1),1,sy(j,j),1) 50 continue endif c add new information: the last row of SY c and the last column of SS: pointr = head do 51 j = 1, col - 1 sy(col,j) = ddot(n,d,1,wy(1,pointr),1) ss(j,col) = ddot(n,ws(1,pointr),1,d,1) pointr = mod(pointr,m) + 1 51 continue if (stp .eq. one) then ss(col,col) = dtd else ss(col,col) = stp*stp*dtd endif sy(col,col) = dr return end c======================= The end of matupd ============================= subroutine prn1lb(n, m, l, u, x, iprint, epsmch) integer n, m double precision epsmch, x(n), l(n), u(n) c ************ c c Subroutine prn1lb c c This subroutine prints the input data, initial point, upper and c lower bounds of each variable, machine precision, as well as c the headings of the output. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ c$$$ integer i integer nprt c limit output to 1st 5 elements nprt = n if (nprt .gt. 5) nprt = 5 cx if (iprint .ge. 0) then cx if (iprint .ge. 1) then cw write (6,7001) epsmch cx call dblepr('RUNNING THE L-BFGS-B CODE with eps=', cx + -1, epsmch, 1) cw write (6,*) 'N = ',n,' M = ',m cx call intpr(' N =',-1, n, 1) cx call intpr(' M =',-1, m, 1) cw write (itfile,2001) epsmch cw write (itfile,*)'N = ',n,' M = ',m cw write (itfile,9001) cx if (iprint .gt. 100) then cw write (6,1004) 'L =',(l(i),i = 1,n) cw write (6,1004) 'X0 =',(x(i),i = 1,n) cw write (6,1004) 'U =',(u(i),i = 1,n) cx call dblepr('L =',-1, l, nprt) cx call dblepr('X0=',-1, x, nprt) cx call dblepr('U =',-1, u, nprt) cx endif cx endif cx endif cw 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) cw 2001 format ('RUNNING THE L-BFGS-B CODE',/,/, cw + 'it = iteration number',/, cw + 'nf = number of function evaluations',/, cw + 'nseg = number of segments explored during the Cauchy search',/, cw + 'nact = number of active bounds at the generalized Cauchy point' cw + ,/, cw + 'sub = manner in which the subspace minimization terminated:' cw + ,/,' con = converged, bnd = a bound was reached',/, cw + 'itls = number of iterations performed in the line search',/, cw + 'stepl = step length used',/, cw + 'tstep = norm of the displacement (total step)',/, cw + 'projg = norm of the projected gradient',/, cw + 'f = function value',/,/, cw + ' * * *',/,/, cw + 'Machine precision =',1p,d10.3) cw 7001 format ('RUNNING THE L-BFGS-B CODE',/,/, cw + ' * * *',/,/, cw + 'Machine precision =',1p,d10.3) cw 9001 format (/,3x,'it',3x,'nf',2x,'nseg',2x,'nact',2x,'sub',2x,'itls', cw + 2x,'stepl',4x,'tstep',5x,'projg',8x,'f') return end c======================= The end of prn1lb ============================= subroutine prn2lb(f, iprint, iter, + sbgnrm, iback, xstep) integer iprint, iter, iback double precision f, sbgnrm, xstep c ************ c c Subroutine prn2lb c c This subroutine prints out new information after a successful c line search. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer imod c 'word' records the status of subspace solutions. c$$$ if (iword .eq. 0) then c$$$c the subspace minimization converged. c$$$ word = 'con' c$$$ else if (iword .eq. 1) then c$$$c the subspace minimization stopped at a bound. c$$$ word = 'bnd' c$$$ else if (iword .eq. 5) then c$$$c the truncated Newton step has been used. c$$$ word = 'TNT' c$$$ else c$$$ word = '---' c$$$ endif cx if (iprint .ge. 99) then cw write (6,*) 'LINE SEARCH',iback,' times; norm of step = ',xstep cx call intpr('LINE SEARCH iback=',-1, iback, 1) cx call dblepr('norm of step =',-1, xstep, 1) cw write (6,2001) iter,f,sbgnrm cx call intpr('At iterate ',-1, iter, 1) cx call dblepr('f =',-1, f, 1) cx call dblepr('|proj g| =',-1, sbgnrm, 1) cx if (iprint .gt. 100) then cw write (6,1004) 'X =',(x(i), i = 1, n) cw write (6,1004) 'G =',(g(i), i = 1, n) cx endif cx else if (iprint .gt. 0) then cx imod = mod(iter,iprint) cw if (imod .eq. 0) write (6,2001) iter,f,sbgnrm cx endif cw if (iprint .ge. 1) write (itfile,3001) cw + iter,nfgv,nseg,nact,word,iback,stp,xstep,sbgnrm,f cw 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) cw 2001 format cw + (/,'At iterate',i5,4x,'f= ',1p,d12.5,4x,'|proj g|= ',1p,d12.5) cw 3001 format(2(1x,i4),2(1x,i5),2x,a3,1x,i4,1p,2(2x,d7.1),1p,2(1x,d10.3)) return end c======================= The end of prn2lb ============================= subroutine prn3lb(n, x, f, itask, iprint, info, k) c character*255 task integer n, iprint, info, k, itask double precision f, x(n) c ************ c c Subroutine prn3lb c c This subroutine prints out information when either a built-in c convergence test is satisfied or when an error message is c generated. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ c$$$ integer i integer nprt c if (task(1:5) .eq. 'ERROR') goto 999 if ((itask .ge. 9) .and. (itask .le. 19)) goto 999 cx if (iprint .ge. 0) then cw write (6,3003) cw 3003 format (/, cw + ' * * *',/,/, cw + 'Tit = total number of iterations',/, cw + 'Tnf = total number of function evaluations',/, cw + 'Tnint = total number of segments explored during', cw + ' Cauchy searches',/, cw + 'Skip = number of BFGS updates skipped',/, cw + 'Nact = number of active bounds at final generalized', cw + ' Cauchy point',/, cw + 'Projg = norm of the final projected gradient',/, cw + 'F = final function value',/,/, cw + ' * * *') cw write (6,3004) cw 3004 format (/,3x,'N',4x,'Tit',5x,'Tnf',2x,'Tnint',2x, cw + 'Skip',2x,'Nact',5x,'Projg',8x,'F') cw write(6,3005) n,iter,nfgv,nintol,nskip,nact,sbgnrm,f cw 3005 format (i5,2(1x,i6),(1x,i6),(2x,i4),(1x,i5),1p,2(2x,d10.3)) cx if (iprint .ge. 100) then cx nprt = n cx if (nprt .gt. 5) nprt = 5 cx call dblepr('X=', -1, x, nprt) cw write (6,1004) 'X =',(x(i),i = 1,n) cw 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) cx endif cw if (iprint .ge. 1) write (6,*) ' F =',f cx if (iprint .ge. 1) call dblepr(' F =',-1,f, 1) cx endif 999 continue cx if (iprint .ge. 0) then cw write (6,3009) itask cx if (info .ne. 0) then cw if (info .eq. -1) write (6,9011) cx if (info .eq. -1) then cx call intpr( cx +' Matrix in 1st Cholesky factorization in formk is not Pos. Def.', cx + -1, 0, 0) cx endif cw if (info .eq. -2) write (6,9012) cx if (info .eq. -2) then cx call intpr( cx +' Matrix in 2nd Cholesky factorization in formk is not Pos. Def.', cx + -1, 0, 0) cx endif cw if (info .eq. -3) write (6,9013) cx if (info .eq. -3) then cx call intpr( cx + ' Matrix in Cholesky factorization in formt is not Pos. Def.', cx + -1, 0, 0) cx endif cw if (info .eq. -4) write (6,9014) cx if (info .eq. -4) then cx call intpr( cx +' Derivative >= 0, backtracking line search impossible.', cx + -1, 0, 0) cx call intpr( cx +' Previous x, f and g restored.', cx + -1, 0, 0) cx call intpr( cx +' Possible causes: 1 error in function or gradient evaluation;', cx + -1, 0, 0) cx call intpr( cx +' 2 rounding errors dominate computation.', cx + -1, 0, 0) cx endif cw if (info .eq. -5) write (6,9015) cx if (info .eq. -5) then cx call intpr( cx +' Warning: more than 10 function and gradient', cx + -1, 0, 0) cx call intpr( cx +' evaluations in the last line search. Termination', cx + -1, 0, 0) cx call intpr( cx +' may possibly be caused by a bad search direction.', cx + -1, 0, 0) cx endif cw if (info .eq. -6) write (6,*)' Input nbd(',k,') is invalid.' cx if (info .eq. -6) then cx call intpr( cx + ' Input nbd(k) is invalid for k = ', cx + -1, k, 1) cx endif cw if (info .eq. -7) cx if (info .eq. -7) then cx call intpr( cx + ' l(k) > u(k). No feasible solution for k=', -1, k, 1) cx endif cw + write (6,*)' l(',k,') > u(',k,'). No feasible solution.' cw if (info .eq. -8) write (6,9018) cx if (info .eq. -8) then cx call intpr(' The triangular system is singular.', cx + -1, 0, 0) cx endif cw if (info .eq. -9) write (6,9019) cx if (info .eq. -9) then cx call intpr( cx +' Line search cannot locate an adequate point after 20 function', cx + -1, 0, 0) cx call intpr( cx +' and gradient evaluations. Previous x, f and g restored.', cx + -1, 0, 0) cx call intpr( cx +' Possible causes: 1 error in function or gradient evaluation;', cx + -1, 0, 0) cx call intpr( cx +' 2 rounding error dominate computation.', cx + -1, 0, 0) cx endif cx endif cw if (iprint .ge. 1) cw write (6,3007) cachyt,sbtime,lnscht cw suppressing time output cw write (6,3008) time cw Note -- suppress time computations here -- use R stuff cw if (iprint .ge. 1) then cw if (info .eq. -4 .or. info .eq. -9) then cw write (itfile,3002) cw + iter,nfgv,nseg,nact,word,iback,stp,xstep cw endif cw write (itfile,3009) itask cw if (info .ne. 0) then cw if (info .eq. -1) write (itfile,9011) cw if (info .eq. -2) write (itfile,9012) cw if (info .eq. -3) write (itfile,9013) cw if (info .eq. -4) write (itfile,9014) cw if (info .eq. -5) write (itfile,9015) cw if (info .eq. -8) write (itfile,9018) cw if (info .eq. -9) write (itfile,9019) cw endif cw write (itfile,3008) time cw endif cx endif cw 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) cw 3002 format(2(1x,i4),2(1x,i5),2x,a3,1x,i4,1p,2(2x,d7.1),6x,'-',10x,'-') cw 3003 format (/, cw + ' * * *',/,/, cw + 'Tit = total number of iterations',/, cw + 'Tnf = total number of function evaluations',/, cw + 'Tnint = total number of segments explored during', cw + ' Cauchy searches',/, cw + 'Skip = number of BFGS updates skipped',/, cw + 'Nact = number of active bounds at final generalized', cw + ' Cauchy point',/, cw + 'Projg = norm of the final projected gradient',/, cw + 'F = final function value',/,/, cw + ' * * *') cw 3007 format (/,' Cauchy time',1p,e10.3,' seconds.',/ cw + ' Subspace minimization time',1p,e10.3,' seconds.',/ cw + ' Line search time',1p,e10.3,' seconds.') cw 3008 format (/,' Total User time',1p,e10.3,' seconds.',/) cw 3009 format (/,i4) cw 9011 format (/, cw +' Matrix in 1st Cholesky factorization in formk is not Pos. Def.') cw 9012 format (/, cw +' Matrix in 2st Cholesky factorization in formk is not Pos. Def.') cw 9013 format (/, cw +' Matrix in the Cholesky factorization in formt is not Pos. Def.') cw 9014 format (/, cw +' Derivative >= 0, backtracking line search impossible.',/, cw +' Previous x, f and g restored.',/, cw +' Possible causes: 1 error in function or gradient evaluation;',/, cw +' 2 rounding errors dominate computation.') cw 9015 format (/, cw +' Warning: more than 10 function and gradient',/, cw +' evaluations in the last line search. Termination',/, cw +' may possibly be caused by a bad search direction.') cw 9018 format (/,' The triangular system is singular.') cw 9019 format (/, cw +' Line search cannot locate an adequate point after 20 function',/ cw +,' and gradient evaluations. Previous x, f and g restored.',/, cw +' Possible causes: 1 error in function or gradient evaluation;',/, cw +' 2 rounding error dominate computation.') return end c======================= The end of prn3lb ============================= subroutine projgr(n, l, u, nbd, x, g, sbgnrm) integer n, nbd(n) double precision sbgnrm, x(n), l(n), u(n), g(n) c ************ c c Subroutine projgr c c This subroutine computes the infinity norm of the projected c gradient. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i double precision gi double precision zero parameter (zero=0.0d0) sbgnrm = zero do 15 i = 1, n gi = g(i) if (nbd(i) .ne. 0) then if (gi .lt. zero) then if (nbd(i) .ge. 2) gi = max((x(i)-u(i)),gi) else if (nbd(i) .le. 2) gi = min((x(i)-l(i)),gi) endif endif sbgnrm = max(sbgnrm,abs(gi)) 15 continue return end c======================= The end of projgr ============================= subroutine subsm ( n, m, nsub, ind, l, u, nbd, x, d, xp, ws, wy, + theta, xx, gg, + col, head, iword, wv, wn, iprint, info ) implicit none integer n, m, nsub, col, head, iword, iprint, info, + ind(nsub), nbd(n) double precision theta, + l(n), u(n), x(n), d(n), xp(n), xx(n), gg(n), + ws(n, m), wy(n, m), + wv(2*m), wn(2*m, 2*m) c ********************************************************************** c c This routine contains the major changes in the updated version. c The changes are described in the accompanying paper c c Jose Luis Morales, Jorge Nocedal c "Remark On Algorithm 788: L-BFGS-B: Fortran Subroutines for Large-Scale c Bound Constrained Optimization". Decemmber 27, 2010. c c J.L. Morales Departamento de Matematicas, c Instituto Tecnologico Autonomo de Mexico c Mexico D.F. c c J, Nocedal Department of Electrical Engineering and c Computer Science. c Northwestern University. Evanston, IL. USA c c January 17, 2011 c c ********************************************************************** c c c Subroutine subsm c c Given xcp, l, u, r, an index set that specifies c the active set at xcp, and an l-BFGS matrix B c (in terms of WY, WS, SY, WT, head, col, and theta), c this subroutine computes an approximate solution c of the subspace problem c c (P) min Q(x) = r'(x-xcp) + 1/2 (x-xcp)' B (x-xcp) c c subject to l<=x<=u c x_i=xcp_i for all i in A(xcp) c c along the subspace unconstrained Newton direction c c d = -(Z'BZ)^(-1) r. c c The formula for the Newton direction, given the L-BFGS matrix c and the Sherman-Morrison formula, is c c d = (1/theta)r + (1/theta*2) Z'WK^(-1)W'Z r. c c where c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c Note that this procedure for computing d differs c from that described in [1]. One can show that the matrix K is c equal to the matrix M^[-1]N in that paper. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c nsub is an integer variable. c On entry nsub is the number of free variables. c On exit nsub is unchanged. c c ind is an integer array of dimension nsub. c On entry ind specifies the coordinate indices of free variables. c On exit ind is unchanged. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is a integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c x is a double precision array of dimension n. c On entry x specifies the Cauchy point xcp. c On exit x(i) is the minimizer of Q over the subspace of c free variables. c c d is a double precision array of dimension n. c On entry d is the reduced gradient of Q at xcp. c On exit d is the Newton direction of Q. c c xp is a double precision array of dimension n. c used to safeguard the projected Newton direction c c xx is a double precision array of dimension n c On entry it holds the current iterate c On output it is unchanged c gg is a double precision array of dimension n c On entry it holds the gradient at the current iterate c On output it is unchanged c c ws and wy are double precision arrays; c theta is a double precision variable; c col is an integer variable; c head is an integer variable. c On entry they store the information defining the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c theta is the scaling factor specifying B_0 = theta I; c col is the number of variable metric corrections stored; c head is the location of the 1st s- (or y-) vector in S (or Y). c On exit they are unchanged. c c iword is an integer variable. c On entry iword is unspecified. c On exit iword specifies the status of the subspace solution. c iword = 0 if the solution is in the box, c 1 if some bound is encountered. c c wv is a double precision working array of dimension 2m. c c wn is a double precision array of dimension 2m x 2m. c On entry the upper triangle of wn stores the LEL^T factorization c of the indefinite matrix c c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] c On exit wn is unchanged. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return, c = nonzero for abnormal return c when the matrix K is ill-conditioned. c c Subprograms called: c c Linpack dtrsl. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer pointr,m2,col2,ibd,jy,js,i,j,k double precision alpha, xk, dk, temp1, temp2 double precision one,zero parameter (one=1.0d0,zero=0.0d0) c double precision dd_p if (nsub .le. 0) return cw if (iprint .ge. 99) write (6,1001) cx if (iprint .ge. 99) then cx call intpr(' ----- SUBSM entered -----', -1, 0, 0) cw 1001 format (/,'----------------SUBSM entered-----------------',/) cx endif c Compute wv = W'Zd. pointr = head do 20 i = 1, col temp1 = zero temp2 = zero do 10 j = 1, nsub k = ind(j) temp1 = temp1 + wy(k,pointr)*d(j) temp2 = temp2 + ws(k,pointr)*d(j) 10 continue wv(i) = temp1 wv(col + i) = theta*temp2 pointr = mod(pointr,m) + 1 20 continue c Compute wv:=K^(-1)wv. m2 = 2*m col2 = 2*col call dtrsl(wn,m2,col2,wv,11,info) if (info .ne. 0) return do 25 i = 1, col wv(i) = -wv(i) 25 continue call dtrsl(wn,m2,col2,wv,01,info) if (info .ne. 0) return c Compute d = (1/theta)d + (1/theta**2)Z'W wv. pointr = head do 40 jy = 1, col js = col + jy do 30 i = 1, nsub k = ind(i) d(i) = d(i) + wy(k,pointr)*wv(jy)/theta + + ws(k,pointr)*wv(js) 30 continue pointr = mod(pointr,m) + 1 40 continue call dscal( nsub, one/theta, d, 1 ) c c----------------------------------------------------------------- c Let us try the projection, d is the Newton direction iword = 0 call dcopy ( n, x, 1, xp, 1 ) c do 50 i=1, nsub k = ind(i) dk = d(i) xk = x(k) if ( nbd(k) .ne. 0 ) then c if ( nbd(k).eq.1 ) then ! lower bounds only x(k) = max( l(k), xk + dk ) if ( x(k).eq.l(k) ) iword = 1 else c if ( nbd(k).eq.2 ) then ! upper and lower bounds xk = max( l(k), xk + dk ) x(k) = min( u(k), xk ) if (x(k) .eq. l(k) .or. x(k) .eq. u(k)) then iword = 1 endif else c if ( nbd(k).eq.3 ) then ! upper bounds only x(k) = min( u(k), xk + dk ) if ( x(k).eq.u(k) ) iword = 1 end if end if end if c else ! free variables x(k) = xk + dk end if 50 continue c if ( iword.eq.0 ) then go to 911 end if c c check sign of the directional derivative c dd_p = zero do 55 i=1, n dd_p = dd_p + (x(i) - xx(i))*gg(i) 55 continue if ( dd_p .gt.zero ) then call dcopy( n, xp, 1, x, 1 ) cx call intpr(' Positive dir derivative in projection ', -1,0,0) cw write(6,*) ' Positive dir derivative in projection ' cx call intpr(' Using the backtracking step ', -1,0,0) cw write(6,*) ' Using the backtracking step ' else go to 911 endif c c----------------------------------------------------------------- c alpha = one temp1 = alpha ibd = 0 do 60 i = 1, nsub k = ind(i) dk = d(i) if (nbd(k) .ne. 0) then if (dk .lt. zero .and. nbd(k) .le. 2) then temp2 = l(k) - x(k) if (temp2 .ge. zero) then temp1 = zero else if (dk*alpha .lt. temp2) then temp1 = temp2/dk endif else if (dk .gt. zero .and. nbd(k) .ge. 2) then temp2 = u(k) - x(k) if (temp2 .le. zero) then temp1 = zero else if (dk*alpha .gt. temp2) then temp1 = temp2/dk endif endif if (temp1 .lt. alpha) then alpha = temp1 ibd = i endif endif 60 continue if (alpha .lt. one) then dk = d(ibd) k = ind(ibd) if (dk .gt. zero) then x(k) = u(k) d(ibd) = zero else if (dk .lt. zero) then x(k) = l(k) d(ibd) = zero endif endif do 70 i = 1, nsub k = ind(i) x(k) = x(k) + alpha*d(i) 70 continue cccccc 911 continue cw if (iprint .ge. 99) write (6,1004) cx if (iprint .ge. 99) call intpr(' exit SUBSM ', -1, 0,0) cw 1001 format (/,'----------------SUBSM entered-----------------',/) cw 1004 format (/,'----------------exit SUBSM --------------------',/) return end c====================== The end of subsm =============================== subroutine dcsrch(f,g,stp,ftol,gtol,xtol,stpmin,stpmax, + itask,isave,dsave) c character*(*) task integer itask, isave(2) double precision f,g,stp,ftol,gtol,xtol,stpmin,stpmax double precision dsave(13) c ********** c c Subroutine dcsrch c c This subroutine finds a step that satisfies a sufficient c decrease condition and a curvature condition. c c Each call of the subroutine updates an interval with c endpoints stx and sty. The interval is initially chosen c so that it contains a minimizer of the modified function c c psi(stp) = f(stp) - f(0) - ftol*stp*f'(0). c c If psi(stp) <= 0 and f'(stp) >= 0 for some step, then the c interval is chosen so that it contains a minimizer of f. c c The algorithm is designed to find a step that satisfies c the sufficient decrease condition c c f(stp) <= f(0) + ftol*stp*f'(0), c c and the curvature condition c c abs(f'(stp)) <= gtol*abs(f'(0)). c c If ftol is less than gtol and if, for example, the function c is bounded below, then there is always a step which satisfies c both conditions. c c If no step can be found that satisfies both conditions, then c the algorithm stops with a warning. In this case stp only c satisfies the sufficient decrease condition. c c A typical invocation of dcsrch has the following outline: c c task = 'START' c 10 continue c call dcsrch( ... ) c if (task .eq. 'FG') then c Evaluate the function and the gradient at stp c goto 10 c end if c c NOTE: The user must no alter work arrays between calls. c c The subroutine statement is c c subroutine dcsrch(f,g,stp,ftol,gtol,xtol,stpmin,stpmax, c task,isave,dsave) c where c c f is a double precision variable. c On initial entry f is the value of the function at 0. c On subsequent entries f is the value of the c function at stp. c On exit f is the value of the function at stp. c c g is a double precision variable. c On initial entry g is the derivative of the function at 0. c On subsequent entries g is the derivative of the c function at stp. c On exit g is the derivative of the function at stp. c c stp is a double precision variable. c On entry stp is the current estimate of a satisfactory c step. On initial entry, a positive initial estimate c must be provided. c On exit stp is the current estimate of a satisfactory step c if task = 'FG'. If task = 'CONV' then stp satisfies c the sufficient decrease and curvature condition. c c ftol is a double precision variable. c On entry ftol specifies a nonnegative tolerance for the c sufficient decrease condition. c On exit ftol is unchanged. c c gtol is a double precision variable. c On entry gtol specifies a nonnegative tolerance for the c curvature condition. c On exit gtol is unchanged. c c xtol is a double precision variable. c On entry xtol specifies a nonnegative relative tolerance c for an acceptable step. The subroutine exits with a c warning if the relative difference between sty and stx c is less than xtol. c On exit xtol is unchanged. c c stpmin is a double precision variable. c On entry stpmin is a nonnegative lower bound for the step. c On exit stpmin is unchanged. c c stpmax is a double precision variable. c On entry stpmax is a nonnegative upper bound for the step. c On exit stpmax is unchanged. c c task is a character variable of length at least 60. c On initial entry task must be set to 'START'. c On exit task indicates the required action: c c If task(1:2) = 'FG' then evaluate the function and c derivative at stp and call dcsrch again. c c If task(1:4) = 'CONV' then the search is successful. c c If task(1:4) = 'WARN' then the subroutine is not able c to satisfy the convergence conditions. The exit value of c stp contains the best point found during the search. c c If task(1:5) = 'ERROR' then there is an error in the c input arguments. c c On exit with convergence, a warning or an error, the c variable task contains additional information. c c isave is an integer work array of dimension 2. c c dsave is a double precision work array of dimension 13. c c Subprograms called c c MINPACK-2 ... dcstep c c MINPACK-1 Project. June 1983. c Argonne National Laboratory. c Jorge J. More' and David J. Thuente. c c MINPACK-2 Project. October 1993. c Argonne National Laboratory and University of Minnesota. c Brett M. Averick, Richard G. Carter, and Jorge J. More'. c c ********** double precision zero,p5,p66 parameter(zero=0.0d0,p5=0.5d0,p66=0.66d0) double precision xtrapl,xtrapu parameter(xtrapl=1.1d0,xtrapu=4.0d0) logical brackt integer stage double precision finit,ftest,fm,fx,fxm,fy,fym,ginit,gtest, + gm,gx,gxm,gy,gym,stx,sty,stmin,stmax,width,width1 c Initialization block. c if (task(1:5) .eq. 'START') then if (itask .eq. 2) then c Check the input arguments for errors. c if (stp .lt. stpmin) task = 'ERROR: STP .LT. STPMIN' c if (stp .gt. stpmax) task = 'ERROR: STP .GT. STPMAX' c if (g .ge. zero) task = 'ERROR: INITIAL G .GE. ZERO' c if (ftol .lt. zero) task = 'ERROR: FTOL .LT. ZERO' c if (gtol .lt. zero) task = 'ERROR: GTOL .LT. ZERO' c if (xtol .lt. zero) task = 'ERROR: XTOL .LT. ZERO' c if (stpmin .lt. zero) task = 'ERROR: STPMIN .LT. ZERO' c if (stpmax .lt. stpmin) task = 'ERROR: STPMAX .LT. STPMIN' if (stp .lt. stpmin) itask = 16 if (stp .gt. stpmax) itask = 15 if (g .ge. zero) itask = 11 if (ftol .lt. zero) itask = 9 if (gtol .lt. zero) itask = 10 if (xtol .lt. zero) itask = 19 if (stpmin .lt. zero) itask = 18 if (stpmax .lt. stpmin) itask = 17 c Exit if there are errors on input. c if (task(1:5) .eq. 'ERROR') return if ((itask .ge. 9) .and. (itask .le. 19)) return c Initialize local variables. brackt = .false. stage = 1 finit = f ginit = g gtest = ftol*ginit width = stpmax - stpmin width1 = width/p5 c The variables stx, fx, gx contain the values of the step, c function, and derivative at the best step. c The variables sty, fy, gy contain the value of the step, c function, and derivative at sty. c The variables stp, f, g contain the values of the step, c function, and derivative at stp. stx = zero fx = finit gx = ginit sty = zero fy = finit gy = ginit stmin = zero stmax = stp + xtrapu*stp c task = 'FG' itask = 4 goto 1000 else c Restore local variables. if (isave(1) .eq. 1) then brackt = .true. else brackt = .false. endif stage = isave(2) ginit = dsave(1) gtest = dsave(2) gx = dsave(3) gy = dsave(4) finit = dsave(5) fx = dsave(6) fy = dsave(7) stx = dsave(8) sty = dsave(9) stmin = dsave(10) stmax = dsave(11) width = dsave(12) width1 = dsave(13) endif c If psi(stp) <= 0 and f'(stp) >= 0 for some step, then the c algorithm enters the second stage. ftest = finit + stp*gtest if (stage .eq. 1 .and. f .le. ftest .and. g .ge. zero) + stage = 2 c Test for warnings. if (brackt .and. (stp .le. stmin .or. stp .ge. stmax)) c + task = 'WARNING: ROUNDING ERRORS PREVENT PROGRESS' + itask = 23 if (brackt .and. stmax - stmin .le. xtol*stmax) c + task = 'WARNING: XTOL TEST SATISFIED' + itask = 26 if (stp .eq. stpmax .and. f .le. ftest .and. g .le. gtest) c + task = 'WARNING: STP = STPMAX' + itask = 24 if (stp .eq. stpmin .and. (f .gt. ftest .or. g .ge. gtest)) c + task = 'WARNING: STP = STPMIN' + itask = 25 c Test for convergence. if (f .le. ftest .and. abs(g) .le. gtol*(-ginit)) c + task = 'CONVERGENCE' + itask = 6 c Test for termination. c if (task(1:4) .eq. 'WARN' .or. task(1:4) .eq. 'CONV') goto 1000 if ((itask .ge. 23) .or. ((itask .le. 8) + .and. (itask .ge. 6))) goto 1000 c A modified function is used to predict the step during the c first stage if a lower function value has been obtained but c the decrease is not sufficient. if (stage .eq. 1 .and. f .le. fx .and. f .gt. ftest) then c Define the modified function and derivative values. fm = f - stp*gtest fxm = fx - stx*gtest fym = fy - sty*gtest gm = g - gtest gxm = gx - gtest gym = gy - gtest c Call dcstep to update stx, sty, and to compute the new step. call dcstep(stx,fxm,gxm,sty,fym,gym,stp,fm,gm, + brackt,stmin,stmax) c Reset the function and derivative values for f. fx = fxm + stx*gtest fy = fym + sty*gtest gx = gxm + gtest gy = gym + gtest else c Call dcstep to update stx, sty, and to compute the new step. call dcstep(stx,fx,gx,sty,fy,gy,stp,f,g, + brackt,stmin,stmax) endif c Decide if a bisection step is needed. if (brackt) then if (abs(sty-stx) .ge. p66*width1) stp = stx + p5*(sty - stx) width1 = width width = abs(sty-stx) endif c Set the minimum and maximum steps allowed for stp. if (brackt) then stmin = min(stx,sty) stmax = max(stx,sty) else stmin = stp + xtrapl*(stp - stx) stmax = stp + xtrapu*(stp - stx) endif c Force the step to be within the bounds stpmax and stpmin. stp = max(stp,stpmin) stp = min(stp,stpmax) c If further progress is not possible, let stp be the best c point obtained during the search. if (brackt .and. (stp .le. stmin .or. stp .ge. stmax) + .or. (brackt .and. stmax-stmin .le. xtol*stmax)) stp = stx c Obtain another function and derivative. c task = 'FG' itask = 4 1000 continue c Save local variables. if (brackt) then isave(1) = 1 else isave(1) = 0 endif isave(2) = stage dsave(1) = ginit dsave(2) = gtest dsave(3) = gx dsave(4) = gy dsave(5) = finit dsave(6) = fx dsave(7) = fy dsave(8) = stx dsave(9) = sty dsave(10) = stmin dsave(11) = stmax dsave(12) = width dsave(13) = width1 return end c====================== The end of dcsrch ============================== subroutine dcstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt, + stpmin,stpmax) logical brackt double precision stx,fx,dx,sty,fy,dy,stp,fp,dp,stpmin,stpmax c ********** c c Subroutine dcstep c c This subroutine computes a safeguarded step for a search c procedure and updates an interval that contains a step that c satisfies a sufficient decrease and a curvature condition. c c The parameter stx contains the step with the least function c value. If brackt is set to .true. then a minimizer has c been bracketed in an interval with endpoints stx and sty. c The parameter stp contains the current step. c The subroutine assumes that if brackt is set to .true. then c c min(stx,sty) < stp < max(stx,sty), c c and that the derivative at stx is negative in the direction c of the step. c c The subroutine statement is c c subroutine dcstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt, c stpmin,stpmax) c c where c c stx is a double precision variable. c On entry stx is the best step obtained so far and is an c endpoint of the interval that contains the minimizer. c On exit stx is the updated best step. c c fx is a double precision variable. c On entry fx is the function at stx. c On exit fx is the function at stx. c c dx is a double precision variable. c On entry dx is the derivative of the function at c stx. The derivative must be negative in the direction of c the step, that is, dx and stp - stx must have opposite c signs. c On exit dx is the derivative of the function at stx. c c sty is a double precision variable. c On entry sty is the second endpoint of the interval that c contains the minimizer. c On exit sty is the updated endpoint of the interval that c contains the minimizer. c c fy is a double precision variable. c On entry fy is the function at sty. c On exit fy is the function at sty. c c dy is a double precision variable. c On entry dy is the derivative of the function at sty. c On exit dy is the derivative of the function at the exit sty. c c stp is a double precision variable. c On entry stp is the current step. If brackt is set to .true. c then on input stp must be between stx and sty. c On exit stp is a new trial step. c c fp is a double precision variable. c On entry fp is the function at stp c On exit fp is unchanged. c c dp is a double precision variable. c On entry dp is the the derivative of the function at stp. c On exit dp is unchanged. c c brackt is an logical variable. c On entry brackt specifies if a minimizer has been bracketed. c Initially brackt must be set to .false. c On exit brackt specifies if a minimizer has been bracketed. c When a minimizer is bracketed brackt is set to .true. c c stpmin is a double precision variable. c On entry stpmin is a lower bound for the step. c On exit stpmin is unchanged. c c stpmax is a double precision variable. c On entry stpmax is an upper bound for the step. c On exit stpmax is unchanged. c c MINPACK-1 Project. June 1983 c Argonne National Laboratory. c Jorge J. More' and David J. Thuente. c c MINPACK-2 Project. October 1993. c Argonne National Laboratory and University of Minnesota. c Brett M. Averick and Jorge J. More'. c c ********** double precision zero,p66,two,three parameter(zero=0.0d0,p66=0.66d0,two=2.0d0,three=3.0d0) double precision gamma,p,q,r,s,sgnd,stpc,stpf,stpq,theta sgnd = dp*(dx/abs(dx)) c First case: A higher function value. The minimum is bracketed. c If the cubic step is closer to stx than the quadratic step, the c cubic step is taken, otherwise the average of the cubic and c quadratic steps is taken. if (fp .gt. fx) then theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dx/s)*(dp/s)) if (stp .lt. stx) gamma = -gamma p = (gamma - dx) + theta q = ((gamma - dx) + gamma) + dp r = p/q stpc = stx + r*(stp - stx) stpq = stx + ((dx/((fx - fp)/(stp - stx) + dx))/two)* + (stp - stx) if (abs(stpc-stx) .lt. abs(stpq-stx)) then stpf = stpc else stpf = stpc + (stpq - stpc)/two endif brackt = .true. c Second case: A lower function value and derivatives of opposite c sign. The minimum is bracketed. If the cubic step is farther from c stp than the secant step, the cubic step is taken, otherwise the c secant step is taken. else if (sgnd .lt. zero) then theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dx/s)*(dp/s)) if (stp .gt. stx) gamma = -gamma p = (gamma - dp) + theta q = ((gamma - dp) + gamma) + dx r = p/q stpc = stp + r*(stx - stp) stpq = stp + (dp/(dp - dx))*(stx - stp) if (abs(stpc-stp) .gt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif brackt = .true. c Third case: A lower function value, derivatives of the same sign, c and the magnitude of the derivative decreases. else if (abs(dp) .lt. abs(dx)) then c The cubic step is computed only if the cubic tends to infinity c in the direction of the step or if the minimum of the cubic c is beyond stp. Otherwise the cubic step is defined to be the c secant step. theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) c The case gamma = 0 only arises if the cubic does not tend c to infinity in the direction of the step. gamma = s*sqrt(max(zero,(theta/s)**2-(dx/s)*(dp/s))) if (stp .gt. stx) gamma = -gamma p = (gamma - dp) + theta q = (gamma + (dx - dp)) + gamma r = p/q if (r .lt. zero .and. abs(gamma-zero) > 1D-5) then stpc = stp + r*(stx - stp) else if (stp .gt. stx) then stpc = stpmax else stpc = stpmin endif stpq = stp + (dp/(dp - dx))*(stx - stp) if (brackt) then c A minimizer has been bracketed. If the cubic step is c closer to stp than the secant step, the cubic step is c taken, otherwise the secant step is taken. if (abs(stpc-stp) .lt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif if (stp .gt. stx) then stpf = min(stp+p66*(sty-stp),stpf) else stpf = max(stp+p66*(sty-stp),stpf) endif else c A minimizer has not been bracketed. If the cubic step is c farther from stp than the secant step, the cubic step is c taken, otherwise the secant step is taken. if (abs(stpc-stp) .gt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif stpf = min(stpmax,stpf) stpf = max(stpmin,stpf) endif c Fourth case: A lower function value, derivatives of the same sign, c and the magnitude of the derivative does not decrease. If the c minimum is not bracketed, the step is either stpmin or stpmax, c otherwise the cubic step is taken. else if (brackt) then theta = three*(fp - fy)/(sty - stp) + dy + dp s = max(abs(theta),abs(dy),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dy/s)*(dp/s)) if (stp .gt. sty) gamma = -gamma p = (gamma - dp) + theta q = ((gamma - dp) + gamma) + dy r = p/q stpc = stp + r*(sty - stp) stpf = stpc else if (stp .gt. stx) then stpf = stpmax else stpf = stpmin endif endif c Update the interval which contains a minimizer. if (fp .gt. fx) then sty = stp fy = fp dy = dp else if (sgnd .lt. zero) then sty = stx fy = fx dy = dx endif stx = stp fx = fp dx = dp endif c Compute the new step. stp = stpf return end lbfgsb3c/src/RcppExports.cpp0000644000176200001440000000230113625774732015537 0ustar liggesusers// Generated by using Rcpp::compileAttributes() -> do not edit by hand // Generator token: 10BE3573-1514-4C36-9D1C-5A225CD40393 #include "../inst/include/lbfgsb3c.h" #include #include using namespace Rcpp; // lbfgsb3cpp Rcpp::List lbfgsb3cpp(NumericVector par, Function fn, Function gr, NumericVector lower, NumericVector upper, List ctrl, Environment rho); RcppExport SEXP _lbfgsb3c_lbfgsb3cpp(SEXP parSEXP, SEXP fnSEXP, SEXP grSEXP, SEXP lowerSEXP, SEXP upperSEXP, SEXP ctrlSEXP, SEXP rhoSEXP) { BEGIN_RCPP Rcpp::RObject rcpp_result_gen; Rcpp::RNGScope rcpp_rngScope_gen; Rcpp::traits::input_parameter< NumericVector >::type par(parSEXP); Rcpp::traits::input_parameter< Function >::type fn(fnSEXP); Rcpp::traits::input_parameter< Function >::type gr(grSEXP); Rcpp::traits::input_parameter< NumericVector >::type lower(lowerSEXP); Rcpp::traits::input_parameter< NumericVector >::type upper(upperSEXP); Rcpp::traits::input_parameter< List >::type ctrl(ctrlSEXP); Rcpp::traits::input_parameter< Environment >::type rho(rhoSEXP); rcpp_result_gen = Rcpp::wrap(lbfgsb3cpp(par, fn, gr, lower, upper, ctrl, rho)); return rcpp_result_gen; END_RCPP } lbfgsb3c/src/timer.f0000644000176200001440000000220513511470416014016 0ustar liggesusersc c L-BFGS-B is released under the “New BSD License” (aka “Modified BSD License” c or “3-clause license”) c Please read attached file License.txt c subroutine timer(ttime) double precision ttime c real temp c c This routine computes cpu time in double precision; it makes use of c the intrinsic f90 cpu_time therefore a conversion type is c needed. c c J.L Morales Departamento de Matematicas, c Instituto Tecnologico Autonomo de Mexico c Mexico D.F. c c J.L Nocedal Department of Electrical Engineering and c Computer Science. c Northwestern University. Evanston, IL. USA c c January 21, 2011 c temp = sngl(ttime) call cpu_time(temp) ttime = dble(temp) return end lbfgsb3c/src/lbfgsb3x.cpp0000644000176200001440000002634013625774623014770 0ustar liggesusers#include #include #include #include #include #include #define max2( a , b ) ( (a) > (b) ? (a) : (b) ) using namespace Rcpp; extern "C" void setulb_(int *n, int *m, double *x, double *l, double *u, int *nbd, double *f, double *g, double *factr, double *pgtol, double *wa, int *iwa, int *itask, int *iprint, int *icsave, int *lsave, int *isave, double *dsave); typedef double optimfn(int n, double *par, void *ex); typedef void optimgr(int n, double *par, double *gr, void *ex); List lbfgsb3Cinfo; extern "C" void lbfgsb3C_(int n, int lmm, double *x, double *lower, double *upper, int *nbd, double *Fmin, optimfn fn, optimgr gr, int *fail, void *ex, double factr, double pgtol, int *fncount, int *grcount, int maxit, char *msg, int trace, int iprint, double atol, double rtol, double *g){ // Optim compatible interface int itask= 2; // *Fmin=; double *lastx = new double[n]; std::copy(&x[0],&x[0]+n,&lastx[0]); int nwa = 2*lmm*n + 11*lmm*lmm + 5*n + 8*lmm; double *wa= new double[nwa]; int niwa = 3*n; int *iwa= new int[niwa]; int icsave = 0; int lsave[4] = {0}; int isave[44] = {0}; int i=0; double dsave[29]= {0}; // Initial setup int doExit=0; fncount[0]=0; grcount[0]=0; int itask2=0; CharacterVector taskList(28); taskList[0]="NEW_X"; taskList[1]="START"; taskList[2]="STOP"; taskList[3]="FG";//, // 1-4 taskList[4]="ABNORMAL_TERMINATION_IN_LNSRCH"; taskList[5]="CONVERGENCE"; //5-6 taskList[6]="CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL";//7 taskList[7]="CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH";//8 taskList[8]="ERROR: FTOL .LT. ZERO"; //9 taskList[9]="ERROR: GTOL .LT. ZERO";//10 taskList[10]="ERROR: INITIAL G .GE. ZERO"; //11 taskList[11]="ERROR: INVALID NBD"; // 12 taskList[12]="ERROR: N .LE. 0"; // 13 taskList[13]="ERROR: NO FEASIBLE SOLUTION"; // 14 taskList[14]="ERROR: STP .GT. STPMAX"; // 15 taskList[15]="ERROR: STP .LT. STPMIN"; // 16 taskList[16]="ERROR: STPMAX .LT. STPMIN"; // 17 taskList[17]="ERROR: STPMIN .LT. ZERO"; // 18 taskList[18]="ERROR: XTOL .LT. ZERO"; // 19 taskList[19]="FG_LNSRCH"; // 20 taskList[20]="FG_START"; // 21 taskList[21]="RESTART_FROM_LNSRCH"; // 22 taskList[22]="WARNING: ROUNDING ERRORS PREVENT PROGRESS"; // 23 taskList[23]="WARNING: STP .eq. STPMAX"; // 24 taskList[24]="WARNING: STP .eq. STPMIN"; // 25 taskList[25]="WARNING: XTOL TEST SATISFIED"; // taskList[26] = "CONVERGENCE: Parameters differences below xtol"; taskList[27] = "Maximum number of iterations reached"; while (true){ if (trace >= 2){ Rprintf("\n================================================================================\nBefore call f=%f task number %d, or \"%s\"\n", *Fmin, itask, (as(taskList[itask-1])).c_str()); } if (itask==3) doExit=1; setulb_(&n, &lmm, x, lower, upper, nbd, Fmin, g, &factr, &pgtol, wa, iwa, &itask, &iprint, &icsave, lsave, isave, dsave); if (trace > 2) { Rprintf("returned from lbfgsb3 \n"); Rprintf("returned itask is %d or \"%s\"\n",itask,(as(taskList[itask-1])).c_str()); } switch (itask){ case 4: case 20: case 21: if (trace >= 2) { Rprintf("computing f and g at prm=\n"); NumericVector xv(n); std::copy(&x[0],&x[0]+n,&xv[0]); print(xv); } // Calculate f and g Fmin[0] = fn(n, x, ex); fncount[0]++; gr(n, x, g, ex); grcount[0]++; if (trace > 0) { Rprintf("At iteration %d f=%f ", isave[33], *Fmin); if (trace > 1) { double tmp = fabs(g[n-1]); for (unsigned int j=n-1; j--;){ if (tmp > fabs(g[j])){ tmp = fabs(g[j]); } } Rprintf("max(abs(g))=%f",tmp); } Rprintf("\n"); } break; case 1: // New x; if (maxit <= fncount[0]){ itask2=28; doExit=1; itask=3; // Stop -- gives the right results and restores gradients if (trace > 2){ Rprintf("Exit becuase maximum number of function calls %d met.\n", maxit); } } else { bool converge=fabs(lastx[n-1]-x[n-1]) < fabs(x[n-1])*rtol+atol; if (converge){ for (i=n-1;i--;){ converge=fabs(lastx[i]-x[i]) < fabs(x[i])*rtol+atol; if (!converge){ break; } } } if (converge){ itask2=27; itask=3; // Stop -- gives the right results and restores gradients if (trace > 2){ Rprintf("CONVERGENCE: Parameters differences below xtol.\n", maxit); } doExit=1; } } std::copy(&x[0],&x[0]+n,&lastx[0]); break; default: doExit=1; } if (doExit) break; } if (itask2){ itask=itask2; } LogicalVector lsaveR(4); NumericVector dsaveR(29); IntegerVector isaveR(44); std::copy(&lsave[0],&lsave[0]+4, &lsaveR[0]); std::copy(&dsave[0],&dsave[0]+29, dsaveR.begin()); std::copy(&isave[0],&isave[0]+44, isaveR.begin());; CharacterVector taskR(1); taskR[0] = taskList[itask-1]; lbfgsb3Cinfo = List::create(_["task"] = taskR, _["itask"]= IntegerVector::create(itask), _["lsave"]= lsaveR, _["icsave"]= IntegerVector::create(icsave), _["dsave"]= dsaveR, _["isave"] = isaveR); // info <- list(task = task, itask = itask, lsave = lsave, // icsave = icsave, dsave = dsave, isave = isave) fail[0]= itask; delete[] wa; delete[] iwa; delete[] lastx; } Environment grho; CharacterVector gnames; List ev; double gfn(int n, double *x, void *ex){ Rcpp::NumericVector par(n); std::copy(&x[0], &x[0]+n, &par[0]); Function fn = as(ev["fn"]); par.attr("names") = ev["pn"]; double ret = as(fn(par, grho)); return ret; } void ggr(int n, double *x, double *gr, void *ex){ Rcpp::NumericVector par(n), ret(n); std::copy(&x[0], &x[0]+n, &par[0]); Function grad = as(ev["gr"]); par.attr("names") = ev["pn"]; ret = grad(par, grho); std::copy(&ret[0], &ret[0]+n, &gr[0]); } //[[Rcpp::export]] Rcpp::List lbfgsb3cpp(NumericVector par, Function fn, Function gr, NumericVector lower, NumericVector upper, List ctrl, Environment rho){ Rcpp::List ret; ev["fn"] = fn; ev["gr"] = gr; ev["pn"] = par.attr("names"); Rcpp::NumericVector g(par.size()); // CONV in 6, 7, 8; ERROR in 9-19; WARN in 23-26 IntegerVector traceI = as(ctrl["trace"]); if (traceI.size() != 1) stop("trace has to have one element in it."); int trace = traceI[0]; NumericVector factrN = as(ctrl["factr"]); if (factrN.size() != 1) stop("factr has to have one element in it."); double factr = factrN[0]; NumericVector pgtolN = as(ctrl["pgtol"]); if (pgtolN.size() != 1) stop("pgtol has to have one element in it."); double pgtol = pgtolN[0]; NumericVector atolN = as(ctrl["abstol"]); if (atolN.size() != 1) stop("abstol has to have one element in it."); double atol = atolN[0]; NumericVector rtolN = as(ctrl["reltol"]); if (atolN.size() != 1) stop("reltol has to have one element in it."); double rtol = rtolN[0]; LogicalVector infoN = as(ctrl["info"]); if (infoN.size() != 1) stop("info has to have one element in it."); bool addInfo = infoN[0]; IntegerVector lmmN = as(ctrl["lmm"]); if (lmmN.size() != 1) stop("lmm has to have one element in it."); int lmm = lmmN.size(); int n = par.size(); IntegerVector maxitN = as(ctrl["maxit"]); if (maxitN.size() != 1) stop("maxit has to have one element in it."); int maxit = maxitN[0]; IntegerVector iprintN = as(ctrl["iprint"]); if (iprintN.size() != 1) stop("iprint has to have one element in it."); int iprint = iprintN[0]; // double *g = new double[par.size()]; double *low = new double[par.size()]; if (lower.size() == 1){ std::fill_n(&low[0],par.size(),lower[0]); } else if (lower.size() == par.size()){ std::copy(lower.begin(),lower.end(),&low[0]); } else { delete [] low; stop("Lower bound must match the size of par or only have one element."); } double *up = new double[par.size()]; if (upper.size() == 1){ std::fill_n(&up[0],par.size(),upper[0]); } else if (upper.size() == par.size()){ std::copy(upper.begin(),upper.end(),&up[0]); } else { delete [] low; delete [] up; stop("Upper bound must match the size of par or only have one element."); } double *x = new double[par.size()]; std::copy(par.begin(),par.end(),&x[0]); int *nbd = new int[par.size()]; int i; for (i = par.size();i--;){ /* nbd(i)=0 if x(i) is unbounded, 1 if x(i) has only a lower bound, 2 if x(i) has both lower and upper bounds, 3 if x(i) has only an upper bound. */ nbd[i] = 0; if (R_FINITE(low[i])) nbd[i] = 1; if (R_FINITE(up[i])) nbd[i] = 3 - nbd[i]; } double fmin=std::numeric_limits::max(); int fail = 0, fncount=0, grcount=0; grho=rho; //void *ex = (void*)rho; //Should work but use global instead. void *ex =NULL; char msg[120]; lbfgsb3C_(n, lmm, x, low, up, nbd, &fmin, gfn, ggr, &fail, ex, factr, pgtol, &fncount, &grcount, maxit, msg, trace, iprint , atol, rtol, &g[0]); NumericVector parf(par.size()); std::copy(&x[0],&x[0]+par.size(),parf.begin()); parf.attr("names")=ev["pn"]; g.attr("names")=ev["pn"]; ret["par"]=parf; ret["grad"]=g; ret["value"] = fmin; IntegerVector cnt = IntegerVector::create(fncount,grcount); ret["counts"] = cnt; switch (fail){ case 6: case 7: case 8: case 27: ret["convergence"]=0; break; case 28: ret["convergence"]=1; break; case 23: case 24: case 25: case 26: ret["convergence"] = 51; break; case 9: case 10: case 11: case 12: case 13: case 14: case 15: case 16: case 17: case 18: case 19: ret["convergence"] = 52; break; } CharacterVector taskList(28); taskList[0]="NEW_X"; taskList[1]="START"; taskList[2]="STOP"; taskList[3]="FG";//, // 1-4 taskList[4]="ABNORMAL_TERMINATION_IN_LNSRCH"; taskList[5]="CONVERGENCE"; //5-6 taskList[6]="CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL";//7 taskList[7]="CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH";//8 taskList[8]="ERROR: FTOL .LT. ZERO"; //9 taskList[9]="ERROR: GTOL .LT. ZERO";//10 taskList[10]="ERROR: INITIAL G .GE. ZERO"; //11 taskList[11]="ERROR: INVALID NBD"; // 12 taskList[12]="ERROR: N .LE. 0"; // 13 taskList[13]="ERROR: NO FEASIBLE SOLUTION"; // 14 taskList[14]="ERROR: STP .GT. STPMAX"; // 15 taskList[15]="ERROR: STP .LT. STPMIN"; // 16 taskList[16]="ERROR: STPMAX .LT. STPMIN"; // 17 taskList[17]="ERROR: STPMIN .LT. ZERO"; // 18 taskList[18]="ERROR: XTOL .LT. ZERO"; // 19 taskList[19]="FG_LNSRCH"; // 20 taskList[20]="FG_START"; // 21 taskList[21]="RESTART_FROM_LNSRCH"; // 22 taskList[22]="WARNING: ROUNDING ERRORS PREVENT PROGRESS"; // 23 taskList[23]="WARNING: STP .eq. STPMAX"; // 24 taskList[24]="WARNING: STP .eq. STPMIN"; // 25 taskList[25]="WARNING: XTOL TEST SATISFIED"; // taskList[26] = "CONVERGENCE: Parameters differences below xtol"; taskList[27] = "Maximum number of iterations reached"; ret["message"]= CharacterVector::create(taskList[fail-1]); if (addInfo) ret["info"] = lbfgsb3Cinfo; delete [] x; delete [] low; delete [] up; delete [] nbd; return ret; } lbfgsb3c/vignettes/0000755000176200001440000000000013627233177013762 5ustar liggesuserslbfgsb3c/vignettes/lbfgsb3c.Rmd0000644000176200001440000002716113626305436016117 0ustar liggesusers--- title: "lbfgsb3c: Using the 2011 version of L-BFGS-B." author: "John C Nash Telfer School of Management, University of Ottawa, nashjc@uottawa.ca" date: "October 22, 2019" output: pdf_document bibliography: lbfgsb3c.bib vignette: > %\VignetteIndexEntry{lbfgsb3c-nocompile} %\VignetteEngine{knitr::rmarkdown} \usepackage[utf8]{inputenc} --- ```{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE) ## Added 20190717 to get vignette to build pkgbuild::compile_dll() ``` # Abstract In 2011 the authors of the L-BFGSB program published a correction and update to their 1995 code. The latter is the basis of the L-BFGS-B method of the `optim()` function in base-R. The package `lbfgsb3` wrapped the updated code using a `.Fortran` call after removing a very large number of Fortran output statements. Matthew Fidler used this Fortran code and an `Rcpp` interface to produce package `lbfgsb3c` where the function `lbfgsb3c()` returns an object similar to that of base-R `optim()` and that of `optimx::optimr()`. Subsequently, in a fine example of the collaborations that have made **R** so useful, we have merged the functionality of package `lbfgsb3` into `lbfgsb3c`, as explained in this vignette. Note that this document is intended primarily to document our efforts to check the differences in variants of the code rather than be expository. ## Provenance of the R optim::L-BFGS-B and related solvers The base-R code lbfgsb.c (at writing in R-3.5.2/src/appl/) is commented: ``` /* l-bfgs-b.f -- translated by f2c (version 19991025). From ?optim: The code for method ‘"L-BFGS-B"’ is based on Fortran code by Zhu, Byrd, Lu-Chen and Nocedal obtained from Netlib (file 'opt/lbfgs_bcm.shar') The Fortran files contained no copyright information. Byrd, R. H., Lu, P., Nocedal, J. and Zhu, C. (1995) A limited memory algorithm for bound constrained optimization. \emph{SIAM J. Scientific Computing}, \bold{16}, 1190--1208. */ ``` The paper @Byrd95 builds on @Lu94limitedmemory. There have been a number of other workers who have followed-up on this work, but **R** code and packages seem to have largely stayed with codes derived from these original papers. Though the date of the paper is 1995, the ideas it embodies were around for a decade and a half at least, in particular in Nocedal80 and LiuN89. The definitive Fortran code was published as @Zhu1997LBFGS. This is available as `toms/778.zip` on www.netlib.org. A side-by-side comparison of the main subroutines in the two downloads from Netlib unfortunately shows a lot of differences. I have not tried to determine if these affect performance or are simply cosmetic. More seriously perhaps, there were some deficiencies in the code(s), and in 2011 Nocedal's team published a Fortran code with some corrections (@Morales2011). Since the **R** code predates this, I prepared package `lbfgsb3` (@lbfgsb3JN) to wrap the Fortran code. However, I did not discover any test cases where the `optim::L-BFGS-B` and `lbfgsb3` were different, though I confess to only running some limited tests. There are, in fact, more in this vignette. In 2016, I was at a Fields Institute optimization conference in Toronto for the 70th birthday of Andy Conn. By sheer serendipity, Nocedal did not attend the conference, but sat down next to me at the conference dinner. When I asked him about the key changes, he said that the most important one was to fix the computation of the machine precision, which was not always correct in the 1995 code. Since **R** gets this number as `.Machine$double.eps`, the offending code is irrelevant. Within @Morales2011, there is also reported an improvement in the subspace minimization that is applied in cases of bounds constraints. Since few of the tests I have applied impose such constraints, it is reasonable that I will not have observed performance differences between the base-R `optim` code and my `lbfsgb3` package. More appropriate tests are welcome, and on my agenda. Besides the ACM TOMS code, there are two related codes from the Northwestern team on NETLIB: http://netlib.org/opt/lbfgs_um.shar is for unconstrained minimization, while http://netlib.org/opt/lbfgs_bcm.shar handles bounds constrained problems. To these are attached references @LiuN89 and @Byrd1995 respectively, most likely reflecting the effort required to implement the constraints. The unconstrained code has been converted to **C** under the leadership of Naoaki Okazaki (see http://www.chokkan.org/software/liblbfgs/, or the fork at https://github.com/MIRTK/LBFGS). This has been wrapped for **R** as @Coppola2014 as the `lbfgs` package. This can be called from `optimx::optimr()`. Using Rcpp (see @RCppDERF2011) and the Fortran code in package `lbfgs3`, Matthew Fidler developed package `lbfgsb3c` (@lbfgsb3cMF). As this provides a more standard call and return than `lbfgsb3` Fidler and I are unified the two packages and released them both under the same name `lbfgsb3c`. ## Functions in package `lbfgsb3c` There is really only one optimizer function in the package, but it may be called by four (4) names: - `lbfgsb3c()` uses Rcpp (@RCppDE2013, @RCppDERF2011, @RCppDEJJB2017) to streamline the call to the underlying Fortran. This is the base function used. - `lbfgsb3x()` is an alias of `lbfgsb3c()`. We were using this name for a while, and have kept the alias to avoid having to edit test scripts. - `lbfgsb3`, which imitates a `.Fortran` call of the compiled 2011 Fortran code. The object returned by this routine is NOT equivalent to the object returned by base-R `optim()` or by `optimx::optimr()`. Instead, it includes a structure `info` which contains the detailed diagnostic information of the Fortran code. For most users, this is not of interest, and I only recommend use of this function for those needing to examine how the optimization has been carried out. - `lbfgsb3f()` is an alias of `lbfsgb3()`. We recommend using the `lbfsgb3c()` call for most uses. ### Candlestick function ```{r, candlestick} # candlestick function # J C Nash 2011-2-3 cstick.f<-function(x,alpha=100){ x<-as.vector(x) r2<-crossprod(x) f<-as.double(r2+alpha/r2) return(f) } cstick.g<-function(x,alpha=100){ x<-as.vector(x) r2<-as.numeric(crossprod(x)) g1<-2*x g2 <- (-alpha)*2*x/(r2*r2) g<-as.double(g1+g2) return(g) } library(lbfgsb3c) nn <- 2 x0 <- c(10,10) lo <- c(1, 1) up <- c(10,10) print(x0) ## c2o <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0)) ## print(summary(c2o, order=value)) c2l1 <- lbfgsb3c(x0, cstick.f, cstick.g, lower=lo, upper=up) c2l1 ## meths <- c("L-BFGS-B", "lbfgsb3c", "Rvmmin", "Rcgmin", "Rtnmin") ## require(optimx) ## cstick2a <- opm(x0, cstick.f, cstick.g, method=meths, upper=up, lower=lo, control=list(kkt=FALSE)) ## print(summary(cstick2a, par.select=1:2, order=value)) lo <- c(4, 4) ## c2ob <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0)) ## print(summary(c2ob, order=value)) c2l2 <- lbfgsb3c(x0, cstick.f, cstick.g, lower=lo, upper=up) c2l2 ## cstick2b <- opm(x0, cstick.f, cstick.g, method=meths, upper=up, lower=lo, control=list(kkt=FALSE)) ## print(summary(cstick2b, par.select=1:2, order=value)) ## nn <- 100 ## x0 <- rep(10, nn) ## up <- rep(10, nn) ## lo <- rep(1e-4, nn) ## cco <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0, kkt=FALSE)) ## print(summary(cco, par.select=1:4, order=value)) ``` ### Extended Rosenbrock function (from funconstrain) ```{r,exrosen} # require(funconstrain) ## not in CRAN, so explicit inclusion of this function # exrosen <- ex_rosen() # exrosenf <- exrosen$fn exrosenf <- function (par) { n <- length(par) if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } fsum <- 0 for (i in 1:(n/2)) { p2 <- 2 * i p1 <- p2 - 1 f_p1 <- 10 * (par[p2] - par[p1]^2) f_p2 <- 1 - par[p1] fsum <- fsum + f_p1 * f_p1 + f_p2 * f_p2 } fsum } # exroseng <- exrosen$gr exroseng <- function (par) { n <- length(par) if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } grad <- rep(0, n) for (i in 1:(n/2)) { p2 <- 2 * i p1 <- p2 - 1 xx <- par[p1] * par[p1] yx <- par[p2] - xx f_p1 <- 10 * yx f_p2 <- 1 - par[p1] grad[p1] <- grad[p1] - 400 * par[p1] * yx - 2 * f_p2 grad[p2] <- grad[p2] + 200 * yx } grad } exrosenx0 <- function (n = 20) { if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } rep(c(-1.2, 1), n/2) } require(lbfgsb3c) ## require(optimx) ## require(optimx) for (n in seq(2,12, by=2)) { cat("ex_rosen try for n=",n,"\n") x0 <- exrosenx0(n) lo <- rep(-1.5, n) up <- rep(3, n) print(x0) cat("optim L-BFGS-B\n") eo <- optim(x0, exrosenf, exroseng, lower=lo, upper=up, method="L-BFGS-B", control=list(trace=0)) print(eo) cat("lbfgsb3c\n") el <- lbfgsb3c(x0, exrosenf, exroseng, lower=lo, upper=up, control=list(trace=0)) print(el) ## erfg <- opm(x0, exrosenf, exroseng, method=meths, lower=lo, upper=up) ## print(summary(erfg, par.select=1:2, order=value)) } ``` ## Using compiled function code While you may use the same interface as described in the writing R extensions to interface compiled code with this function, see [L-BFGS-B](https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Optimization), it is sometimes more convenient to use your own compiled code. The following example shows how this is done using the file `jrosen.f`. We have unfortunately found that compilation is not always portable across systems, so this example is presented without execution. ``` subroutine rosen(n, x, fval) double precision x(n), fval, dx integer n, i fval = 0.0D0 do 10 i=1,(n-1) dx = x(i + 1) - x(i) * x(i) fval = fval + 100.0 * dx * dx dx = 1.0 - x(i) fval = fval + dx * dx 10 continue return end ``` Here is the example script. Note that we must have the file `jrosen.f` available. Because the executable files on different systems use different conventions and structures, we have turned evaluation off here so this vignette can be built on multiple platforms. However, we wished to provide examples of how compiled code could be used. ```{r, usingFortran, eval=FALSE } system("R CMD SHLIB jrosen.f") dyn.load("jrosen.so") is.loaded("rosen") x0 <- as.double(c(-1.2,1)) fv <- as.double(-999) n <- as.double(2) testf <- .Fortran("rosen", n=as.integer(n), x=as.double(x0), fval=as.double(fv)) testf rrosen <- function(x) { fval <- 0.0 for (i in 1:(n-1)) { dx <- x[i + 1] - x[i] * x[i] fval <- fval + 100.0 * dx * dx dx <- 1.0 - x[i] fval <- fval + dx * dx } fval } (rrosen(x0)) frosen <- function(x){ nn <- length(x) if (nn > 100) { stop("max number of parameters is 100")} fv <- -999.0 val <- .Fortran("rosen", n=as.integer(nn), x=as.double(x), fval=as.double(fv)) val$fval # NOTE--need ONLY function value returned } # Test the funcion tval <- frosen(x0) str(tval) cat("Run with Nelder-Mead using R function\n") mynm <- optim(x0, rrosen, control=list(trace=0)) print(mynm) cat("\n\n Run with Nelder-Mead using Fortran function") mynmf <- optim(x0, frosen, control=list(trace=0)) print(mynmf) library(lbfgsb3c) library(microbenchmark) cat("try lbfgsb3c, no Gradient \n") cat("R function\n") tlR<-microbenchmark(myopR <- lbfgsb3c(x0, rrosen, gr=NULL, control=list(trace=0))) print(tlR) print(myopR) cat("Fortran function\n") tlF<-microbenchmark(myop <- lbfgsb3c(x0, frosen, gr=NULL, control=list(trace=0))) print(tlF) print(myop) ``` In this example, Fortran execution was actually SLOWER than plain R on the system where it was run. ## References lbfgsb3c/vignettes/lbfgsb3c.bib0000644000176200001440000035710613511470416016130 0ustar liggesusers@Manual{lbfgsb3cMF, title = {{lbfgsb3c}: Limited Memory BFGS Minimizer with Bounds on Parameters with optim() 'C' Interface}, author = {Matthew L Fidler and John C Nash and Ciyou Zhu and Richard Byrd and Jorge Nocedal and Jose Luis Morales}, year = {2018}, note = {R package version 2018-2.13}, url = {https://CRAN.R-project.org/package=lbfgsb3c}, } @Manual{lbfgsb3JN, title = {{lbfgsb3}: Limited Memory BFGS Minimizer with Bounds on Parameters}, author = {John C Nash and Ciyou Zhu and Richard Byrd and Jorge Nocedal and Jose Luis Morales}, year = {2015}, note = {R package version 2015-2.13}, url = {https://CRAN.R-project.org/package=lbfgsb3}, } @Article{RCppDERF2011, title = {{Rcpp}: Seamless {R} and {C++} Integration}, author = {Dirk Eddelbuettel and Romain Fran\c{c}ois}, journal = {Journal of Statistical Software}, year = {2011}, volume = {40}, number = {8}, pages = {1--18}, url = {http://www.jstatsoft.org/v40/i08/}, doi = {10.18637/jss.v040.i08}, } @Book{RCppDE2013, title = {Seamless {R} and {C++} Integration with {Rcpp}}, author = {Dirk Eddelbuettel}, publisher = {Springer}, address = {New York}, year = {2013}, note = {ISBN 978-1-4614-6867-7}, doi = {10.1007/978-1-4614-6868-4}, } @Article{RCppDEJJB2017, title = {{Extending extit{R} with extit{C++}: A Brief Introduction to extit{Rcpp}}}, author = {Dirk Eddelbuettel and James Joseph Balamuta}, journal = {PeerJ Preprints}, year = {2017}, month = {aug}, volume = {5}, pages = {e3188v1}, issn = {2167-9843}, url = {https://doi.org/10.7287/peerj.preprints.3188v1}, doi = {10.7287/peerj.preprints.3188v1}, } @Manual{Varadhan2015, title = {alabama: Constrained Nonlinear Optimization}, author = {Ravi Varadhan}, year = {2015}, note = {R package version 2015.3-1}, url = {https://CRAN.R-project.org/package=alabama}, } @Article{DEJJB2017, title = {{Extending extit{R} with extit{C++}: A Brief Introduction to extit{Rcpp}}}, author = {Dirk Eddelbuettel and James Joseph Balamuta}, journal = {PeerJ Preprints}, year = {2017}, month = {aug}, volume = {5}, pages = {e3188v1}, issn = {2167-9843}, url = {https://doi.org/10.7287/peerj.preprints.3188v1}, doi = {10.7287/peerj.preprints.3188v1}, } @Manual{Fidler2018, title = {{lbfgsb3c}: Limited Memory BFGS Minimizer with Bounds on Parameters with optim() 'C' Interface}, author = {Matthew L Fidler and John C Nash and Ciyou Zhu and Richard Byrd and Jorge Nocedal and Jose Luis Morales}, year = {2018}, note = {R package version 2018-2.13-1}, url = {https://CRAN.R-project.org/package=lbfgsb3c}, } @Manual{Coppola2014, title = {lbfgs: Limited-memory BFGS Optimization}, author = {Antonio Coppola and Brandon Stewart and Naoaki Okazaki}, year = {2014}, note = {R package version 1.2.1}, url = {https://CRAN.R-project.org/package=lbfgs}, } @article{Byrd1995, author = {Byrd, Richard H. and Lu, Peihuang and Nocedal, Jorge and Zhu, Ciyou}, title = {A Limited Memory Algorithm for Bound Constrained Optimization}, journal = {SIAM J. Sci. Comput.}, issue_date = {Sept. 1995}, volume = {16}, number = {5}, month = sep, year = {1995}, issn = {1064-8275}, pages = {1190--1208}, numpages = {19}, url = {http://dx.doi.org/10.1137/0916069}, doi = {10.1137/0916069}, acmid = {210980}, publisher = {Society for Industrial and Applied Mathematics}, address = {Philadelphia, PA, USA} } @article{Nocedal80, title = "Updating Quasi-Newton Matrices with Limited Storage", author = "Jorge Nocedal", year = "1980", month = "7", doi = "10.1090/S0025-5718-1980-0572855-7", language = "English", volume = "35", pages = "773--782", journal = "Mathematics of Computation" } @article{LiuN89, author = {Dong C. Liu and Jorge Nocedal}, title = {On the limited memory {BFGS} method for large scale optimization}, journal = {Math. Program.}, volume = {45}, number = {1-3}, pages = {503--528}, year = {1989}, url = {https://doi.org/10.1007/BF01589116}, doi = {10.1007/BF01589116}, biburl = {https://dblp.org/rec/bib/journals/mp/LiuN89}, bibsource = {dblp computer science bibliography, https://dblp.org} } @article{polak1969, author = {Polak, E. and Ribiere, G.}, journal = {Revue Fran{\c{c}}aise d'Informatique et de Recherche Op{\'{e}}rationnelle}, pages = {35--43}, title = {{Note sur la convergence de m{\'{e}}thodes de directions conjugu{\'{e}}es}}, volume = {16}, year = {1969} } @article{Sorenson1969, author = {Sorenson, Harold}, year = {1969}, month = {12}, pages = {421--441}, title = {Comparison of some conjugate direction procedures for function minimization}, volume = {288}, journal = {Journal of The Franklin Institute - Engineering and Applied Mathematics}, doi = {10.1016/0016-0032(69)90253-1} } @incollection{Beale1972, author = {Beale, E. M. L.}, title = {A Derivation of Conjugate Gradients}, booktitle = {Numerical Methods for Nonlinear Optimization}, editor = {F. A. Lootsma}, publisher = {Academic Press}, address = {London}, pages = {39--43}, year = 1972 } @article{Gorn1960, author = {Gorn, S.}, title = {Algorithms: Bisection Routine}, journal = {Commun. ACM}, issue_date = {March 1960}, volume = {3}, number = {3}, month = mar, year = {1960}, issn = {0001-0782}, pages = {174--}, url = {http://doi.acm.org/10.1145/367149.367173}, doi = {10.1145/367149.367173}, acmid = {367173}, publisher = {ACM}, address = {New York, NY, USA}, } @article{Rader1961, author = {Rader, Patty Jane}, title = {Certification of Algorithm 4: Bisection Routine}, journal = {Commun. ACM}, issue_date = {March 1961}, volume = {4}, number = {3}, month = mar, year = {1961}, issn = {0001-0782}, pages = {153--}, url = {http://doi.acm.org/10.1145/366199.366275}, doi = {10.1145/366199.366275}, acmid = {366275}, publisher = {ACM}, address = {New York, NY, USA}, } @article{Muller56, author = {Muller, David E.}, journal = {Mathematical Tables and Other Aids to Computation}, note = {Later Mathematics of Computation.}, pages = { 208--215}, title = {A Method for Solving Algebraic Equations Using an Automatic Computer}, volume = 10, year = 1956 } @article{Ridders79, author = {Ridders, C.}, journal = {Circuits and Systems, IEEE Transactions on}, month = {11}, number = 11, pages = { 979 - 980}, title = {A new algorithm for computing a single root of a real continuous function}, volume = 26, year = 1979 } @InProceedings{Gonnet2001, author="Gonnet, Gaston H.", editor="Casacuberta, Carles and Mir{\'o}-Roig, Rosa Maria and Verdera, Joan and Xamb{\'o}-Descamps, Sebasti{\`a}", title="A Study of Iteration Formulas for Root Finding, Where Mathematics, Computer Algebra and Software Engineering Meet", booktitle="European Congress of Mathematics", year="2001", publisher="Birkh{\"a}user Basel", address="Basel", pages="143--156" } @Misc{CoreTeam2013, Title = {{Writing \R\ Extensions}}, Author = {{\R\ Core Team}}, Year = {2013}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://CRAN.R-project.org/doc/manuals/R-exts.html} } @Misc{R-extns, Title = {{Writing \R\ Extensions}}, Author = {{\R\ Core Team}}, Year = {2013}, Url = {http://CRAN.R-project.org/doc/manuals/R-exts.html} } @Book{acton70, Title = {Numerical Methods that Work}, Author = {Forman S. Acton}, Publisher = {Harper and Row}, Year = {1970}, Address = {New York} } @Article{Grassmann12, Title = {{GrassmannOptim}: An {R} Package for Grassmann Manifold Optimization}, Author = {Kofi P. Adragni and R. Dennis Cook and Seongho Wu}, Journal = {Journal of Statistical Software}, Year = {2012}, Number = {5}, Pages = {1--18}, Volume = {50}, Url = {http://www.jstatsoft.org/v50/i05/} } @Article{DEoptimWilmott11, Title = {{J}ump-Diffusion Calibration using {D}ifferential {E}volution}, Author = {David Ardia and Juan Ospina Arango and Norman Giraldo Gomez}, Journal = {Wilmott Magazine}, Year = {2011}, Pages = {76--79}, Volume = {55}, Url = {http://www.wilmott.com/} } @Article{DEoptimRjournal11, Title = {{D}ifferential {E}volution with {DEoptim}: An Application to Non-Convex Portfolio Optimization}, Author = {David Ardia and Kris Boudt and Peter Carl and Katharine M. Mullen and Brian G. Peterson}, Journal = {The R Journal}, Year = {2011}, Number = {1}, Pages = {27--34}, Volume = {3}, Url = {http://journal.r-project.org/archive/2011-1/2011-1\_index.html} } @Manual{DEoptim13, Title = {{DEoptim}: Differential Evolution in {R}}, Author = {David Ardia and Katharine M. Mullen and Brian G. Peterson and Joshua Ulrich}, Note = {version 2.2-2}, Year = {2013}, Url = {http://CRAN.R-project.org/package=DEoptim} } @Article{Barzilai88, Title = {Two-point step size gradient methods}, Author = {Jonathan Barzilai and Jonathan M. Borwein}, Journal = {IMA Journal of Numerical Analysis}, Year = {1988}, Number = {1}, Pages = {141--148}, Volume = {8} } @Manual{p-minqa, Title = {\pkg{minqa}: Derivative-Free Optimization Algorithms by Quadratic Approximation}, Author = {Douglas Bates and Katharine M. Mullen and John C. Nash and Ravi Varadhan}, Note = {R package version 1.1.13}, Organization = {R Foundation for Statistical Computing}, Year = {2010}, Url = {http://CRAN.R-project.org/package=minqa} } @Book{bateswatts, Title = {Nonlinear Regression Analysis and Its Applications}, Author = {D. M. Bates and D. G. Watts}, Publisher = {Wiley}, Year = {1988} } @Article{BatesWatts81, Title = {A Relative Off set Orthogonality Convergence Criterion for Nonlinear least Squares}, Author = {Bates, Douglas M. and Watts, Donald G.}, Journal = {Technometrics}, Year = {1981}, Number = {2}, Pages = {179-183}, Volume = {23} } @Manual{p-nlstools, Title = {nlstools: tools for nonlinear regression diagnostics}, Author = {Florent Baty and Marie-Laure Delignette-Muller}, Note = {R package version 0.0-15}, Year = {2013} } @Article{Belisle1992, Title = {Convergence Theorems for a Class of Simulated Annealing Algorithms on {\$\R\^{}d\$}}, Author = {C. J. P. Belisle}, Journal = {Annals of Applied Probability}, Year = {1992}, Pages = {885--895}, Volume = {29}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @Article{Belisle92, Title = {Convergence Theorems for a Class of Simulated Annealing Algorithms on {$\R^d$}}, Author = {C. J. P. Belisle}, Journal = {Annals of Applied Probability}, Year = {1992}, Pages = {885--895}, Volume = {29} } @Manual{Rmalschains12, Title = {Continuous Optimization using Memetic Algorithms with Local Search Chains (MA-LS-Chains) in R}, Author = {Christoph Bergmeir and Daniel Molina and Jos\'e M. Ben\'itez}, Note = {R package version 0.1}, Year = {2012} } @Manual{p-lpSolve, Title = {{lpSolve: Interface to Lp\_solve v. 5.5 to solve linear / integer programs}}, Author = {Michel Berkelaar and {others}}, Note = {R package version 5.6.7}, Year = {2013}, Url = {http://CRAN.R-project.org/package=lpSolve} } @Manual{p-quadprog, Title = {quadprog: Functions to solve Quadratic Programming Problems.}, Author = {S original by Berwin A. Turlach R port by Andreas Weingessel}, Note = {R package version 1.5-5}, Year = {2013}, Url = {http://CRAN.R-project.org/package=quadprog} } @PhdThesis{Bhat2011, Title = {Some aspects of mathematical programming in statistics}, Author = {Bhat, K. A.}, School = {Post Graduate Deptartment of Statistics, Univ. of Kashmir}, Year = {2011}, Address = {Srinagar, India}, Note = {Ph. D. Dissertation} } @Article{Birgin2000, Title = {Nonmonotone Spectral Projected Gradient Methods on Convex Sets}, Author = {Ernesto G. Birgin and Jos{\'e} Mario Mart{\'\i}nez and Marcos Raydan}, Journal = {SIAM J Optimization}, Year = {2000}, Month = {June--July}, Number = {4}, Pages = {1196--1211}, Volume = {10}, Url = {http://epubs.siam.org/sam-bin/dbq/article/33096} } @Article{Birgin2000a, Title = {Nonmonotone Spectral Projected Gradient Methods on Convex Sets}, Author = {Ernesto G. Birgin and Jos{\'e} Mario Mart{\'\i}nez and Marcos Raydan}, Journal = {SIAM J Optimization}, Year = {2000}, Month = jun, Number = {4}, Pages = {1196--1211}, Volume = {10}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://epubs.siam.org/sam-bin/dbq/article/33096} } @Article{Birgin01spg, Title = {SPG: Software for convex-constrained optimization}, Author = {Ernesto G. Birgin and Mart'inez Marcos Raydan}, Journal = {ACM Transactions on Mathematical Software}, Year = {2001}, Pages = {340--349}, Volume = {27} } @Book{AIMMS06, Title = {{AIMMS optimization modeling}}, Author = {Bisschop, Johannes and Roelofs, Marcel}, Publisher = {Paragon Decision Technology}, Year = {2006}, ISBN = {1847539122} } @Article{Bjork:1967, Title = {Solving Least Squares Problems by {Gram--Schmidt} Orthogonalization}, Author = {A. Bjork}, Journal = {BIT}, Year = {1967}, Pages = {1--21}, Volume = {7} } @Article{Boer02atutorial, Title = {A Tutorial on the Cross-Entropy Method}, Author = {P. T. De Boer and D.P. Kroese and S. Mannor and R.Y. Rubinstein}, Journal = {Annals of Operations Research}, Year = {2002}, Volume = {134} } @Manual{p-R2admb, Title = {R2admb: ADMB to R interface functions}, Author = {Ben Bolker and Hans Skaug and Jeff Laake}, Note = {R package version 0.7.10}, Year = {2013}, Url = {http://CRAN.R-project.org/package=R2admb} } @Manual{p-bbmle, Title = {bbmle: Tools for general maximum likelihood estimation}, Author = {Ben Bolker and R Development Core Team}, Note = {R package version 1.0.13}, Year = {2013}, Url = {http://CRAN.R-project.org/package=bbmle} } @Manual{Borchers-pracma, Title = {pracma: Practical Numerical Math Functions}, Author = {Hans Werner Borchers}, Note = {R package version 2.0.7}, Year = {2017}, Url = {https://CRAN.R-project.org/package=pracma} } @Manual{p-adagio, Title = {adagio: Discrete and Global Optimization Routines}, Author = {Hans Werner Borchers}, Note = {R package version 0.5.9}, Year = {2013}, Url = {http://CRAN.R-project.org/package=adagio} } @Manual{p-nloptwrap, Title = {nloptwrap: Wrapper for Package nloptr}, Author = {Hans W Borchers}, Note = {R package version 0.5-1}, Year = {2013}, Url = {http://CRAN.R-project.org/package=nloptwrap} } @Manual{p-pracma, Title = {pracma: Practical Numerical Math Functions}, Author = {Hans W Borchers}, Note = {R package version 1.5.0}, Year = {2013}, Url = {http://CRAN.R-project.org/package=pracma} } @Manual{p-trustOptim, Title = {trustOptim: Trust region nonlinear optimization, efficient for sparse Hessians}, Author = {Michael Braun}, Note = {R package version 0.8.1}, Year = {2013}, Url = {http://CRAN.R-project.org/package=trustOptim} } @Article{Bremmerman70, Title = {A method for unconstrained global optimization}, Author = {Hans Bremmerman}, Journal = {Math. Biosci.}, Year = {1970}, Pages = {1--15}, Volume = {9} } @Book{Brent1973, Title = {Algorithms for Minimization without Derivatives}, Author = {Brent, R.}, Publisher = {Prentice-Hall}, Year = {1973}, Address = {Englewood Cliffs, NJ}, Owner = {john}, Timestamp = {2018.08.29} } @Article{Broyden70, Title = {The convergence of a class of double-rank minimization algorithms}, Author = {Broyden, C. G.}, Journal = {Journal of the Institute of Mathematics and Its Applications}, Year = {1970}, Pages = {76-–90}, Volume = {6} } @Article{Byrd95, Title = {A Limited Memory Algorithm for Bound Constrained Optimization}, Author = {Richard H. Byrd and Peihuang Lu and Jorge Nocedal and Ci You Zhu}, Journal = {SIAM Journal on Scientific Computing}, Year = {1995}, Month = sep, Number = {5}, Pages = {1190--1208}, Volume = {16}, Acknowledgement = {#ack-nhfb#}, Bibdate = {Fri Dec 4 16:17:35 MST 1998}, Coden = {SJOCE3}, ISSN = {1064-8275 (print), 1095-7197 (electronic)}, Mrclass = {90C30 (65K05)}, Mrnumber = {96e:90039}, Mrreviewer = {Henry Wolkowicz} } @Article{Davidon91, Title = {Variable Metric Method for Minimization}, Author = {C. Davidon, William}, Journal = {SIAM J. Optimization}, Year = {1991}, Month = {02}, Pages = {1 - 17}, Volume = {1}, Doi = {10.1137/0801001} } @Article{Cauchy1848, Title = {Méthode générale pour la resolution des systémes d’équations simultanées}, Author = {Cauchy, A.}, Journal = {C. R. Acad. Sci.}, Year = {1848}, Pages = {536-8}, Volume = {27}, Address = {Paris} } @Article{Cauchy1848a, Title = {M\'{e}thode g\'{e}n\'{e}rale pour la resolution des syst\'{e}mes d'\'{e}quations simultan\'{e}es}, Author = {Cauchy, A.}, Journal = {C. R. Acad. Sci.}, Year = {1848}, Pages = {536--8}, Volume = {27}, __markedentry = {[john:6]}, Address = {Paris}, Owner = {john}, Timestamp = {2018.11.16} } @Article{Chan:1983, Title = {Algorithms for Computing the Sample Variance: Analysis and Recommendations}, Author = {T. F. C. Chan and G. H. Golub and R. J. LeVeque}, Journal = {The American Statistician}, Year = {1983}, Number = {3}, Pages = {242--247}, Volume = {37} } @Manual{Colombi2012, Title = {smco: A simple Monte Carlo optimizer using adaptive coordinate sampling}, Author = {Prof. Juan David Velasquez H. Universidad Nacional de Colombia}, Note = {R package version 0.1}, Year = {2012}, Url = {http://CRAN.R-project.org/package=smco} } @Manual{smco12, Title = {smco: A simple Monte Carlo optimizer using adaptive coordinate sampling}, Author = {Prof. Juan David Velasquez H. Universidad Nacional de Colombia}, Note = {R package version 0.1}, Year = {2012}, Url = {http://CRAN.R-project.org/package=smco} } @Book{Conn09, Title = {Introduction to Derivative-Free Optimization}, Author = {A. R. Conn and K. Scheinberg and L.N.Vicente}, Publisher = {SIAM}, Year = {2009}, Address = {Philadelphia}, __markedentry = {[john:6]}, ISBN = {0898716683, 9780898716689}, Owner = {john}, Timestamp = {2018.11.16} } @Book{Conn2009, Title = {Introduction to Derivative-Free Optimization}, Author = {Conn, Andrew R. and Scheinberg, Katya and Vicente, Luis N.}, Publisher = {Society for Industrial and Applied Mathematics}, Year = {2009}, Address = {Philadelphia, PA, USA}, ISBN = {0898716683, 9780898716689} } @Article{Cruz2006, Title = {Spectral residual method without gradient information for solving large-scale nonlinear systems: Theory and experiments}, Author = {William La Cruz and Jos\'{e} Mario Mart\'{\i}nez and Marcos Raydan}, Journal = {Mathematics of Computation}, Year = {2006}, Pages = {1429--1448}, Volume = {75}, __markedentry = {[john:6]}, Institution = {unknown}, Owner = {john}, Timestamp = {2018.11.16} } @Article{Cruz06spectralresidual, Title = {Spectral residual method without gradient information for solving large-scale nonlinear systems: Theory and experiments}, Author = {William La Cruz and José Mario Martínez and Marcos Raydan}, Journal = {Mathematics of Computation}, Year = {2006}, Pages = {1429--1448}, Volume = {75} } @TechReport{Cruz04spectralresidual, Title = {Spectral residual method without gradient information for solving large-scale nonlinear systems: Theory and experiments}, Author = {William La Cruz and José Mario Martínez and Marcos Raydan}, Institution = {unknown}, Year = {2004} } @Article{Dai01, Title = {An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization}, Author = {Y. H. Dai and Y. Yuan}, Journal = {Annals of Operations Research}, Year = {2001}, Number = {1-4}, Pages = {33-47}, Volume = {103}, Bibsource = {DBLP, http://dblp.uni-trier.de}, Ee = {http://dx.doi.org/10.1023/A:1012930416777} } @Article{Dai99, Title = {A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property}, Author = {Y. H. Dai and Y. Yuan}, Journal = {SIAM Journal on Optimization}, Year = {1999}, Pages = {177--182}, Volume = {10} } @Article{Davidon59, Title = {Variable metric method for minimization}, Author = {Davidon, W.C.}, Year = {1959}, Month = {11}, Doi = {10.2172/4222000}, Place = {United States} } @InProceedings{Dekker69, Title = {Finding a zero by means of successive linear interpolation}, Author = {Dekker, T. J.}, Booktitle = {Constructive Aspects of the Fundamental Theorem of Algebra}, Year = {1969}, Editor = {Dejon, B. and Henrici, P.}, Publisher = {Wiley-Interscience}, Location = {London}, Owner = {john}, Timestamp = {2018.08.29} } @Manual{p-fitdistrplus, Title = {fitdistrplus: help to fit of a parametric distribution to non-censored or censored data }, Author = {Marie Laure Delignette-Muller and Regis Pouillot and Jean-Baptiste Denis and Christophe Dutang}, Note = {R package version 1.0-1}, Year = {2013} } @InProceedings{Deng06, Title = {{Adaptation of the UOBYQA Algorithm for Noisy Functions}}, Author = {Deng, Geng and Ferris, Michael C.}, Booktitle = {{WSC '06: Proceedings of the 38th Winter Simulation Conference}}, Year = {2006}, Address = {Monterey, California}, Pages = {312--319}, Publisher = {Winter Simulation Conference}, ISBN = {1-4244-0501-7} } @Book{DennisSchnabel96, Title = {Numerical Methods for Unconstrained Optimization and Nonlinear Equations}, Author = {Dennis, J. and Schnabel, R.}, Publisher = {Society for Industrial and Applied Mathematics}, Year = {1996}, Address = {Englewood Cliffs, NJ}, Doi = {10.1137/1.9781611971200}, Eprint = {http://epubs.siam.org/doi/pdf/10.1137/1.9781611971200}, Url = {http://epubs.siam.org/doi/abs/10.1137/1.9781611971200} } @Book{DenSchnab83, Title = {Numerical Methods for Unconstrained Optimization and Nonlinear Equations}, Author = {Dennis, J. E. and Schnabel, R. B.}, Publisher = {Prentice-Hall}, Year = {1983}, Address = {Englewood Cliffs, NJ} } @Book{dixon1972nonlinear, Title = {Nonlinear optimisation}, Author = {Dixon, L.C.W.}, Publisher = {The English Universities Press Ltd.}, Year = {1972}, Address = {London} } @Article{Domini02, Title = {On the Use of Generalized Additive Models in Time-Series Studies of Air Pollution and Health.}, Author = {F. Dominici}, Journal = {American Journal of Epidemiology}, Year = {2002}, Number = {3}, Pages = {193--202}, Volume = {156}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @Article{Dominici2002, Title = {On the Use of Generalized Additive Models in Time-Series Studies of Air Pollution and Health.}, Author = {F. Dominici}, Journal = {American Journal of Epidemiology}, Year = {2002}, Number = {3}, Pages = {193-202}, Volume = {156} } @Article{linbench03, Title = {The {LINPACK} Benchmark: Past, Present, and Future}, Author = {J. J. Dongarra and P. Luszczek and A. Petitet}, Journal = {Concurrency and Computation: Practice and Experience}, Year = {2003}, Month = {August}, Number = {9}, Pages = {803--820}, Volume = {15} } @Article{Dugg84, Title = {Regression Analysis of Nonlinear {Arrhenius} Plots: An Empirical Model and a Computer Program.}, Author = {Duggleby, R G}, Journal = {Computers in Biology and Medicine}, Year = {1984}, Number = {4}, Pages = {447--55}, Volume = {14}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @Article{Duggleby1984, Title = {Regression Analysis of Nonlinear {Arrhenius} Plots: An Empirical Model and a Computer Program.}, Author = {Duggleby, R G}, Journal = {Computers in Biology and Medicine}, Year = {1984}, Number = {4}, Pages = {447-55}, Volume = {14} } @Manual{RcppDE12, Title = {RcppDE: Global optimization by differential evolution in C++}, Author = {Dirk Eddelbuettel}, Note = {R package version 0.1.1. Extends DEoptim which itself is based on DE-Engine (by Rainer Storn)}, Year = {2012}, Url = {http://CRAN.R-project.org/package=RcppDE} } @Manual{minpacklm12, Title = {minpack.lm: R interface to the Levenberg-Marquardt nonlinear least-squares algorithm found in MINPACK, plus support for bounds}, Author = {Timur V. Elzhov and Katharine M. Mullen and Andrej-Nikolai Spiess and Ben Bolker}, Note = {R package version 1.1-6}, Organization = {R Project for Statistical Computing}, Year = {2012}, Url = {http://CRAN.R-project.org/package=minpack.lm} } @Article{Eugster2012, Title = {Domain-Based Benchmark Experiments: Exploratory and Inferential Analysis}, Author = {Manuel J. A. Eugster and Torsten Hothorn and Friedrich Leisch}, Journal = {Austrian Journal of Statistics}, Year = {2012}, Number = {1}, Pages = {5--26}, Volume = {41}, Url = {http://www.stat.tugraz.at/AJS/ausg121/121Leisch.pdf} } @Article{Eugster2012a, Title = {Domain-Based Benchmark Experiments: Exploratory and Inferential Analysis}, Author = {Manuel J. A. Eugster and Torsten Hothorn and Friedrich Leisch}, Journal = {Austrian Journal of Statistics}, Year = {2012}, Number = {1}, Pages = {5--26}, Volume = {41}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://www.stat.tugraz.at/AJS/ausg121/121Leisch.pdf} } @Article{Eugster2011, Title = {Exploratory Analysis of Benchmark Experiments -- An Interactive Approach}, Author = {Manuel J. A. Eugster and Friedrich Leisch}, Journal = {Computational Statistics}, Year = {2011}, Number = {4}, Pages = {699--710}, Volume = {26}, Doi = {10.1007/s00180-010-0227-z}, Preprinturl = {http://epub.ub.uni-muenchen.de/10604/} } @Article{Eugster2011a, Title = {Exploratory Analysis of Benchmark Experiments -- An Interactive Approach}, Author = {Manuel J. A. Eugster and Friedrich Leisch}, Journal = {Computational Statistics}, Year = {2011}, Number = {4}, Pages = {699--710}, Volume = {26}, __markedentry = {[john:6]}, Doi = {10.1007/s00180-010-0227-z}, Owner = {john}, Preprinturl = {http://epub.ub.uni-muenchen.de/10604/}, Timestamp = {2018.11.16} } @Article{Feldman91, Title = {Availability of f2c - a Fortran to C converter}, Author = {Feldman, S. I. and Gay, D. M. and Maimone, M. W. and Schryer, N. L.}, Journal = {SIGPLAN Fortran Forum}, Year = {1991}, Month = jul, Number = {2}, Pages = {14--15}, Volume = {10}, Address = {New York, NY, USA}, Issue_date = {July 1991}, Numpages = {2}, Publisher = {ACM} } @Article{Fletcher70, Title = {A New Approach to Variable Metric Algorithms}, Author = {R. Fletcher}, Journal = {Computer Journal}, Year = {1970}, Number = {3}, Pages = {317-322}, Volume = {13}, Bibsource = {DBLP, http://dblp.uni-trier.de} } @Article{FletReeves64, Title = {Function minimization by conjugate gradients}, Author = {Fletcher, R. and Reeves, C. M.}, Journal = {The Computer Journal}, Year = {1964}, Number = {2}, Pages = {149-154}, Volume = {7} } @Book{Forsythe77, Title = {Computer Methods for Mathematical Computations}, Author = {Forsythe, George E. and Malcolm, Michael A. and Moler, Cleve B.}, Publisher = {Prentice Hall Professional Technical Reference}, Year = {1977}, ISBN = {0131653326}, Owner = {john}, Timestamp = {2018.08.29} } @Misc{Fox2006, Title = {{TEACHER'S CORNER} Structural Equation Modeling With the sem Package in {R}}, Author = {John Fox}, Year = {2006}, __markedentry = {[john:6]}, Journal = {STRUCTURAL EQUATION MODELING}, Numeber = {3}, Owner = {john}, Pages = {465–-486}, Timestamp = {2018.11.16}, Volume = {13} } @Misc{Fox-sem, Title = {{TEACHER’S CORNER} Structural Equation Modeling With the sem Package in {R}}, Author = {John Fox}, Year = {2006}, Journal = {STRUCTURAL EQUATION MODELING}, Numeber = {3}, Pages = {465–-486}, Volume = {13} } @Manual{p-sem, Title = {sem: Structural Equation Models}, Author = {John Fox and Zhenghua Nie and Jarrett Byrnes}, Note = {R package version 3.1-301}, Year = {2013}, Url = {http://CRAN.R-project.org/package=sem} } @Misc{Fox1997, Title = {The {Port Mathematical Subroutine Library}, Version 3}, Author = {Phyllis Fox}, Year = {1997}, __markedentry = {[john:6]}, Address = {Murray Hill, NJ}, Owner = {john}, Publisher = {AT\&T Bell Laboratories}, Timestamp = {2018.11.16}, Url = {http://www.bell-labs.com/project/PORT/} } @Misc{PORTlib, Title = {The {Port Mathematical Subroutine Library}, Version 3}, Author = {Phyllis Fox}, Year = {1997}, Address = {Murray Hill, NJ}, Publisher = {AT\&T Bell Laboratories}, Url = {http://www.bell-labs.com/project/PORT/} } @Article{Fox1978PORT, Title = {The {PORT} Mathematical Subroutine Library}, Author = {P. A. Fox and A. D. Hall and N. L. Schryer}, Journal = {ACM Transactions on Mathematical Software (TOMS)}, Year = {1978}, Month = jun, Number = {2}, Pages = {104--126}, Volume = {4} } @Book{Gentle:1998, Title = {Numerical Linear Algebra for Applications in Statistics}, Author = {James E. Gentle}, Publisher = {Springer}, Year = {1998} } @Article{Bioc04, Title = {Bioconductor: Open Software Development for Computational Biology and Bioinformatics}, Author = {Robert C Gentleman and Vincent J. Carey and Douglas M. Bates and {others}}, Journal = {Genome Biology}, Year = {2004}, Pages = {R80}, Volume = {5}, Url = {http://genomebiology.com/2004/5/10/R80} } @Article{Geradin71, Title = {The computational efficiency of a new minimization algorithm for eigenvalue analysis}, Author = {Geradin, M.}, Journal = {J. Sound Vib.}, Year = {1971}, Pages = {319--331}, Volume = {19} } @Manual{p-trust, Title = {trust: Trust Region Optimization}, Author = {Charles J. Geyer}, Note = {R package version 0.1-4}, Year = {2013}, Url = {http://CRAN.R-project.org/package=trust} } @InProceedings{Geyer91, Title = {Markov chain Monte Carlo maximum likelihood}, Author = {C. J. Geyer}, Booktitle = {{Computing Science and Statistics, Proceedings of the 23rd Symposium on the Interface}}, Year = {1991}, Pages = {n.a.}, Publisher = {American Statistical Association, New York} } @Manual{Ghalanos2012, Title = {Rsolnp: General Non-linear Optimization Using Augmented Lagrange Multiplier Method}, Author = {Alexios Ghalanos and Stefan Theussl}, Note = {R package version 1.14}, Year = {2012}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://CRAN.R-project.org/package=Rsolnp} } @Manual{p-Rsolnp, Title = {Rsolnp: General Non-linear Optimization Using Augmented Lagrange Multiplier Method}, Author = {Alexios Ghalanos and Stefan Theussl}, Note = {R package version 1.14}, Year = {2012}, Url = {http://CRAN.R-project.org/package=Rsolnp} } @Manual{p-numDeriv, Title = {\pkg{numDeriv}: Accurate Numerical Derivatives}, Author = {Paul Gilbert}, Note = {R package version 2009.2-1}, Organization = {R Foundation for Statistical Computing}, Year = {2009}, Url = {http://www.bank-banque-canada.ca/pgilbert} } @Book{gillmurraywright, Title = {Practical Optimization}, Author = {P. E.~Gill and W.~Murray and M. H.~Wright}, Publisher = {Academic Press}, Year = {1981} } @Misc{Gil93, Title = {p2c}, Author = {Dave Gillespie}, Year = {1993}, __markedentry = {[john:6]}, Address = {Boston, MA}, Owner = {john}, Publisher = {Free Software Foundation}, Timestamp = {2018.11.16}, Url = {http://directory.fsf.org/project/p2c/} } @Misc{Gillespie1993, Title = {p2c}, Author = {Dave Gillespie}, Year = {1993}, Address = {Boston, MA}, Publisher = {Free Software Foundation}, Url = {http://directory.fsf.org/project/p2c/} } @Article{Glover89, Title = {Tabu Search - Part 1}, Author = {Fred Glover}, Journal = {ORSA Journal on Computing}, Year = {1989}, Number = {2}, Pages = {190--206}, Volume = {1} } @Article{Goldberg:1991, Title = {What Every Computer Scientist Should Know About Floating-Point Arithmetic}, Author = {David Goldberg}, Journal = {ACM Computing Surveys}, Year = {1991}, Pages = {5--48}, Colume = {23}, Lnumber = {1}, Url = {http://www.validlab.com/goldberg/paper.pdf} } @Article{Goldfarb70, Title = {A Family of Variable Metric Updates Derived by Variational Means}, Author = {Goldfarb, D.}, Journal = {Mathematics of Computation}, Year = {1970}, Number = {109}, Pages = {23--26}, Volume = {24} } @InProceedings{Golub1979, Title = {Extensions and Uses of the Variable Projection Algorithm for Solving Nonlinear Least Squares Problems}, Author = {Gene H. Golub and Randall J. LeVeque}, Booktitle = {Proceedings of the Army Numerical Analysis and Computing Conference}, Year = {1979}, Pages = {1--12}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @InProceedings{GolubLeveque79, Title = {Extensions and Uses of the Variable Projection Algorithm for Solving Nonlinear Least Squares Problems}, Author = {Gene H. Golub and Randall J. LeVeque}, Booktitle = {Proceedings of the Army Numerical Analysis and Computing Conference}, Year = {1979}, Pages = {1--12} } @Article{Golub1973, Title = {The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate}, Author = {G. H. Golub and V. Pereyra}, Journal = {SIAM Journal of Numerical Analysis}, Year = {1973}, Month = apr, Number = {2}, Pages = {413--432}, Volume = {10}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @Article{GolubPereyra73, Title = {The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate}, Author = {G. H. Golub and V. Pereyra}, Journal = {SIAM Journal of Numerical Analysis}, Year = {1973}, Month = apr, Number = {2}, Pages = {413--432}, Volume = {10} } @Article{Hager2006b, Title = {A Survey of Nonlinear Conjugate Gradient Methods}, Author = {William W. Hager and Hongchao Zhang}, Journal = {Pacific Journal of Optimization}, Year = {2006}, Pages = {35--58}, Volume = {2} } @Article{Hager2005, Title = {A new conjugate gradient method with guaranteed descent and an efficient line search}, Author = {William W. Hager and Hongchao Zhang}, Journal = {SIAM J. Optim.}, Year = {2005}, Number = {1}, Pages = {170--192}, Volume = {16} } @Article{Hager2006a, Title = {{Algorithm 851}: {CG\_DESCENT}, a conjugate gradient method with guaranteed descent}, Author = {William W. Hager and Hongchao Zhang}, Journal = {ACM Transactions on Mathematical Software}, Year = {2006}, Month = mar, Number = {1}, Pages = {113--137}, Volume = {32} } @Book{HansenWalster2004, Title = {Global Optimization Using Interval Analysis}, Author = {E. R. Hansen and G. W. Walster}, Publisher = {MIT Press}, Year = {2004}, Address = {Cambridge, Massachusetts} } @Manual{p-Rsymphony, Title = {Rsymphony: Symphony in R}, Author = {Reinhard Harter and Kurt Hornik and Stefan Theussl}, Note = {R package version 0.1-16}, Year = {2013}, Url = {http://CRAN.R-project.org/package=Rsymphony} } @Article{Hartley61, Title = {The Modified Gauss-Newton Method for the Fitting of Non-linear Regression Functions by Least Squares}, Author = {Hartley, H. O.}, Journal = {Technometrics}, Year = {1961}, Pages = {269-–280}, Volume = {3} } @Manual{p-nleqslv, Title = {nleqslv: Solve systems of non linear equations}, Author = {Berend Hasselman}, Note = {R package version 2.0}, Year = {2013}, Url = {http://CRAN.R-project.org/package=nleqslv} } @Manual{HHpackage:2015, Title = {HH: Statistical Analysis and Data Display: Heiberger and Holland.}, Author = {Richard M. Heiberger}, Note = {R package version 3.1-15}, Year = {2015}, Url = {http://CRAN.R-project.org/package=HH} } @Book{Heiberger:1989, Title = {Computation for the Analysis of Designed Experiments}, Author = {Richard M. Heiberger}, Publisher = {Wiley}, Year = {1989} } @Book{HH2, Title = {Statistical Analysis and Data Display: An Intermediate Course with Examples in R}, Author = {Richard M. Heiberger and Burt Holland}, Publisher = {Springer-Verlag, New York}, Year = {2015}, Edition = {Second}, Url = {http://www.springer.com/978-1-4939-2121-8} } @Book{RthroughExcelbook:2009pub, Title = {R through Excel: A Spreadsheet Interface for Statistics, Data Analysis, and Graphics}, Author = {Richard M. Heiberger and Erich Neuwirth}, Publisher = {Springer}, Year = {2009}, Url = {http://www.springer.com/978-1-4419-0051-7} } @Manual{p-linprog, Title = {{linprog: Linear Programming / Optimization}}, Author = {Arne Henningsen}, Note = {R package version 0.9-2}, Year = {2012}, Url = {http://CRAN.R-project.org/package=linprog} } @Article{p-systemfit, Title = {systemfit: A Package for Estimating Systems of Simultaneous Equations in R}, Author = {Arne Henningsen and Jeff D. Hamann}, Journal = {Journal of Statistical Software}, Year = {2007}, Number = {4}, Pages = {1--40}, Volume = {23}, Url = {http://www.jstatsoft.org/v23/i04/} } @Article{Hestenes1952, Title = {Methods of conjugate gradients for solving linear systems}, Author = {Hestenes, M. R. and Stiefel, E.}, Journal = {Journal of research of the National Bureau of Standards}, Year = {1952}, Pages = {409--436}, Volume = {49} } @Book{holland75, Title = {{Adaptation in Natural and Artificial Systems}}, Author = {Holland, J. H.}, Publisher = {University of Michigan Press}, Year = {1975}, Address = {Ann Arbor, MI, USA} } @Article{Hooke61, Title = {``Direct Search'' Solution of Numerical and Statistical Problems}, Author = {Robert Hooke and T. A. Jeeves}, Journal = {J. ACM}, Year = {1961}, Number = {2}, Pages = {212-229}, Volume = {8}, Bibsource = {DBLP, http://dblp.uni-trier.de}, Ee = {http://doi.acm.org/10.1145/321062.321069} } @Manual{p-ROI, Title = {ROI: R Optimization Infrastructure}, Author = {Kurt Hornik and David Meyer and Stefan Theussl}, Note = {R package version 0.0-7}, Year = {2011}, Url = {http://CRAN.R-project.org/package=ROI} } @Book{Huet1996, Title = {Statistical tools for nonlinear regression: a practical guide with {S-PLUS} examples}, Author = {S. (Sylvie) Huet and others}, Publisher = {Springer-Verlag}, Year = {1996}, Address = {Berlin \& New York}, Series = {Springer series in statistics}, __markedentry = {[john:6]}, Acknowledgement = {\#ack-nhfb\#}, Bibdate = {Tue Sep 22 08:25:09 MDT 1998}, ISBN = {0-387-94727-2}, Isbn-13 = {978-0-387-94727-3}, Keywords = {Regression analysis; Nonlinear theories; Parameter estimation.}, Lccn = {QA278.2.S73 1996}, Owner = {john}, Pages = {ix + 154}, Timestamp = {2018.11.16} } @Book{Huet1996a, Title = {Statistical tools for nonlinear regression: a practical guide with {S-PLUS} examples}, Author = {S. (Sylvie) Huet and others}, Publisher = {Springer-Verlag}, Year = {1996}, Address = {Berlin \& New York}, Series = {Springer series in statistics}, Acknowledgement = {#ack-nhfb#}, Bibdate = {Tue Sep 22 08:25:09 MDT 1998}, ISBN = {0-387-94727-2}, Isbn-13 = {978-0-387-94727-3}, Keywords = {Regression analysis; Nonlinear theories; Parameter estimation.}, Lccn = {QA278.2.S73 1996}, Pages = {ix + 154} } @Manual{soma11, Title = {soma: General-purpose optimisation with the Self-Organising Migrating Algorithm}, Author = {Jon Clayden; based on the work of Ivan Zelinka}, Note = {R package version 1.1.0}, Year = {2011}, Url = {http://CRAN.R-project.org/package=soma} } @Article{Jacobi1846, Title = {{\"U}ber ein leichtes Verfahren die in der Theorie der S{\"a}cul{\"a}rst{\"o}rungen vorkommenden Gleichungen numerisch aufzul{\"o}sen}, Author = {C. G. J. Jacobi}, Journal = {Crelle}, Year = {1846}, Pages = {51--94}, Volume = {30} } @Article{JoeNash, Title = {Numerical Optimization and Surface Estimation with Imprecise Function Evaluations}, Author = {Joe, Harry and Nash, John C.}, Journal = {Statistics and Computing}, Year = {2003}, Number = {3}, Pages = {277--286}, Volume = {13}, Address = {Hingham, MA, USA}, Doi = {http://dx.doi.org/10.1023/A:1024226918553}, ISSN = {0960-3174}, Publisher = {Kluwer Academic Publishers} } @Article{Joh03, Title = {Mineralization of Aged Atrazine, Terbuthylazine, 2,4-D, and Mecoprop in Soil and Aquifer Sediment}, Author = {H. Johannesen and J. Aamand}, Journal = {Environmental Toxicology and Chemistry}, Year = {2003}, Pages = {722--729}, Volume = {22}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @Article{Johannesen2003, Title = {Mineralization of Aged Atrazine, Terbuthylazine, 2,4-D, and Mecoprop in Soil and Aquifer Sediment}, Author = {H. Johannesen and J. Aamand}, Journal = {Environmental Toxicology and Chemistry}, Year = {2003}, Pages = {722--729}, Volume = {22} } @Misc{Johnson2008, Title = {{The NLopt nonlinear-optimization package}}, Author = {Steven G. Johnson}, Year = {2008}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://ab-initio.mit.edu/nlopt} } @Misc{NLoptSJ, Title = {{The NLopt nonlinear-optimization package}}, Author = {Steven G. Johnson}, Year = {2008}, Url = {http://ab-initio.mit.edu/nlopt} } @Article{Jones70, Title = {Spiral -- a new algorithm for non-linear parameter estimation using least squares}, Author = {Jones, A}, Journal = {Computer Journal}, Year = {1970}, Number = {3}, Pages = {301--308}, Volume = {13} } @Book{Kahaner89, Title = {Numerical Methods and Software}, Author = {Kahaner, David and Moler, Cleve and Nash, Stephen}, Publisher = {Prentice-Hall, Inc.}, Year = {1989}, Address = {Upper Saddle River, NJ, USA}, ISBN = {0-13-627258-4} } @Article{grofit10, Title = {{grofit}: Fitting Biological Growth Curves with {R}}, Author = {Matthias Kahm and Guido Hasenbrink and Hella Lichtenberg-Frat\'{e} and Jost Ludwig and Maik Kschischo}, Journal = {Journal of Statistical Software}, Year = {2010}, Number = {7}, Pages = {1--21}, Volume = {33}, Url = {http://www.jstatsoft.org/v33/i07/} } @Article{p-kernlab, Title = {kernlab -- An {S4} Package for Kernel Methods in {R}}, Author = {Alexandros Karatzoglou and Alex Smola and Kurt Hornik and Achim Zeileis}, Journal = {Journal of Statistical Software}, Year = {2004}, Number = {9}, Pages = {1--20}, Volume = {11}, Url = {http://www.jstatsoft.org/v11/i09/} } @MastersThesis{Karush39, Title = {Minima of Functions of Several Variables with Inequalities as Side Constraints}, Author = {W. Karush}, School = {Dept. of Mathematics, Univ. of Chicago}, Year = {1939}, Address = {Chicago, Illinois}, Note = {M.Sc. Dissertation} } @Book{kelley1987iterative, Title = {Iterative Methods for Optimization}, Author = {Kelley, C.T.}, Publisher = {Society for Industrial and Applied Mathematics}, Year = {1987}, Series = {Frontiers in Applied Mathematics}, ISBN = {9780898714333}, Lccn = {lc99011141}, Url = {http://books.google.ca/books?id=Bq6VcmzOe1IC} } @Article{Kelley97, Title = {Detection and Remediation of Stagnation in the Nelder-Mead Algorithm Using a Sufficient Decrease Condition}, Author = {C. T. Kelley}, Journal = {SIAM Journal on Optimization}, Year = {1997}, Pages = {43--55}, Volume = {10} } @InProceedings{Kennedy95PSO, Title = {Particle swarm optimization}, Author = {Kennedy, J. and Eberhart, R.}, Booktitle = {Neural Networks, 1995. Proceedings., IEEE International Conference on}, Year = {1995}, Pages = {1942-1948}, Volume = {4} } @Article{Kirkpatrick83, Title = {Optimization by Simulated Annealing}, Author = {S. Kirkpatrick and C. D. Gelatt Jr. and M. P. Vecchi}, Journal = {Science}, Year = {1983}, Month = {May}, Number = {4598}, Pages = {671--680}, Volume = {220} } @Manual{quantreg13, Title = {quantreg: Quantile Regression}, Author = {Roger Koenker}, Note = {R package version 5.05}, Year = {2013}, Url = {http://CRAN.R-project.org/package=quantreg} } @Book{Koenker2005, Title = {Quantile regression}, Author = {Koenker, R.}, Publisher = {Cambridge University Press}, Year = {2005}, Address = {Cambridge}, Doi = {10.1017/CBO9780511754098} } @Book{Koenker2005a, Title = {Quantile regression}, Author = {Koenker, R.}, Publisher = {Cambridge University Press}, Year = {2005}, Address = {Cambridge}, Doi = {10.1017/CBO9780511754098} } @Article{Kovalchik12, Title = {A general binomial regression model to estimate standardized risk differences from binary response data}, Author = {Kovalchik, Stephanie A. and Varadhan, Ravi and Fetterman, Barbara and Poitras, Nancy E. and Wacholder, Sholom and Katki, Hormuzd A.}, Journal = {Statistics in Medicine}, Year = {2013}, Number = {5}, Pages = {808--821}, Volume = {32}, Publisher = {John Wiley \& Sons, Ltd} } @InProceedings{KuhnTucker51, Title = {Nonlinear programming}, Author = {Kuhn, H. W. and Tucker, A. W.}, Booktitle = {{Proceedings of 2nd Berkeley Symposium}}, Year = {1951}, Address = {{Berkeley, USA}}, Pages = {481 -– 492}, Institution = {{University of California Press}} } @Article{Kulkarni2011PCA, Title = {A probability collectives approach with a feasibility-based rule for constrained optimization}, Author = {Kulkarni, Anand J. and Tai, K.}, Journal = {Appl. Comp. Intell. Soft Comput.}, Year = {2011}, Month = jan, Pages = {12:12--12:12}, Volume = {2011}, Acmid = {2208425}, Address = {New York, NY, United States}, Articleno = {12}, Doi = {10.1155/2011/980216}, ISSN = {1687-9724}, Issue_date = {January 2011}, Numpages = {1}, Publisher = {Hindawi Publishing Corp.}, Url = {http://dx.doi.org/10.1155/2011/980216} } @Manual{rbenchmark12, Title = {rbenchmark: Benchmarking routine for R}, Author = {Wacek Kusnierczyk}, Note = {R package version 1.0.0}, Year = {2012}, Url = {http://CRAN.R-project.org/package=rbenchmark} } @Book{Lanczos1956, Title = {Applied Analysis}, Author = {Cornelius Lanczos}, Publisher = {Prentice-Hall}, Year = {1956}, Address = {Englewood Cliffs, NJ}, Note = {Reprinted by Dover, New York, 1988.}, Pages = {xx + 539} } @Book{Lange:1999, Title = {Numerical Analysis for Statisticians}, Author = {Kenneth Lange}, Publisher = {Springer}, Year = {1999} } @Article{Levenberg44, Title = {A Method for the Solution of Certain Non-Linear Problems in Least Squares}, Author = {Kenneth Levenberg}, Journal = {Quarterly of Applied Mathematics}, Year = {1944}, Pages = {164-–168}, Volume = {2} } @Article{Lu94limitedmemory, Title = {A Limited-Memory Algorithm for Bound Constrained Optimization}, Author = {Peihuang Lu and Jorge Nocedal and Ciyou Zhu and Richard H. Byrd}, Journal = {SIAM Journal on Scientific Computing}, Year = {1994}, Pages = {1190--1208}, Volume = {16} } @Article{BUGS09, Title = {The BUGS project: Evolution, critique and future directions}, Author = {Lunn, David and Spiegelhalter, David and Thomas, Andrew and Best, Nicky}, Journal = {Statistics in Medicine}, Year = {2009}, Number = {25}, Pages = {3049--3067}, Volume = {28}, Publisher = {John Wiley \& Sons, Ltd.} } @Article{Lyness1971, Title = {Algorithm 413: ENTCAF and ENTCRE: evaluation of normalized Taylor coefficients of an analytic function}, Author = {Lyness, J. N. and Sande, G.}, Journal = {Communications of the ACM}, Year = {1971}, Month = oct, Number = {10}, Pages = {669--675}, Volume = {14}, Issue_date = {Oct. 1971} } @Article{MSB91, Title = {{The Parallel Genetic Algorithm as Function Optimizer}}, Author = {H. M\"{u}hlenbein and D. Schomisch and J. Born}, Journal = {{Parallel Computing}}, Year = {1991}, Number = {6-7}, Pages = {619-632}, Volume = {17} } @Manual{p-Rmpfr, Title = {Rmpfr: R MPFR - Multiple Precision Floating-Point Reliable}, Author = {Martin Maechler}, Note = {R package version 0.5-3}, Year = {2013}, Url = {http://CRAN.R-project.org/package=Rmpfr} } @Article{Marq63, Title = {{An Algorithm for Least-Squares Estimation of Nonlinear Parameters}}, Author = {Marquardt, Donald W.}, Journal = {SIAM Journal on Applied Mathematics}, Year = {1963}, Number = {2}, Pages = {431--441}, Volume = {11}, Publisher = {SIAM} } @Article{martins03, Title = {The complex-step derivative approximation}, Author = {Martins, Joaquim R. R. A. and Sturdza, Peter and Alonso, Juan J.}, Journal = {ACM Transactions on Mathematical Software}, Year = {2003}, Number = {3}, Pages = {245--262}, Volume = {29}, Timestamp = {2010-11-14T22:23:38.000+0100}, Url = {http://dx.doi.org/10.1145/838250.838251} } @InProceedings{McCarthy1988, Title = {The use of mathematical programming for designing dual frame surveys}, Author = {William F. McCarthy and David V. Bateman}, Booktitle = {Proceedings of the Survey Research Methods Section, 1988}, Year = {1988}, Pages = {652--653}, Publisher = {American Statistical Association, Alexandria, VA}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @InProceedings{McCarthy88, Title = {The use of mathematical programming for designing dual frame surveys}, Author = {William F. McCarthy and David V. Bateman}, Booktitle = {Proceedings of the Survey Research Methods Section, 1988}, Year = {1988}, Pages = {652--653}, Publisher = {American Statistical Association, Alexandria, VA} } @Book{mccullagh1989generalized, Title = {Generalized Linear Models, Second Edition}, Author = {McCullagh, P. and Nelder, J.A.}, Publisher = {Chapman and Hall}, Year = {1989}, Series = {Monographs on statistics and applied probability}, ISBN = {9780412317606}, Lccn = {99013896}, Url = {http://books.google.ca/books?id=h9kFH2\_FfBkC} } @InCollection{bdm04, Title = {Some Details of Nonlinear Estimation}, Author = {McCullough, B. D.}, Booktitle = {Numerical Issues in Statistical Computing for the Social Scientist}, Publisher = {John Wiley \& Sons, Inc.}, Year = {2004}, Chapter = {8}, Editor = {Altman, Micah and Gill, Jeff and McDonald, Michael P.}, Pages = {199--218} } @Article{mccren00, Title = {Some Numerical Aspects of Nonlinear Estimation}, Author = {B. D. McCullough and Charles G. Renfro}, Journal = {Journal of Economic and Social Measurement}, Year = {2000}, Number = {1}, Pages = {63--77}, Volume = {26} } @Article{rgenoud11, Title = {Genetic Optimization Using Derivatives: The {rgenoud} Package for {R}}, Author = {Walter R. {Mebane, Jr.} and Jasjeet S. Sekhon}, Journal = {Journal of Statistical Software}, Year = {2011}, Number = {11}, Pages = {1--26}, Volume = {42}, Url = {http://www.jstatsoft.org/v42/i11/} } @Manual{Melville17, Title = {mize: Unconstrained Numerical Optimization Algorithms}, Author = {James Melville}, Note = {R package version 0.1.1}, Year = {2017}, Url = {https://CRAN.R-project.org/package=mize} } @Manual{microbench13, Title = {microbenchmark: Sub microsecond accurate timing functions.}, Author = {Olaf Mersmann}, Note = {R package version 1.3-0}, Year = {2013}, Url = {http://CRAN.R-project.org/package=microbenchmark} } @Misc{Mittelm2008, Title = {Decision Tree for Optimization Software}, Author = {H. Mittelman}, Year = {2008}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://plato.asu.edu/guide.html} } @Misc{plato, Title = {Decision Tree for Optimization Software}, Author = {H. Mittelman}, Year = {2008}, Url = {http://plato.asu.edu/guide.html} } @Article{moler2003ndw, Title = {Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later}, Author = {Moler, Cleve and Loan, Charles Van}, Journal = {SIAM Review}, Year = {2003}, Number = {1}, Pages = {3-49}, Volume = {45}, Publisher = {SIAM} } @Manual{Molga2005, Title = {Test functions for optimization needs}, Author = {Molga, M. and Smutnicki, C.}, Year = {2005}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://www.zsd.ict.pwr.wroc.pl/files/docs/functions.pdf} } @Manual{Molga2005a, Title = { Test functions for optimization needs}, Author = { Molga, M. and Smutnicki, C. }, Year = {2005}, Url = {http://www.zsd.ict.pwr.wroc.pl/files/docs/functions.pdf} } @Book{Maple10, Title = {Maple~10 Programming Guide}, Author = {Michael B.~Monagan and Keith O.~Geddes and K.~Michael Heal and George Labahn and Stefan M.~Vorkoetter and James McCarron and Paul DeMarco}, Publisher = {Maplesoft}, Year = {2005}, Address = {Waterloo ON, Canada} } @TechReport{more80, Title = {{ANL-80-74}, {User Guide for MINPACK-1}}, Author = {Mor\'{e}, J. J. and Garbow, B. S. and Hillstrom, K. E.}, Institution = {Argonne National Laboratory}, Year = {1980}, Keywords = {algorithm, fittiookeng, optimization}, Url = {http://www.mcs.anl.gov/\~{}more/ANL8074a.pdf} } @Article{Morales2011, Title = {Remark on {Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization}}, Author = {Morales, Jos{\'e} Luis and Nocedal, Jorge}, Journal = {ACM Trans. Math. Softw.}, Year = {2011}, Month = {Dec}, Number = {1}, Pages = {7:1--7:4}, Volume = {38}, Address = {New York, NY, USA}, Articleno = {7}, Issue_date = {November 2011}, Numpages = {4}, Publisher = {ACM}, Url = {http://doi.acm.org/10.1145/2049662.2049669} } @Article{More92linesearch, Title = {Line Search Algorithms With Guaranteed Sufficient Decrease}, Author = {Jorge J. More and David J. Thuente and Preprint Mcs-p}, Journal = {ACM Trans. Math. Software}, Year = {1992}, Pages = {286--307}, Volume = {20} } @Article{DEoptimJSS11, Title = {{DEoptim}: An {R} Package for Global Optimization by Differential Evolution}, Author = {Katharine Mullen and David Ardia and David Gil and Donald Windover and James Cline}, Journal = {Journal of Statistical Software}, Year = {2011}, Number = {6}, Pages = {1--26}, Volume = {40}, Url = {http://www.jstatsoft.org/v40/i06/} } @Manual{minpack.lm, Title = {minpack.lm: R interface to the Levenberg-Marquardt nonlinear least-squares algorithm found in MINPACK, plus support for bounds}, Author = {Katharine M. Mullen and Timur V. Elzhov and Ben Bolker}, Note = {R package version 1.1-6}, Year = {2012} } @Manual{p-nnls, Title = {{nnls:} The Lawson-Hanson algorithm for non-negative least squares {(NNLS)}}, Author = {Katharine M. Mullen and Ivo H. M. van Stokkum}, Note = {R package version 1.4}, Year = {2012}, Url = {http://CRAN.R-project.org/package=nnls} } @Article{Mullen2007, Title = {Algorithms for separable nonlinear least squares with application to modelling time-resolved spectra}, Author = {Mullen, Katharine M. and Vengris, Mikas and Stokkum, Ivo H.}, Journal = {J. of Global Optimization}, Year = {2007}, Month = jun, Number = {2}, Pages = {201--213}, Volume = {38}, ISSN = {0925-5001}, Issue_date = {June 2007}, Numpages = {13} } @Manual{LMurphy12, Title = {likelihood: Methods for maximum likelihood estimation}, Author = {Lora Murphy}, Note = {R package version 1.5}, Year = {2012}, Url = {http://CRAN.R-project.org/package=likelihood} } @Manual{p-likelihood, Title = {likelihood: Methods for maximum likelihood estimation}, Author = {Lora Murphy}, Note = {R package version 1.5}, Year = {2012}, Url = {http://CRAN.R-project.org/package=likelihood} } @InProceedings{jn1987termin, Title = {Termination strategies for nonlinear parameter determination}, Author = {John C. Nash}, Booktitle = {Proceedings of the Australian Society for Operations Research Annual Conference}, Year = {1987}, Address = {Melbourne, Australia}, Editor = {Santosh Kumar}, Month = {October}, Pages = {322 -- 334} } @InProceedings{jnCTAC87, Title = {Function minimization on small or special computers}, Author = {John C. Nash}, Booktitle = {Computational Techniques and Applications: CTAC-87}, Year = {1988}, Address = {Amsterdam}, Editor = {Noye, J. and Fletcher, C.}, Pages = {509--516}, Publisher = {Elsevier Science Publishers} } @Book{jncnm79, Title = { Compact numerical methods for computers : linear algebra and function minimisation}, Author = {Nash, J. C.}, Publisher = { Hilger, Bristol : }, Year = {1979 }, Catalogue-url = { http://nla.gov.au/nla.cat-vn1060620 }, ISBN = { 0852743300 }, Language = { English }, Life-dates = { 1979 - }, Pages = { ix, 227 p. : }, Subjects = { Numerical analysis - Data processing. }, Type = { Book } } @Manual{rootoned18, Title = {rootoned: Roots of one-dimensional functions in R-only code}, Author = {John C Nash}, Note = {R package version 2018-7.30}, Year = {2018}, Owner = {john}, Timestamp = {2018.08.29}, Url = {https://r-forge.r-project.org/R/?group_id=395} } @Manual{Nash-snewton, Title = {snewton: Safeguarded Newton and General Gradient Approach for Function Minimization}, Author = {John C. Nash}, Note = {R package version 2017-4.20/r1516}, Year = {2017}, Url = {https://R-Forge.R-project.org/projects/optimizer/} } @Manual{optimr16, Title = {{optimr}: A Replacement and Extension of the 'optim' Function}, Author = {John C Nash}, Year = {2016}, Url = {http://CRAN.R-project.org/package=optimr} } @Article{NashBest14, Title = {On Best Practice Optimization Methods in {R}}, Author = {John C. Nash}, Journal = {Journal of Statistical Software}, Year = {2014}, Number = {2}, Pages = {1--14}, Volume = {60}, Url = {http://www.jstatsoft.org/v60/i02/} } @Book{nlpor14, Title = {{Nonlinear Parameter Optimization Using R Tools}}, Author = {Nash, John C.}, Publisher = {John Wiley \& Sons: Chichester}, Year = {2014}, Month = may, Note = {Companion website (see http://www.wiley.com//legacy/wileychi/nash/). JNfile: 14nlpor.pdf}, ISBN = {978-1-118-56928-3}, Language = {English}, Pages = {304 p.}, Type = {Book}, Url = {http://www.wiley.com//legacy/wileychi/nash/} } @Article{jnsign12, Title = {{Letter: Weigh less in tonnes?}}, Author = {Nash, John C.}, Journal = {Significance}, Year = {2012}, Month = {December}, Number = {6}, Pages = {45}, Volume = {9}, Institution = {{Royal Statistical Society}} } @Manual{p-Rcgmin, Title = {{\pkg{Rcgmin}}: Conjugate Gradient Minimization of Nonlinear Functions with Box Constraints}, Author = {John C. Nash}, Note = {R package version 2011-2.10}, Organization = {Nash Information Services Inc.}, Year = {2011} } @Manual{p-Rvmmin, Title = {{\pkg{Rvmmin}}: Variable Metric Nonlinear Function Minimization with Bounds Constraints}, Author = {John C. Nash}, Note = {R package version 2011-2.25}, Organization = {Nash Information Services Inc.}, Year = {2011} } @Article{nash2000toptenDantzig, Title = {The {Dantzig} Simplex Method for Linear Programming}, Author = {Nash, John C.}, Journal = {Computing in Science and Engineering}, Year = {2000}, Month = {January}, Number = {1}, Pages = {29-31}, Volume = {2}, Address = {Piscataway, NJ, USA}, Publisher = {IEEE Educational Activities Department} } @Article{nashderive95, Title = {Computer algebra systems: DERIVE}, Author = {Nash, John C.}, Journal = {The American Statistician}, Year = {1995}, Month = {February}, Note = {JNFile: ??}, Number = {1}, Pages = {93-99}, Volume = {49} } @Book{cnm79, Title = {Compact Numerical Methods for Computers: Linear Algebra and Function Minimisation}, Author = {John C. Nash}, Publisher = {Adam Hilger}, Year = {1979}, Address = {Bristol}, Note = {Second Edition, 1990, Bristol: Institute of Physics Publications.} } @Book{Nash1979b, Title = {Compact numerical methods for computers : linear algebra and function minimisation}, Author = {Nash, J. C.}, Publisher = {Hilger, Bristol :}, Year = {1979}, Address = {Bristol}, Note = {Second Edition, 1990, Bristol: Institute of Physics Publications.}, __markedentry = {[john:6]}, Catalogue-url = {http://nla.gov.au/nla.cat-vn1060620}, ISBN = {852743300}, Language = {English}, Life-dates = {1979 -}, Owner = {john}, Pages = {ix, 227 p. :}, Subjects = {Numerical analysis - Data processing.}, Timestamp = {2018.11.16}, Type = {Book} } @InProceedings{jnkp79tsl, Title = {Fitting two straight lines}, Author = {Nash, John C. and Kevin Price}, Booktitle = {Proceedings of Computer Science and Statistics: 12th annual Symposium on the Interface : May 10 \& 11, 1979, University of Waterloo, Waterloo, Ontario, Canada, Jane Gentleman (ed.)}, Year = {1979}, Month = {May}, Note = {JNFile: 78Fitting2StraightLines.pdf. Page numbers may be in error. Also Engineering and Statistical Research Institute, Agriculture Canada, Contribution No. I-99, 1978}, Pages = { 363--367 }, Publisher = {University of Waterloo, Ontario, Canada} } @Article{JNRV11, Title = {Unifying Optimization Algorithms to Aid Software System Users: {optimx} for {R}}, Author = {John C. Nash and Ravi Varadhan}, Journal = {Journal of Statistical Software}, Year = {2011}, Month = {8}, Number = {9}, Pages = {1--14}, Volume = {43}, Accepted = {2011-08-05}, Bibdate = {2011-08-05}, Coden = {JSSOBK}, Day = {24}, ISSN = {1548-7660}, Submitted = {2010-08-12}, Url = {http://www.jstatsoft.org/v43/i09/} } @Article{jnrv2011JSSOBKv43i09, Title = {Unifying Optimization Algorithms to Aid Software System Users: optimx for {R}}, Author = {John C. Nash and Ravi Varadhan}, Journal = {Journal of Statistical Software}, Year = {2011}, Month = {8}, Number = {9}, Pages = {1--14}, Volume = {43}, Accepted = {2011-08-05}, Bibdate = {2011-08-05}, Coden = {JSSOBK}, Day = {24}, ISSN = {1548-7660}, Submitted = {2010-08-12}, Url = {http://www.jstatsoft.org/v43/i09} } @Manual{p-optimx, Title = {\pkg{optimx}: A Replacement and Extension of the {optim()} function}, Author = {John C Nash and Ravi Varadhan}, Note = {R package version 2011-3.5}, Organization = {Nash Information Services Inc. and Johns Hopkins University}, Year = {2011} } @Book{jnmws87, Title = {{Nonlinear Parameter Estimation: An Integrated System in BASIC}}, Author = {John C. Nash and Mary Walker-Smith}, Publisher = {Marcel Dekker}, Year = {1987}, Address = {New York}, Note = {See \url{http://www.nashinfo.com/nlpe.htm} for an expanded downloadable version.} } @InBook{SGN83, Author = {Stephen G. Nash}, Chapter = {Truncated-Newton Methods for Large-Scale Minimization}, Editor = {H. E. Rauch}, Pages = {91--100}, Booktitle = {Applications of Nonlinear Programming to Optimization and Control} } @Article{NashTN2000, Title = {A survey of truncated-{Newton} methods}, Author = {Stephen G. Nash}, Journal = {Journal of Computational and Applied Mathematics}, Year = {2000}, Pages = {45--59}, Volume = {124} } @Article{NelderMead65, Title = {A Simplex Method for Function Minimization}, Author = {J. A. Nelder and R. Mead}, Journal = {Computer Journal}, Year = {1965}, Month = {January}, Number = {4}, Pages = {308--313}, Volume = {7} } @Misc{RExcel:2014, Title = {{RExcel}}, Author = {Erich Neuwirth}, Year = {2014}, Url = {http://rcom.univie.ac.at} } @TechReport{Nielsen2000, Title = {{UCMINF} - an Algorithm for Unconstrained, Nonlinear Optimization}, Author = {H. B. Nielsen}, Institution = {{ Department of Mathematical Modelling, Technical University of Denmark.}}, Year = {2000}, Month = {dec}, Note = {Report IMM-REP-2000-18}, Pages = {24}, Url = {http://www2.imm.dtu.dk/~hbn/publ/TR0019.ps} } @Manual{p-ucminf, Title = {ucminf: General-purpose unconstrained non-linear optimization}, Author = {Hans Bruun Nielsen and Stig Bousgaard Mortensen}, Note = {R package version 1.1-3}, Year = {2012}, Url = {http://CRAN.R-project.org/package=ucminf} } @Article{OlearyRust13, Title = {Variable projection for nonlinear least squares problems}, Author = {O'Leary, Dianne P. and Rust, Bert W.}, Journal = {Computational Optimization and Applications}, Year = {2013}, Number = {3}, Pages = {579-593}, Volume = {54}, ISSN = {0926-6003}, Publisher = {Springer US} } @Article{Pachon2009, Title = {{Barycentric-Remez algorithms for best polynomial approximation in the chebfun system}}, Author = {Pach\'{o}n, R. and Trefethen, L. N.}, Journal = {BIT Numerical Mathematics}, Year = {2009}, Number = {4}, Pages = {721--741}, Volume = {49}, Doi = {10.1007/s10543-009-0240-1}, Url = {http://www.springerlink.com/index/10.1007/s10543-009-0240-1} } @Misc{genstat, Title = {GenStat for Windows (12th Edition) Introduction}, Author = {R.W. Payne and D.A. Murray and S.A. Harding and D.B. Baird and D.M. Soutar}, Year = {2009}, __markedentry = {[john:6]}, Address = {Hemel Hempstead}, Owner = {john}, Publisher = {VSN International}, Timestamp = {2018.11.16} } @Misc{Payne2009, Title = {GenStat for Windows (12th Edition) Introduction}, Author = {R.W. Payne and D.A. Murray and S.A. Harding and D.B. Baird and D.M. Soutar}, Year = {2009}, Address = {Hemel Hempstead}, Publisher = {VSN International} } @Book{Pereyra10, Title = {Exponential Data Fitting and its Applications}, Author = {Pereyra, Victor and Scherer, Godela}, Publisher = {Bentham Science Publishers}, Year = {2010}, Address = {Sharjah} } @Manual{p-nlme, Title = {nlme: Linear and Nonlinear Mixed Effects Models}, Author = {Jose Pinheiro and Douglas Bates and Saikat DebRoy and Deepayan Sarkar and {R Core Team}}, Note = {R package version 3.1-111}, Year = {2013} } @Misc{Powell, Title = {Direct search algorithms for optimization calculations}, Author = {Powell,M. J. D.}, __markedentry = {[john:6]}, Doi = {10.1017/S0962492900002841}, Eprint = {http://journals.cambridge.org/article\_S0962492900002841}, ISSN = {1474-0508}, Journal = {Acta Numerica}, Numpages = {50}, Owner = {john}, Pages = {287--336}, Timestamp = {2018.11.16}, Url = {http://dx.doi.org/10.1017/S0962492900002841}, Volume = {7} } @Misc{Powell98, Title = {Direct search algorithms for optimization calculations}, Author = {Powell,M. J. D.}, Doi = {10.1017/S0962492900002841}, Eprint = {http://journals.cambridge.org/article\_S0962492900002841}, ISSN = {1474-0508}, Journal = {Acta Numerica}, Numpages = {50}, Pages = {287--336}, Url = {http://dx.doi.org/10.1017/S0962492900002841}, Volume = {7} } @Misc{Powell09, Title = {{The BOBYQA algorithm for bound constrained optimization without derivatives}}, Author = {Powell, M. J. D.}, Month = aug, Year = {2009}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://www.damtp.cam.ac.uk/user/na/NA\_papers/NA2009\_06.pdf} } @Misc{Powell2009, Title = {{The BOBYQA algorithm for bound constrained optimization without derivatives}}, Author = {Powell, M. J. D.}, Month = aug, Year = {2009}, Url = {http://www.damtp.cam.ac.uk/user/na/NA\_papers/NA2009\_06.pdf} } @Article{Powell07, Title = {{Developments of NEWUOA for Minimization Without Derivatives}}, Author = {Powell, M. J. D.}, Journal = {IMA Journal of Numerical Analysis}, Year = {2008}, Number = {4}, Pages = {649-664}, Volume = {28} } @Article{Powell06, Title = {The {NEWUOA} software for unconstrained optimization without derivatives}, Author = {M. J. D. Powell}, Journal = {Large Scale Nonlinear Optimization}, Year = {2006}, Pages = {255--297}, Date-added = {2008-11-26 00:45:42 +0100}, Date-modified = {2008-11-26 00:52:47 +0100} } @Article{Powell02, Title = {UOBYQA: unconstrained optimization by quadratic approximation}, Author = {M. J. D. Powell}, Journal = {Mathematical Programming}, Year = {2002}, Month = {May}, Number = {3}, Pages = {555--582}, Volume = {92}, ISSN = {0025-5610}, Keywords = {approximation}, Posted-at = {2010-08-31 04:22:15}, Priority = {2}, Publisher = {Springer Berlin / Heidelberg}, Url = {http://dx.doi.org/10.1007/s101070100290} } @Book{DEbook6, Title = {{D}ifferential {E}volution - A Practical Approach to Global Optimization}, Author = {Kenneth V. Price and Rainer M. Storn and Jouni A. Lampinen}, Publisher = {Springer-Verlag}, Year = {2006}, Address = {New York}, Month = jan, Note = {ISBN 540209506}, Series = {Natural Computing}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @Book{Price2006, Title = {{D}ifferential {E}volution - A Practical Approach to Global Optimization}, Author = {Kenneth V. Price and Rainer M. Storn and Jouni A. Lampinen}, Publisher = {Springer-Verlag}, Year = {2006}, Month = {January}, Note = {ISBN 540209506}, Series = {Natural Computing} } @Book{Price05, Title = {Differential Evolution - A Practical Approach to Global Optimization}, Author = {K. V. Price and R. M. Storn and J. A. Lampinen}, Publisher = {Springer}, Year = {2005}, Address = {New York} } @Article{Price77, Title = {A Controlled Random Search Procedure for Global Optimisation}, Author = {Wyn L. Price}, Journal = {Computer Journal}, Year = {1977}, Number = {4}, Pages = {367-370}, Volume = {20}, Bibsource = {DBLP, http://dblp.uni-trier.de} } @Misc{FAQ731, Author = {{R Core Team}}, Year = {2017}, Address = {Vienna, Austria}, Organization = {R Foundation for Statistical Computing}, Url = {https://cran.r-project.org/doc/FAQ/R-FAQ.html\#Why-doesn\_0027t-R-think-these-numbers-are-equal\_003f} } @Manual{citeR, Title = {R: A Language and Environment for Statistical Computing}, Address = {Vienna, Austria}, Author = {{R Core Team}}, Organization = {R Foundation for Statistical Computing}, Year = {2013}, Url = {http://www.R-project.org/} } @Manual{Rcite, Title = {{R}: A Language and Environment for Statistical Computing}, Address = {Vienna, Austria}, Author = {{R Development Core Team}}, Note = {{ISBN} 3-900051-07-0}, Organization = {R Foundation for Statistical Computing}, Year = {2008}, Url = {http://www.R-project.org} } @Article{FletPow63, Title = {A rapidly convergent descent method for minimization}, Author = { R. Fletcher, R. and Powell, M. J. D.}, Journal = {Computer Journal}, Year = {1963}, Pages = {163--168}, Volume = {6}, Doi = {10.2172/4222000}, Place = {United States} } @InProceedings{Rajnarayan2006, Title = {Optimization Under Uncertainty Using Probability Collectives}, Author = {Rajnarayan, D. and Wolpert, D.H. and Kroo, I.}, Booktitle = {{Proceedings of 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Portsmouth, VA}}, Year = {2006}, Note = {{AIAA-2006-7033}} } @Book{Ratkowsky83, Title = {Nonlinear Regression Modeling: A Unified Practical Approach}, Author = {David A. Ratkowsky}, Publisher = {Marcel Dekker Inc.}, Year = {1983}, Address = {New York and Basel} } @Article{Swendsen86, Title = {Replica Monte Carlo simulation of spin glasses}, Author = {Swendsen RH and Wang JS}, Journal = { Physical Review Letters}, Year = {1986}, Pages = {2607--2609}, Volume = {57} } @Book{RitzStreibig2008, Title = {Nonlinear Regression with R}, Author = {C. Ritz and J. C. Streibig}, Publisher = {Springer}, Year = {2008}, Address = {New York} } @Article{Rosenbrock60, Title = {An automatic method for finding the greatest or least value of a function}, Author = {Rosenbrock, H. H.}, Journal = {The Computer Journal}, Year = {1960}, Pages = {175–184}, Volume = {3} } @Book{Ross90, Title = {Nonlinear estimation.}, Author = {Ross, Gavin J.S.}, Publisher = {New York etc.: Springer-Verlag}, Year = {1990}, ISBN = {3-540-97278-1}, Pages = {viii, 189 p.} } @Article{Schnabel1985, Title = {A modular system of algorithms for unconstrained minimization}, Author = {Schnabel, Robert B. and Koonatz, John E. and Weiss, Barry E.}, Journal = {ACM Trans. Math. Softw.}, Year = {1985}, Month = dec, Note = {Original report CU-CS-480-82 from 1982, revised. The UNCMIN Manual.}, Number = {4}, Pages = {419--440}, Volume = {11}, __markedentry = {[john:6]}, Acmid = {6192}, Address = {New York, NY, USA}, Doi = {10.1145/6187.6192}, ISSN = {0098-3500}, Issue_date = {Dec. 1985}, Numpages = {22}, Owner = {john}, Publisher = {ACM}, Timestamp = {2018.11.16}, Url = {http://doi.acm.org/10.1145/6187.6192} } @Article{Schnabel85, Title = {A Modular System of Algorithms for Unconstrained Minimization}, Author = {Robert B. Schnabel and John E. Koontz and Barry E. Weiss}, Journal = {ACM Transactions on Mathematical Software}, Year = {1985}, Month = dec, Number = {4}, Pages = {419--440}, Volume = {11} } @Manual{p-PBSadmb, Title = {PBSadmb: ADMB for R Using Scripts or GUI}, Author = {Jon T. Schnute and Rowan Haigh and Alex Couture-Beil}, Note = {R package version 0.66.87}, Year = {2013}, Url = {http://CRAN.R-project.org/package=PBSadmb} } @Article{p-GA, Title = {{GA}: A Package for Genetic Algorithms in {R}}, Author = {Luca Scrucca}, Journal = {Journal of Statistical Software}, Year = {2013}, Number = {4}, Pages = {1--37}, Volume = {53}, Url = {http://www.jstatsoft.org/v53/i04/} } @Book{Seber89Wild, Title = {{Nonlinear regression}}, Author = {Seber, G. A. F. and Wild, C. J.}, Publisher = {Wiley}, Year = {1989}, Address = {New York} } @Article{rgenoudpolitical, Title = {Genetic Optimization Using Derivatives: Theory and Application to Nonlinear Models}, Author = {Jasjeet S. Sekhon and Walter R. {Mebane, Jr.}}, Journal = {Political Analysis}, Year = {1998}, Pages = {189--213}, Volume = {7} } @Article{Shanno70A, Title = {Conditioning of quasi-Newton methods for function minimization}, Author = {Shanno, David F.}, Journal = {Mathematics of Computation}, Year = {1970}, Number = {111}, Pages = {647--656}, Volume = {24} } @Article{Shanno70B, Title = {Optimal conditioning of quasi-Newton methods}, Author = {Shanno, David F. and Kettler, Paul C.}, Journal = {Mathematics of Computation}, Year = {1970}, Month = jul, Number = {111}, Pages = {657--664}, Volume = {24} } @Article{Shrager85RRR, Title = {A Rapid Robust Rootfinder}, Author = {Richard I. Shrager}, Journal = {Mathematics of Computation}, Year = {1985}, Month = jan, Number = {169}, Pages = {151--165}, Volume = {44} } @Manual{Soetaert-rootSolve, Title = {rootSolve: Nonlinear root finding, equilibrium and steady-state analysis of ordinary differential equations }, Author = {Karline Soetaert}, Note = {R package 1.6}, Year = {2009}, Owner = {john}, Timestamp = {2018.08.29} } @Book{Soetaert2009, Title = {A Practical Guide to Ecological Modelling. Using R as a Simulation Platform}, Author = {Karline Soetaert and Peter M.J. Herman}, Publisher = {Springer}, Year = {2009}, Note = {ISBN 978-1-4020-8623-6}, Owner = {john}, Pages = {372}, Timestamp = {2018.08.29} } @Article{Squire98, Title = {Using Complex Variables to Estimate Derivatives of Real Functions}, Author = {Squire, William and Trapp, George}, Journal = {SIAM Review}, Year = {1998}, Month = mar, Number = {1}, Pages = {110--112}, Volume = {40}, Issue_date = {March 1998} } @Misc{stata, Title = {Stata Statistical Software: Release 11}, Author = {StataCorp}, Year = {2009}, __markedentry = {[john:6]}, Address = {College Station, TX}, Owner = {john}, Publisher = {StataCorp LP}, Timestamp = {2018.11.16} } @Misc{StataCorp2009, Title = {Stata Statistical Software: Release 11}, Author = {StataCorp}, Year = {2009}, Address = {College Station, TX}, Publisher = {StataCorp LP} } @Article{TZ89, Title = {{Global Optimization}}, Author = {A. T\"{o}rn and A. Zilinskas}, Journal = {{Lecture Notes in Computer Science}}, Year = {1989}, Volume = {350}, Address = {New York, NY, USA}, Publisher = {{Springer-Verlag}} } @Manual{p-gaoptim, Title = {gaoptim: Genetic Algorithm optimization for real-based and permutation-based problems}, Author = {Fernando Tenorio}, Note = {R package version 1.1}, Year = {2013}, Url = {http://CRAN.R-project.org/package=gaoptim} } @Article{Thacher1960, Title = {Remark on Algorithm 2: Rootfinder}, Author = {Thacher,Jr., Henry D.}, Journal = {Commun. ACM}, Year = {1960}, Month = aug, Number = {8}, Pages = {475--}, Volume = {3}, Acmid = {367385}, Address = {New York, NY, USA}, Doi = {10.1145/367368.367385}, ISSN = {0001-0782}, Issue_date = {Aug. 1960}, Owner = {john}, Publisher = {ACM}, Timestamp = {2018.08.29}, Url = {http://doi.acm.org/10.1145/367368.367385} } @Manual{p-Rglpk, Title = {Rglpk: R/GNU Linear Programming Kit Interface}, Author = {Stefan Theussl and Kurt Hornik}, Note = {R package version 0.5-1}, Year = {2013}, Url = {http://CRAN.R-project.org/package=Rglpk} } @Article{TienEarn10, Title = {Multiple transmission pathways and disease dynamics in a waterborne pathogen model}, Author = {J H Tien and D J D Earn}, Journal = {Bulletin of Mathematical Biology}, Year = {2010}, Number = {6}, Pages = {1506--1533}, Volume = {72} } @Article{Toomet2008, Title = {Sample Selection Models in R: Package sampleSelection}, Author = {Ott Toomet and Arne Henningsen}, Journal = {Journal of Statistical Software}, Year = {2008}, Month = {7}, Number = {7}, Pages = {1--23}, Volume = {27}, Day = {29} } @Article{Torczon95patternsearch, Title = {Pattern Search Methods for Nonlinear Optimization}, Author = {Virginia Torczon}, Journal = {SIAG/OPT Views and News}, Year = {1995}, Pages = {7--11}, Volume = {6} } @Manual{p-gnm, Title = {Generalized nonlinear models in R: An overview of the gnm package}, Author = {Heather Turner and David Firth}, Note = {R package version 1.0-6}, Year = {2012}, Url = {http://CRAN.R-project.org/package=gnm} } @Manual{p-alabama, Title = {alabama: Constrained nonlinear optimization}, Author = {Ravi Varadhan}, Note = {R package version 2011.9-1}, Year = {2012}, Url = {http://CRAN.R-project.org/package=alabama} } @Manual{p-dfoptim, Title = {dfoptim: Derivative-free Optimization}, Author = {Ravi Varadhan and Hans W. Borchers}, Note = {R package version 2011.12-9}, Organization = {Johns Hopkins University and ABB Corporate Research.}, Year = {2011}, Url = {http://CRAN.R-project.org/package=dfoptim} } @Article{p-BB, Title = {{\pkg{BB}}: An {\R} Package for Solving a Large System of Nonlinear Equations and for Optimizing a High-Dimensional Nonlinear Objective Function}, Author = {Ravi Varadhan and Paul Gilbert}, Journal = {Journal of Statistical Software}, Year = {2009}, Number = {4}, Pages = {1--26}, Volume = {32}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://www.jstatsoft.org/v32/i04/} } @Article{Vara09, Title = {{BB}: An {\R} Package for Solving a Large System of Nonlinear Equations and for Optimizing a High-Dimensional Nonlinear Objective Function}, Author = {Ravi Varadhan and Paul Gilbert}, Journal = {Journal of Statistical Software}, Year = {2009}, Number = {4}, Pages = {1--26}, Volume = {32}, Url = {http://www.jstatsoft.org/v32/i04/} } @Article{Varadhan2009, Title = {{\pkg{BB}}: An {\R} Package for Solving a Large System of Nonlinear Equations and for Optimizing a High-Dimensional Nonlinear Objective Function}, Author = {Ravi Varadhan and Paul Gilbert}, Journal = {Journal of Statistical Software}, Year = {2009}, Number = {4}, Pages = {1--26}, Volume = {32}, Url = {http://www.jstatsoft.org/v32/i04/} } @Book{SAS, Title = {SAS}, Author = {various}, Publisher = {SAS Institute Inc.}, Address = {Cary, NC} } @Book{various, Title = {SAS}, Author = {various}, Publisher = {SAS Institute Inc.}, Address = {Cary, NC} } @Book{VenRip1994, Title = {Modern Applied Statistics with {S-PLUS}}, Author = {William N. Venables and Brian D. Ripley}, Publisher = {Springer-Verlag}, Year = {1994}, Address = {Berlin, Germany~/ London, UK}, Pages = {xii + 462} } @Article{WatsonHompack97, Title = {Algorithm 777: HOMPACK90: a suite of Fortran 90 codes for globally convergent homotopy algorithms}, Author = {Watson, Layne T. and Sosonkina, Maria and Melville, Robert C. and Morgan, Alexander P. and Walker, Homer F.}, Journal = {ACM Trans. Math. Softw.}, Year = {1997}, Month = dec, Number = {4}, Pages = {514--549}, Volume = {23}, Address = {New York, NY, USA}, ISSN = {0098-3500}, Issue_date = {Dec. 1997}, Numpages = {36}, Publisher = {ACM} } @Article{Wegstein1960, Title = {Certification: Rootfinder}, Author = {Wegstein, J. and Thacher,Jr., Henry C.}, Journal = {Commun. ACM}, Year = {1960}, Month = jun, Number = {6}, Pages = {354--}, Volume = {3}, Acmid = {367337}, Address = {New York, NY, USA}, Doi = {10.1145/367297.367337}, ISSN = {0001-0782}, Issue_date = {June 1960}, Owner = {john}, Publisher = {ACM}, Timestamp = {2018.08.29}, Url = {http://doi.acm.org/10.1145/367297.367337} } @Article{Wenzel99, Title = {Stochastic Tunneling Approach for Global Minimization of Complex Potential Energy Landscapes}, Author = {Wenzel, W. and Hamacher, K.}, Journal = {Phys. Rev. Lett.}, Year = {1999}, Month = {Apr}, Pages = {3003--3007}, Volume = {82}, Doi = {10.1103/PhysRevLett.82.3003}, Issue = {15}, Publisher = {American Physical Society}, Url = {http://link.aps.org/doi/10.1103/PhysRevLett.82.3003} } @Article{Wilkinson1961, Title = {Remarks on Algorithms 2 and 3}, Author = {Wilkinson, J. H.}, Journal = {Commun. ACM}, Year = {1961}, Month = mar, Number = {3}, Pages = {153--}, Volume = {4}, Acmid = {366273}, Address = {New York, NY, USA}, Doi = {10.1145/366199.366273}, ISSN = {0001-0782}, Issue_date = {March 1961}, Owner = {john}, Publisher = {ACM}, Timestamp = {2018.08.29} } @Book{WilkinsonReinsch1971, Title = {Handbook for Automatic Computation: Linear Algebra (Grundlehren Der Mathematischen Wissenschaften, Vol 186)}, Author = {Wilkinson, J. H. and Reinsch, C. and Bauer, F. L.}, Publisher = {SpringerVerlag}, Year = {1986}, ISBN = {387054146} } @Book{Mathematica4, Title = {The Mathematica Book}, Author = {Stephen Wolfram}, Publisher = {Wolfram Research Inc.}, Year = {2000} } @Article{GenSA13, Title = { Generalized Simulated Annealing for Global Optimization: the GenSA Package for R }, Author = { Yang Xiang and Sylvain Gubian and Brian Suomela and Julia Hoeng }, Journal = { The R Journal }, Year = { 2013 }, Month = jun, Number = { 1 }, Pages = { 13--29 }, Volume = { 5 }, Url = { http://journal.r-project.org/archive/2013-1/xiang-gubian-suomela-etal.pdf } } @Article{Xiang2013, Title = {Generalized Simulated Annealing for Global Optimization: the GenSA Package for R}, Author = {Yang Xiang and Sylvain Gubian and Brian Suomela and Julia Hoeng}, Journal = {The R Journal}, Year = {2013}, Month = jun, Number = {1}, Pages = {13--29}, Volume = {5}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16}, Url = {http://journal.r-project.org/archive/2013-1/xiang-gubian-suomela-etal.pdf} } @InCollection{Xie-reproducible, Title = {knitr: A Comprehensive Tool for Reproducible Research in {R}}, Author = {Yihui Xie}, Booktitle = {Implementing Reproducible Computational Research}, Publisher = {Chapman and Hall/CRC}, Year = {2014}, Editor = {Victoria Stodden and Friedrich Leisch and Roger D. Peng}, Note = {ISBN 978-1466561595}, Url = {http://www.crcpress.com/product/isbn/9781466561595} } @Manual{Xie-knitr, Title = {knitr: A General-Purpose Package for Dynamic Report Generation in R}, Author = {Yihui Xie}, Note = {R package version 1.20}, Year = {2018}, Url = {https://yihui.name/knitr/} } @Book{Xie-Dynamic, Title = {Dynamic Documents with {R} and knitr}, Author = {Yihui Xie}, Publisher = {Chapman and Hall/CRC}, Year = {2015}, Address = {Boca Raton, Florida}, Edition = {2nd}, Note = {ISBN 978-1498716963}, Url = {https://yihui.name/knitr/} } @PhdThesis{Ye1987, Title = {Interior Algorithms for Linear, Quadratic, and Linearly Constrained Non-Linear Programming}, Author = {Yinyu Ye}, School = {Department of EES, Stanford University}, Year = {1987}, Address = {Palo Alto, California}, Note = {Ph. D. Dissertation}, __markedentry = {[john:6]}, Owner = {john}, Timestamp = {2018.11.16} } @PhdThesis{Ye1987a, Title = {Interior Algorithms for Linear, Quadratic, and Linearly Constrained Non-Linear Programming}, Author = {Yinyu Ye}, School = {Department of EES, Stanford University}, Year = {1987}, Address = {Palo Alto, California}, Note = {Ph. D. Dissertation} } @Manual{p-nloptr, Title = {nloptr: \R\ interface to {NLopt}}, Author = {Jelmer Ypma}, Note = {R package version 0.9.3}, Year = {2013}, Url = {http://CRAN.R-project.org/package=nloptr} } @Article{Zhu1997LBFGS, Title = {Algorithm 778: {L-BFGS-B}: Fortran subroutines for large-scale bound-constrained optimization}, Author = {Zhu, Ciyou and Byrd, Richard H. and Lu, Peihuang and Nocedal, Jorge}, Journal = {ACM Trans. Math. Softw.}, Year = {1997}, Month = dec, Number = {4}, Pages = {550--560}, Volume = {23}, Acmid = {279236}, Address = {New York, NY, USA}, Doi = {10.1145/279232.279236}, ISSN = {0098-3500}, Issue_date = {Dec. 1997}, Keywords = {large-scale optimization, limited-memory method, nonlinear optimization, variable metric method}, Numpages = {11}, Publisher = {ACM}, Url = {http://doi.acm.org/10.1145/279232.279236} } @Book{Rauch83, Title = {Applications of Nonlinear Programming to Optimization and Control}, Editor = {H. E. Rauch}, Publisher = {Pergamon Press}, Year = {1983}, Address = {Oxford}, Booktitle = {Applications of Nonlinear Programming to Optimization and Control} } lbfgsb3c/vignettes/jrosen.f0000644000176200001440000000047313513624255015430 0ustar liggesusers subroutine rosen(n, x, fval) double precision x(n), fval, dx integer n, i fval = 0.0D0 do 10 i=1,(n-1) dx = x(i + 1) - x(i) * x(i) fval = fval + 100.0 * dx * dx dx = 1.0 - x(i) fval = fval + dx * dx 10 continue return end lbfgsb3c/NEWS0000644000176200001440000000074713511470416012450 0ustar liggesusersTo do Add test using a plain C function for optimization. lbfgsb3c is supposed to handle this. -------------------------------------------------------------- 2019-03-19 o Packages lbfgsb3 and lbfgsb3c merged into latter. Vignette added. o Suppressed printout when trace>2 and starting (f not defined) 2015-01-20 o Fixup line longer than 72 chars in lbfgsb.f. Undeclared integer itask in errclb subroutine. Thanks to Berend Hasselman. New package lbfgsb3 2014.7.31 lbfgsb3c/R/0000755000176200001440000000000013625774732012160 5ustar liggesuserslbfgsb3c/R/lbfgsb3.R0000644000176200001440000002132113625545750013620 0ustar liggesusers##' @importFrom Rcpp evalCpp ##' @importFrom methods is ##' @useDynLib lbfgsb3c, .registration=TRUE "lbfgsb3c" ##' Interfacing wrapper for the Nocedal - Morales LBFGSB3 (Fortran) limited memory BFGS solver. ##' ##' @param par A parameter vector which gives the initial guesses to ##' the parameters that will minimize \code{fn}. This can be ##' named, for example, we could use par=c(b1=1, b2=2.345, ##' b3=0.123) ##' @param fn A function that evaluates the objective function to be ##' minimized. This can be a R function or a Rcpp function ##' pointer. ##' @param gr If present, a function that evaluates the gradient ##' vector for the objective function at the given parameters ##' computing the elements of the sum of squares function at the ##' set of parameters \code{start}. This can be a R function or a ##' Rcpp function pointer. ##' @param lower Lower bounds on the parameters. If a single number, ##' this will be applied to all parameters. Default -Inf. ##' @param upper Upper bounds on the parameters. If a single number, ##' this will be applied to all parameters. Default Inf. ##' @param control An optional list of control settings. See below in ##' details. ##' @param ... Any data needed for computation of the objective ##' function and gradient. ##' @param rho An Environment to use for function evaluation. If ##' present the arguments in ... are ignored. Otherwise the ##' ... are converted to an environment for evaluation. ##' @details ##' ##' See the notes below for a general appreciation of this package. ##' ##' The control list can contain: ##' \itemize{ ##' \item{trace} If positive, tracing information on the progress of the optimization is produced. Higher values may produce more tracing information: for method "L-BFGS-B" there are six levels of tracing. (To understand exactly what these do see the source code: higher levels give more detail.) ##' \item{factr} controls the convergence of the "L-BFGS-B" method. Convergence occurs when the reduction in the objective is within this factor of the machine tolerance. Default is 1e7, that is a tolerance of about 1e-8. ##' \item{pgtol} helps control the convergence of the "L-BFGS-B" method. It is a tolerance on the projected gradient in the current search direction. This defaults to zero, when the check is suppressed. ##' \item{abstol} helps control the convergence of the "L-BFGS-B" method. It is an absolute tolerance difference in x values. This defaults to zero, when the check is suppressed. ##' \item{reltol} helps control the convergence of the "L-BFGS-B" method. It is an relative tolerance difference in x values. This defaults to zero, when the check is suppressed. ##' \item{lmm} is an integer giving the number of BFGS updates retained in the "L-BFGS-B" method, It defaults to 5. ##' \item{maxit} maximum number of iterations. ##' \item{iprint} Provided only for compatibility with older codes. This control is no longer active.) ##' \item{info} a boolean to indicate if more optimization information is captured and output in a $info list ##' } ##' ##' @return ##' A list of the following items ##' \itemize{ ##' \item{par} The best set of parameters found. ##' \item{value} The value of fn corresponding to par. ##' \item{counts} A two-element integer vector giving the number of calls to fn and gr respectively. This excludes any calls to fn to compute a finite-difference approximation to the gradient. ##' \item{convergence} An integer code. 0 indicates successful completion ##' } ##' @seealso Packages \code{\link{optim}} and \code{optimx}. ##' @keywords nonlinear parameter optimization ##' @author Matthew Fidler (move to C and add more options for adjustments), ##' John C Nash (of the wrapper and edits to Fortran code to allow R output) ##' Ciyou Zhu, Richard Byrd, Jorge Nocedal, Jose Luis Morales (original Fortran packages) ##' ##' @references ##' Morales, J. L.; Nocedal, J. (2011). "Remark on 'algorithm 778: L-BFGS-B: ##' Fortran subroutines for large-scale bound constrained optimization' ". ##' ACM Transactions on Mathematical Software 38: 1. ##' ##' Byrd, R. H.; Lu, P.; Nocedal, J.; Zhu, C. (1995). "A Limited Memory Algorithm ##' for Bound Constrained Optimization". SIAM J. Sci. Comput. 16 (5): 1190-1208. ##' ##' Zhu, C.; Byrd, Richard H.; Lu, Peihuang; Nocedal, Jorge (1997). "L-BFGS-B: ##' Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained ##' optimization". ACM Transactions on Mathematical Software 23 (4): 550-560. ##' ##' @note ##' This package is a wrapper to the Fortran code released by Nocedal and Morales. ##' This poses several difficulties for an R package. While the \code{.Fortran()} ##' tool exists for the interfacing, we must be very careful to align the arguments ##' with those of the Fortran subroutine, especially in type and storage. ##' ##' A more annoying task for interfacing the Fortran code is that Fortran WRITE or ##' PRINT statements must all be replaced with calls to special R-friendly output ##' routines. Unfortunately, the Fortran is full of output statements. Worse, we may ##' wish to be able to suppress such output, and there are thus many modifications ##' to be made. This means that an update of the original code cannot be simply ##' plugged into the R package \code{src} directory. ##' ##' Finally, and likely because L-BFGS-B has a long history, the Fortran code is far ##' from well-structured. For example, the number of function and gradient evaluations ##' used is returned as the 34'th element of an integer vector. There does not appear ##' to be an easy way to stop the program after some maximum number of such evaluations ##' have been performed. ##' ##' On the other hand, the version of L-BFGS-B in \code{optim()} is a \code{C} translation ##' of a now-lost Fortran code. It does not implement the improvements Nocedal and ##' Morales published in 2011. Hence, despite its deficiencies, this wrapper has been ##' prepared. ##' ##' In addition to the above reasons for the original lbfgsb3 package, ##' this additional package allows C calling of L-BFGS-B 3.0 by a ##' program as well as adjustments to the tolerances that were not ##' present in the original CRAN package. Also adjustments were made ##' to have outputs conform with R's optim routine. ##' @examples ##' # Rosenbrock's banana function ##' n=3; p=100 ##' ##' fr = function(x) ##' { ##' f=1.0 ##' for(i in 2:n) { ##' f=f+p*(x[i]-x[i-1]**2)**2+(1.0-x[i])**2 ##' } ##' f ##' } ##' ##' grr = function(x) ##' { ##' g = double(n) ##' g[1]=-4.0*p*(x[2]-x[1]**2)*x[1] ##' if(n>2) { ##' for(i in 2:(n-1)) { ##' g[i]=2.0*p*(x[i]-x[i-1]**2)-4.0*p*(x[i+1]-x[i]**2)*x[i]-2.0*(1.0-x[i]) ##' } ##' } ##' g[n]=2.0*p*(x[n]-x[n-1]**2)-2.0*(1.0-x[n]) ##' g ##' } ##' x = c(a=1.02, b=1.02, c=1.02) ##' (opc <- lbfgsb3c(x,fr, grr)) ##' (op <- lbfgsb3(x,fr, grr, control=list(trace=1))) ##' (opx <- lbfgsb3x(x,fr, grr)) ##' (opf <- lbfgsb3f(x,fr, grr)) ##' @export lbfgsb3c <- function(par, fn, gr=NULL, lower = -Inf, upper = Inf, control=list(), ..., rho=NULL){ # control defaults -- idea from spg ctrl <- list(trace= 0L, maxit=100L, iprint= -1L, lmm=5, factr=1e7, pgtol=0, reltol=0, abstol=0, info=FALSE); callstak <- sys.calls() # get the call stack lcs <- length(callstak) fstr <- as.character(callstak[lcs]) fstr <- strsplit(fstr, "(", fixed=TRUE)[[1]][1] if (ctrl$trace > 0) { cat("Using function ",fstr,"\n") } if ( (fstr == "lbfgsb3") || (fstr == "lbfgsb3f") ) { ctrl$info <- TRUE } # This emits more information from lbfgsb3 Fortran code. namc <- names(control) if (!all(namc %in% names(ctrl))) stop("unknown names in control: ", namc[!(namc %in% names(ctrl))]) ctrl[namc] <- control if (missing(rho)){ rho <- as.environment(list(...)); } if (is.null(gr)){ gr <- function(x, ...){ numDeriv::grad(fn, x, ...); } } if (is(fn, "function") & is (gr, "function")){ ## cat("USING fnR, grR\n") fnR <- function(x, rho){ do.call(fn, c(list(x), as.list(rho))); } grR <- function(x, rho){ do.call(gr, c(list(x), as.list(rho))); } return(lbfgsb3cpp(par, fnR, grR, lower, upper, ctrl, rho)); } else { return(lbfgsb3cpp(par, fn, gr, lower, upper, ctrl, rho)); } } # end of lbfgsb3() ##'@rdname lbfgsb3c ##'@export lbfgsb3 <- lbfgsb3c ##'@rdname lbfgsb3c ##'@export lbfgsb3f <- lbfgsb3c ##'@rdname lbfgsb3c ##'@export lbfgsb3x <- lbfgsb3c lbfgsb3c/R/RcppExports.R0000644000176200001440000000041113625774732014570 0ustar liggesusers# Generated by using Rcpp::compileAttributes() -> do not edit by hand # Generator token: 10BE3573-1514-4C36-9D1C-5A225CD40393 lbfgsb3cpp <- function(par, fn, gr, lower, upper, ctrl, rho) { .Call(`_lbfgsb3c_lbfgsb3cpp`, par, fn, gr, lower, upper, ctrl, rho) } lbfgsb3c/MD50000644000176200001440000000303213627435412012254 0ustar liggesuserse5387fddc4ecde8a6dee0ae6fd0e6c02 *DESCRIPTION f8f501c3f84d5b90345eb1b3ad757c5a *LICENSE.note b20fdc1819c0d30c2316bf678f98235d *NAMESPACE e8bb773781afe8bc4b51a4ba7062b553 *NEWS 253afb08ed03e672e1d6e3cb732433c1 *R/RcppExports.R 86010858590e6b44e55872d93080bc12 *R/lbfgsb3.R d2df4867e38ed95dcb5b0868bfaae0c2 *README.md 4fda00923c6537479c0f5d2825bda932 *build/vignette.rds 5d94ea6a65871b83d8e2c31f18ba3c46 *inst/License-lbfgsb-orig.txt 99087982354202068e89de9c73db36e9 *inst/doc/lbfgsb3c.R a9dcb37fb52c3f80a7db133b516acff5 *inst/doc/lbfgsb3c.Rmd 3d71c84f934641dd25c3f8e0dc2f5552 *inst/doc/lbfgsb3c.pdf 47bd40af787b00f36feb0ca97d64b366 *inst/drlbfgsb3.R 8d037fe66106ffb23c656f0d043a2afc *inst/include/lbfgsb3c.h 8caf0f8e25b4d7b8ea17c494371fa9f7 *inst/include/lbfgsb3x.h d72333ecc568497a25caf0d520c6fd22 *inst/lbfgsb140731.f a61fd32ec1282b1c7a87fb17379ab150 *man/lbfgsb3c.Rd bbd02ee9ac813e011babc30f84fa5043 *src/RcppExports.cpp c1cbc852cb905e68febf95c151fb994d *src/blas.f 6d12f656af0ffce27c28f1ca821c055a *src/init.c ea65d7d51ba4440bad3255b50bc9f4dc *src/lbfgsb.f a9c87c33274f0d048088ca9756135acb *src/lbfgsb3x.cpp 3fc355272a3ea35f446ddf790c594653 *src/timer.f 56f89a83382cc6d2428cb982ff9d7197 *tests/testthat.R 93f45c2e0c83744f55ead6178582f07c *tests/testthat/test-bounds.R f48091ab47417e5fcb4ac5a1a382205e *tests/testthat/test-cyq.R 481b8e505a6fd33c3d65eb32ebc07a63 *tests/testthat/test-genrose.R 5f3f8b03bc1b03477072a9dedbc7fea0 *vignettes/jrosen.f a9dcb37fb52c3f80a7db133b516acff5 *vignettes/lbfgsb3c.Rmd d938b619628e03c2c368c47a1a6fb60e *vignettes/lbfgsb3c.bib lbfgsb3c/inst/0000755000176200001440000000000013627233177012727 5ustar liggesuserslbfgsb3c/inst/License-lbfgsb-orig.txt0000644000176200001440000001071513511470416017240 0ustar liggesusers3-clause license ("New BSD License" or "Modified BSD License") New BSD License Author Regents of the University of California Publisher Public Domain Published July 22, 1999[8] DFSG compatible Yes[7] FSF approved Yes[1] OSI approved Yes[3] GPL compatible Yes[1] Copyleft No[1] Copyfree Yes Linking from code with a different license Yes The advertising clause was removed from the license text in the official BSD on July 22, 1999 by William Hoskins, Director of the Office of Technology Licensing for UC Berkeley.[8] Other BSD distributions removed the clause, but many similar clauses remain in BSD-derived code from other sources, and unrelated code using a derived license. While the original license is sometimes referred to as "BSD-old", the resulting 3-clause version is sometimes referred to by "BSD-new." Other names include "New BSD", "revised BSD", "BSD-3", or "3-clause BSD". This version has been vetted as an Open source license by the OSI as the "The BSD License".[3] The Free Software Foundation, which refers to the license as the "Modified BSD License", states that it is compatible with the GNU GPL. The FSF encourages users to be specific when referring to the license by name (i.e. not simply referring to it as "a BSD license" or "BSD-style") to avoid confusion with the original BSD license.[1] This version allows unlimited redistribution for any purpose as long as its copyright notices and the license's disclaimers of warranty are maintained. The license also contains a clause restricting use of the names of contributors for endorsement of a derived work without specific permission. Copyright (c) , All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. References 1. ^ a b c d e "Various Licenses and Comments about Them - GNU Project - Free Software Foundation (FSF): Modified BSD license". Free Software Foundation. Retrieved 02 October 2010. 2. ^ a b c d e "Various Licenses and Comments about Them - GNU Project - Free Software Foundation (FSF): FreeBSD license". Free Software Foundation. Retrieved 02 October 2010. 3. ^ a b c d e f "Open Source Initiative OSI - The BSD License:Licensing". Open Source Initiative. Retrieved 06 December 2009. 4. ^ a b c d "Various Licenses and Comments about Them - GNU Project - Free Software Foundation (FSF): Original BSD license". Free Software Foundation. Retrieved 02 October 2010. 5. ^ The year given is the year 4.3BSD-Tahoe was released. Whether this is the first use of the license is not known. 6. ^ The year given is the year 4.3BSD-Reno was released. Whether this is the first use of the license is not known. 7. ^ a b "Debian -- License information". Debian. Retrieved 18 February 2010. 8. ^ a b c "To All Licensees, Distributors of Any Version of BSD". University of California, Berkeley. 1999-07-22. Retrieved 2006-11-15. 9. ^ Richard Stallman. "The BSD License Problem". Free Software Foundation. Retrieved 2006-11-15. 10. ^ "The FreeBSD Copyright". The FreeBSD Project. Retrieved 6 December 2009. 11. ^ "NetBSD Licensing and Redistribution". The NetBSD Foundation. Retrieved 06 December 2009. lbfgsb3c/inst/doc/0000755000176200001440000000000013627233177013474 5ustar liggesuserslbfgsb3c/inst/doc/lbfgsb3c.Rmd0000644000176200001440000002716113626305436015631 0ustar liggesusers--- title: "lbfgsb3c: Using the 2011 version of L-BFGS-B." author: "John C Nash Telfer School of Management, University of Ottawa, nashjc@uottawa.ca" date: "October 22, 2019" output: pdf_document bibliography: lbfgsb3c.bib vignette: > %\VignetteIndexEntry{lbfgsb3c-nocompile} %\VignetteEngine{knitr::rmarkdown} \usepackage[utf8]{inputenc} --- ```{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE) ## Added 20190717 to get vignette to build pkgbuild::compile_dll() ``` # Abstract In 2011 the authors of the L-BFGSB program published a correction and update to their 1995 code. The latter is the basis of the L-BFGS-B method of the `optim()` function in base-R. The package `lbfgsb3` wrapped the updated code using a `.Fortran` call after removing a very large number of Fortran output statements. Matthew Fidler used this Fortran code and an `Rcpp` interface to produce package `lbfgsb3c` where the function `lbfgsb3c()` returns an object similar to that of base-R `optim()` and that of `optimx::optimr()`. Subsequently, in a fine example of the collaborations that have made **R** so useful, we have merged the functionality of package `lbfgsb3` into `lbfgsb3c`, as explained in this vignette. Note that this document is intended primarily to document our efforts to check the differences in variants of the code rather than be expository. ## Provenance of the R optim::L-BFGS-B and related solvers The base-R code lbfgsb.c (at writing in R-3.5.2/src/appl/) is commented: ``` /* l-bfgs-b.f -- translated by f2c (version 19991025). From ?optim: The code for method ‘"L-BFGS-B"’ is based on Fortran code by Zhu, Byrd, Lu-Chen and Nocedal obtained from Netlib (file 'opt/lbfgs_bcm.shar') The Fortran files contained no copyright information. Byrd, R. H., Lu, P., Nocedal, J. and Zhu, C. (1995) A limited memory algorithm for bound constrained optimization. \emph{SIAM J. Scientific Computing}, \bold{16}, 1190--1208. */ ``` The paper @Byrd95 builds on @Lu94limitedmemory. There have been a number of other workers who have followed-up on this work, but **R** code and packages seem to have largely stayed with codes derived from these original papers. Though the date of the paper is 1995, the ideas it embodies were around for a decade and a half at least, in particular in Nocedal80 and LiuN89. The definitive Fortran code was published as @Zhu1997LBFGS. This is available as `toms/778.zip` on www.netlib.org. A side-by-side comparison of the main subroutines in the two downloads from Netlib unfortunately shows a lot of differences. I have not tried to determine if these affect performance or are simply cosmetic. More seriously perhaps, there were some deficiencies in the code(s), and in 2011 Nocedal's team published a Fortran code with some corrections (@Morales2011). Since the **R** code predates this, I prepared package `lbfgsb3` (@lbfgsb3JN) to wrap the Fortran code. However, I did not discover any test cases where the `optim::L-BFGS-B` and `lbfgsb3` were different, though I confess to only running some limited tests. There are, in fact, more in this vignette. In 2016, I was at a Fields Institute optimization conference in Toronto for the 70th birthday of Andy Conn. By sheer serendipity, Nocedal did not attend the conference, but sat down next to me at the conference dinner. When I asked him about the key changes, he said that the most important one was to fix the computation of the machine precision, which was not always correct in the 1995 code. Since **R** gets this number as `.Machine$double.eps`, the offending code is irrelevant. Within @Morales2011, there is also reported an improvement in the subspace minimization that is applied in cases of bounds constraints. Since few of the tests I have applied impose such constraints, it is reasonable that I will not have observed performance differences between the base-R `optim` code and my `lbfsgb3` package. More appropriate tests are welcome, and on my agenda. Besides the ACM TOMS code, there are two related codes from the Northwestern team on NETLIB: http://netlib.org/opt/lbfgs_um.shar is for unconstrained minimization, while http://netlib.org/opt/lbfgs_bcm.shar handles bounds constrained problems. To these are attached references @LiuN89 and @Byrd1995 respectively, most likely reflecting the effort required to implement the constraints. The unconstrained code has been converted to **C** under the leadership of Naoaki Okazaki (see http://www.chokkan.org/software/liblbfgs/, or the fork at https://github.com/MIRTK/LBFGS). This has been wrapped for **R** as @Coppola2014 as the `lbfgs` package. This can be called from `optimx::optimr()`. Using Rcpp (see @RCppDERF2011) and the Fortran code in package `lbfgs3`, Matthew Fidler developed package `lbfgsb3c` (@lbfgsb3cMF). As this provides a more standard call and return than `lbfgsb3` Fidler and I are unified the two packages and released them both under the same name `lbfgsb3c`. ## Functions in package `lbfgsb3c` There is really only one optimizer function in the package, but it may be called by four (4) names: - `lbfgsb3c()` uses Rcpp (@RCppDE2013, @RCppDERF2011, @RCppDEJJB2017) to streamline the call to the underlying Fortran. This is the base function used. - `lbfgsb3x()` is an alias of `lbfgsb3c()`. We were using this name for a while, and have kept the alias to avoid having to edit test scripts. - `lbfgsb3`, which imitates a `.Fortran` call of the compiled 2011 Fortran code. The object returned by this routine is NOT equivalent to the object returned by base-R `optim()` or by `optimx::optimr()`. Instead, it includes a structure `info` which contains the detailed diagnostic information of the Fortran code. For most users, this is not of interest, and I only recommend use of this function for those needing to examine how the optimization has been carried out. - `lbfgsb3f()` is an alias of `lbfsgb3()`. We recommend using the `lbfsgb3c()` call for most uses. ### Candlestick function ```{r, candlestick} # candlestick function # J C Nash 2011-2-3 cstick.f<-function(x,alpha=100){ x<-as.vector(x) r2<-crossprod(x) f<-as.double(r2+alpha/r2) return(f) } cstick.g<-function(x,alpha=100){ x<-as.vector(x) r2<-as.numeric(crossprod(x)) g1<-2*x g2 <- (-alpha)*2*x/(r2*r2) g<-as.double(g1+g2) return(g) } library(lbfgsb3c) nn <- 2 x0 <- c(10,10) lo <- c(1, 1) up <- c(10,10) print(x0) ## c2o <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0)) ## print(summary(c2o, order=value)) c2l1 <- lbfgsb3c(x0, cstick.f, cstick.g, lower=lo, upper=up) c2l1 ## meths <- c("L-BFGS-B", "lbfgsb3c", "Rvmmin", "Rcgmin", "Rtnmin") ## require(optimx) ## cstick2a <- opm(x0, cstick.f, cstick.g, method=meths, upper=up, lower=lo, control=list(kkt=FALSE)) ## print(summary(cstick2a, par.select=1:2, order=value)) lo <- c(4, 4) ## c2ob <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0)) ## print(summary(c2ob, order=value)) c2l2 <- lbfgsb3c(x0, cstick.f, cstick.g, lower=lo, upper=up) c2l2 ## cstick2b <- opm(x0, cstick.f, cstick.g, method=meths, upper=up, lower=lo, control=list(kkt=FALSE)) ## print(summary(cstick2b, par.select=1:2, order=value)) ## nn <- 100 ## x0 <- rep(10, nn) ## up <- rep(10, nn) ## lo <- rep(1e-4, nn) ## cco <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0, kkt=FALSE)) ## print(summary(cco, par.select=1:4, order=value)) ``` ### Extended Rosenbrock function (from funconstrain) ```{r,exrosen} # require(funconstrain) ## not in CRAN, so explicit inclusion of this function # exrosen <- ex_rosen() # exrosenf <- exrosen$fn exrosenf <- function (par) { n <- length(par) if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } fsum <- 0 for (i in 1:(n/2)) { p2 <- 2 * i p1 <- p2 - 1 f_p1 <- 10 * (par[p2] - par[p1]^2) f_p2 <- 1 - par[p1] fsum <- fsum + f_p1 * f_p1 + f_p2 * f_p2 } fsum } # exroseng <- exrosen$gr exroseng <- function (par) { n <- length(par) if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } grad <- rep(0, n) for (i in 1:(n/2)) { p2 <- 2 * i p1 <- p2 - 1 xx <- par[p1] * par[p1] yx <- par[p2] - xx f_p1 <- 10 * yx f_p2 <- 1 - par[p1] grad[p1] <- grad[p1] - 400 * par[p1] * yx - 2 * f_p2 grad[p2] <- grad[p2] + 200 * yx } grad } exrosenx0 <- function (n = 20) { if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } rep(c(-1.2, 1), n/2) } require(lbfgsb3c) ## require(optimx) ## require(optimx) for (n in seq(2,12, by=2)) { cat("ex_rosen try for n=",n,"\n") x0 <- exrosenx0(n) lo <- rep(-1.5, n) up <- rep(3, n) print(x0) cat("optim L-BFGS-B\n") eo <- optim(x0, exrosenf, exroseng, lower=lo, upper=up, method="L-BFGS-B", control=list(trace=0)) print(eo) cat("lbfgsb3c\n") el <- lbfgsb3c(x0, exrosenf, exroseng, lower=lo, upper=up, control=list(trace=0)) print(el) ## erfg <- opm(x0, exrosenf, exroseng, method=meths, lower=lo, upper=up) ## print(summary(erfg, par.select=1:2, order=value)) } ``` ## Using compiled function code While you may use the same interface as described in the writing R extensions to interface compiled code with this function, see [L-BFGS-B](https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Optimization), it is sometimes more convenient to use your own compiled code. The following example shows how this is done using the file `jrosen.f`. We have unfortunately found that compilation is not always portable across systems, so this example is presented without execution. ``` subroutine rosen(n, x, fval) double precision x(n), fval, dx integer n, i fval = 0.0D0 do 10 i=1,(n-1) dx = x(i + 1) - x(i) * x(i) fval = fval + 100.0 * dx * dx dx = 1.0 - x(i) fval = fval + dx * dx 10 continue return end ``` Here is the example script. Note that we must have the file `jrosen.f` available. Because the executable files on different systems use different conventions and structures, we have turned evaluation off here so this vignette can be built on multiple platforms. However, we wished to provide examples of how compiled code could be used. ```{r, usingFortran, eval=FALSE } system("R CMD SHLIB jrosen.f") dyn.load("jrosen.so") is.loaded("rosen") x0 <- as.double(c(-1.2,1)) fv <- as.double(-999) n <- as.double(2) testf <- .Fortran("rosen", n=as.integer(n), x=as.double(x0), fval=as.double(fv)) testf rrosen <- function(x) { fval <- 0.0 for (i in 1:(n-1)) { dx <- x[i + 1] - x[i] * x[i] fval <- fval + 100.0 * dx * dx dx <- 1.0 - x[i] fval <- fval + dx * dx } fval } (rrosen(x0)) frosen <- function(x){ nn <- length(x) if (nn > 100) { stop("max number of parameters is 100")} fv <- -999.0 val <- .Fortran("rosen", n=as.integer(nn), x=as.double(x), fval=as.double(fv)) val$fval # NOTE--need ONLY function value returned } # Test the funcion tval <- frosen(x0) str(tval) cat("Run with Nelder-Mead using R function\n") mynm <- optim(x0, rrosen, control=list(trace=0)) print(mynm) cat("\n\n Run with Nelder-Mead using Fortran function") mynmf <- optim(x0, frosen, control=list(trace=0)) print(mynmf) library(lbfgsb3c) library(microbenchmark) cat("try lbfgsb3c, no Gradient \n") cat("R function\n") tlR<-microbenchmark(myopR <- lbfgsb3c(x0, rrosen, gr=NULL, control=list(trace=0))) print(tlR) print(myopR) cat("Fortran function\n") tlF<-microbenchmark(myop <- lbfgsb3c(x0, frosen, gr=NULL, control=list(trace=0))) print(tlF) print(myop) ``` In this example, Fortran execution was actually SLOWER than plain R on the system where it was run. ## References lbfgsb3c/inst/doc/lbfgsb3c.R0000644000176200001440000001161013627233177015303 0ustar liggesusers## ----setup, include=FALSE----------------------------------------------------- knitr::opts_chunk$set(echo = TRUE) ## Added 20190717 to get vignette to build pkgbuild::compile_dll() ## ---- candlestick------------------------------------------------------------- # candlestick function # J C Nash 2011-2-3 cstick.f<-function(x,alpha=100){ x<-as.vector(x) r2<-crossprod(x) f<-as.double(r2+alpha/r2) return(f) } cstick.g<-function(x,alpha=100){ x<-as.vector(x) r2<-as.numeric(crossprod(x)) g1<-2*x g2 <- (-alpha)*2*x/(r2*r2) g<-as.double(g1+g2) return(g) } library(lbfgsb3c) nn <- 2 x0 <- c(10,10) lo <- c(1, 1) up <- c(10,10) print(x0) ## c2o <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0)) ## print(summary(c2o, order=value)) c2l1 <- lbfgsb3c(x0, cstick.f, cstick.g, lower=lo, upper=up) c2l1 ## meths <- c("L-BFGS-B", "lbfgsb3c", "Rvmmin", "Rcgmin", "Rtnmin") ## require(optimx) ## cstick2a <- opm(x0, cstick.f, cstick.g, method=meths, upper=up, lower=lo, control=list(kkt=FALSE)) ## print(summary(cstick2a, par.select=1:2, order=value)) lo <- c(4, 4) ## c2ob <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0)) ## print(summary(c2ob, order=value)) c2l2 <- lbfgsb3c(x0, cstick.f, cstick.g, lower=lo, upper=up) c2l2 ## cstick2b <- opm(x0, cstick.f, cstick.g, method=meths, upper=up, lower=lo, control=list(kkt=FALSE)) ## print(summary(cstick2b, par.select=1:2, order=value)) ## nn <- 100 ## x0 <- rep(10, nn) ## up <- rep(10, nn) ## lo <- rep(1e-4, nn) ## cco <- opm(x0, cstick.f, cstick.g, lower=lo, upper=up, method=meths, control=list(trace=0, kkt=FALSE)) ## print(summary(cco, par.select=1:4, order=value)) ## ----exrosen------------------------------------------------------------------ # require(funconstrain) ## not in CRAN, so explicit inclusion of this function # exrosen <- ex_rosen() # exrosenf <- exrosen$fn exrosenf <- function (par) { n <- length(par) if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } fsum <- 0 for (i in 1:(n/2)) { p2 <- 2 * i p1 <- p2 - 1 f_p1 <- 10 * (par[p2] - par[p1]^2) f_p2 <- 1 - par[p1] fsum <- fsum + f_p1 * f_p1 + f_p2 * f_p2 } fsum } # exroseng <- exrosen$gr exroseng <- function (par) { n <- length(par) if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } grad <- rep(0, n) for (i in 1:(n/2)) { p2 <- 2 * i p1 <- p2 - 1 xx <- par[p1] * par[p1] yx <- par[p2] - xx f_p1 <- 10 * yx f_p2 <- 1 - par[p1] grad[p1] <- grad[p1] - 400 * par[p1] * yx - 2 * f_p2 grad[p2] <- grad[p2] + 200 * yx } grad } exrosenx0 <- function (n = 20) { if (n%%2 != 0) { stop("Extended Rosenbrock: n must be even") } rep(c(-1.2, 1), n/2) } require(lbfgsb3c) ## require(optimx) ## require(optimx) for (n in seq(2,12, by=2)) { cat("ex_rosen try for n=",n,"\n") x0 <- exrosenx0(n) lo <- rep(-1.5, n) up <- rep(3, n) print(x0) cat("optim L-BFGS-B\n") eo <- optim(x0, exrosenf, exroseng, lower=lo, upper=up, method="L-BFGS-B", control=list(trace=0)) print(eo) cat("lbfgsb3c\n") el <- lbfgsb3c(x0, exrosenf, exroseng, lower=lo, upper=up, control=list(trace=0)) print(el) ## erfg <- opm(x0, exrosenf, exroseng, method=meths, lower=lo, upper=up) ## print(summary(erfg, par.select=1:2, order=value)) } ## ---- usingFortran, eval=FALSE------------------------------------------------ # system("R CMD SHLIB jrosen.f") # dyn.load("jrosen.so") # is.loaded("rosen") # x0 <- as.double(c(-1.2,1)) # fv <- as.double(-999) # n <- as.double(2) # testf <- .Fortran("rosen", n=as.integer(n), x=as.double(x0), fval=as.double(fv)) # testf # # rrosen <- function(x) { # fval <- 0.0 # for (i in 1:(n-1)) { # dx <- x[i + 1] - x[i] * x[i] # fval <- fval + 100.0 * dx * dx # dx <- 1.0 - x[i] # fval <- fval + dx * dx # } # fval # } # # (rrosen(x0)) # # frosen <- function(x){ # nn <- length(x) # if (nn > 100) { stop("max number of parameters is 100")} # fv <- -999.0 # val <- .Fortran("rosen", n=as.integer(nn), x=as.double(x), fval=as.double(fv)) # val$fval # NOTE--need ONLY function value returned # } # # Test the funcion # tval <- frosen(x0) # str(tval) # # cat("Run with Nelder-Mead using R function\n") # mynm <- optim(x0, rrosen, control=list(trace=0)) # print(mynm) # cat("\n\n Run with Nelder-Mead using Fortran function") # mynmf <- optim(x0, frosen, control=list(trace=0)) # print(mynmf) # # # library(lbfgsb3c) # library(microbenchmark) # cat("try lbfgsb3c, no Gradient \n") # cat("R function\n") # tlR<-microbenchmark(myopR <- lbfgsb3c(x0, rrosen, gr=NULL, control=list(trace=0))) # print(tlR) # print(myopR) # cat("Fortran function\n") # tlF<-microbenchmark(myop <- lbfgsb3c(x0, frosen, gr=NULL, control=list(trace=0))) # print(tlF) # print(myop) lbfgsb3c/inst/doc/lbfgsb3c.pdf0000644000176200001440000077264413627233177015700 0ustar liggesusers%PDF-1.5 % 34 0 obj << /Length 3087 /Filter /FlateDecode >> stream xڍYYsF~ϯ`%`Ja\~ٵ];)Yyfk"\R!Z `Ξ>>W/7w_\|%SdlVw4Z$YJLfw?ltj( ~~HRGERRaCյ4CFuVR<ƉȵYڈ,=baOwLٞf>k!4]Mо+65 VyoJDZX,xĹ[ SOy^> $NWʅzǏmeDn(r1 MBy$L䊅z3}QC&\JD@UVtu3/'SLgSZ R{C3?M] {壨)r$%LDŽm1cw+<'j+@tYH**WҺGduΦt1G/͝~HcGMyZס BrR`T KX5FqI$wS\ukϲ᭠;CQ{~@I4(/G;^|J8 @'1H3QIؤY)*82pdku`$"q`46sOi%ʖ{ˢYw#nf45 'OU=pgjpӆ/-ьt'vxcs;@;7H I">٨85^Ӏk 9%q`~\ =_\,I-A%.宊d* m3NOO=޹]䱈ӳ=mIc`ZJ; 0 ʋP} 1x<ԷNL;͖#M US҇eX5 v #6|텇HgT| 6GyO>i# Kcl!W\59ISaRVf`u&+TR l~i-EsLA%vʮ \O_L$D4[os҂8#6ْOIH8:3ttn"i r{zflOYQQcĖQĖQ@fc^ϛT$T.2TuHP!WzPU};(y[ }w Ȟh=y=9ՏA3nbWMW5JKKh4VwSOLCS(G"a& 4%͎%>cf=T] a`p{ea=qey@>ƒr} g ?s4#3ڢEӘhh-#zBY8 NdƱ]k!zFbRHK uD4/Ҭڢ/ ̒!V#EMABm#З@i@g]}~Ϟm5~:x5C/܈}! Ur fIXB tM5o$!Jb6'[cymzLiϗ(^_Q[ pO|e?]-ha{ohSvoFyߕv[|fdqDގu91nWՌq4hpw nh ~{S6b؃1|hngg sm)2.w͂sᩯ#˩6i -3c.(GX8뽅) w"Ԛl ^5@7rv $:=+KQw`L97o>`u ՟75Wm>|-ʶcvaB nzLyREzk?2ԣ7 -#^ ᔼ7 oebI ;梅//{L'"*$E_*wܥH2- zpЎ4H(Y6HQ p:*>ũ,DЫ 5-* W.0vp:|2/v(\(Zw秼&8?8# 3|t58cԠĮMQ0m@E9#?1+vd< l]?5uWly;,P G' #Ʃ+ti3+vjY_B=(>, c_h7TMjotP8J6,.,͵灡jg  W8*9nu?U7 tJxኆH~Ma Cic6[|F4`_Q9n|΂~gqH/G9S1$?O?-{KJ*,>X^ЙK#߃=b >T$X|}5?Cn/#ZHp׋=go=ieTĽ14RrsKu%DK\Iճ#3A-[1gҴh\}J<3Lt[&h6YtE+طƷPcԫAl^_B:|tY9lWa2<+rE y|?JܘKr?0~#]#j?B>R$00:j֡LSOGOPIaYbALj^1(Gj:Wl|neEP(ˬ /AN BkX/[& endstream endobj 51 0 obj << /Length 3583 /Filter /FlateDecode >> stream xZYs~`UUD+yXVފR 1%o*==8(z< c,n?_ݼ E꧑7Ei߂E"?fwU-ʻ3xYK}YG=. S-+34wŮ,9E]Q纮61Yy\ߜ%0":7uCnkݖ&Rz~Si뺪|@GAGk4"-2%w@6yVؼȩQ5&¡GpxcK{;)Z)j5n{_lθ;t`yEU?[FI}Z-UUC"LNbGe@t76Pp3eUp U"6+r79;\fWUUJkt)}+}vHHS W>ŒXawszc$Tacۢ}cE P±Лԏ?[[ ,Ȃ TSkMR,K55Ϩ;85„-:[ӵEK1Y*dQߕaZ&TO6VC&lm^f!6M9a^ D}h`UKLG^fD+dRHCL%uV=&6CxC14pkE>┿ÁۃMZ8+Z"_7vELڎgFOD*A DJ&9`/ c^AuB8X ۮ x T yۘ*x;v5i\x:5H_۞(5_t|h&1Y[Wh#4Wq_%u@c8Ojg϶oU/ya9ϖzάlI=cXV BG } TD0[Q(oGi*A$Pڴ+y M) [cIar:TAM*X >L78>ނ{A_xJ",L[VR*pHopoǷ50|o)& *M_l -4n74lG]i;TV`욄:NiX:͆< J紼>Ii Ԕc/6JAg-:+]o;FF8PqqOs}wUx[1Dstc.U=_S$C1:Bu'JZ W> ۫kn~x{jkm7>$3\kg ?M@ԩŹuy&^l x6R_޿.s#G NdqlX2uق2Rdbߌkkdm(L,~;o$(</_ggLu"}ȤHԨbxeE:5X v\L9K%B,^Uk$L 􃭁'D6.kf.OT(5Rg%+ ؗQ4Y[p[ wYTP:ӘSܰ٫d_vْ+>%t?4rKaJ$'.z0t 4R:q z8 1g t YLP͘l=Va2g``$* 4s"TM;'o!/GH.TJǢPgNS(~TմgA:"k)B:}סtlLي)wn NN} l]4 c9ʖ 4P֕lMd/JW@X`va%nasH#eLsX6n ,ΗDچke(lD!Y縠VN2?$dg|TG;úa2 * !Z.O}oo^)~ڪC,}P*xK1K|/jјşF_! /iҬG># Ro xzGsf-a̴xvK- 4b)OYkK%!/R$7빶L`ΣG=ì*pŻc[ IsdsAsƟg Q:\ ^aC2܌ԜF|=}@ mM?Kz>TGeko**j~;-xџYM tܦn`# 0Q’POP>D&bͨ l3eu%dQ)~:Q$^:fߎQ0 endstream endobj 60 0 obj << /Length 1148 /Filter /FlateDecode >> stream xXKs6W`4LtUmǵ[Yh$f4+ 骇/"arN #w~=qN0`&DJu&AlL H1w_B, <Φ&4a`~ 55=پUNp*T!gSq/S԰L2<W!tt<h+>pz(@i@[8鏏*>?lRUUN!j B7{`B! s|wB"Q'UI+*FgV\mP1T R3*Г!jeIIZ8NVAw(Oٳ'7,Kqlgdn1r:i9S^mz!`Բ3u\[Auw5lbLv"MۓȿD2#_%E sj!{$߻t8+0l:Vwa=doO^w9nU:fܘrC.7/*8k@))DTg*_3Ͽ`d Dw!:\ND)d}9 .6Dƺw)nGnPL0$,֣t(N|i#$535)f,~ _ K7 +".7 G(nCI6DXEͣV/i; ]}uKy2\CNQbFmv8%× ; Q0[Aơ\C$~2G`svųhح~.&Wɛ|i6\&6x}.x>{5`̈́X#OwE3g$BM endstream endobj 64 0 obj << /Length 1334 /Filter /FlateDecode >> stream xYmo6_-t3XA!K`Et/Yf82%U C{IRLYQmv,IysG pB0X`066FQ իG>1tL3m<|}?-XՌ+|Pbb>1J,$֦XO0K-=w31gq\~:[:61)(^)xUޠ9``9a "ĸaS>0mb10iVesM@(ԍϽ#7vz DYp?N%+.Z!P=΂,~̋.鞝|"Ay\/QͬPQc6cnD јc)YTCxu` .9W*HXM%̂r m<-H7h„B ,(I I4rY7SV s ^A%jJ4L>yRqԆV=$Qva?DMϯx/w/өžI'.c/o{k2:F ;ʆ ,Ni#}U:D~Q2‹lџrB^ $TY8\(*cC8\/ d$&^GKZ>2L٤MFvQ))BRhڈ,B_H Fh[_-F't>b4X$9c42bȋIY)؍M0ux* ؄C1Hp D(ծjG(^z7o lGvmLÙj96p¨qHR`u%NH-h_;/mibˆ;p?%ĵ$}wQ2jGq !I5** ٽv 3|M .#Z#OKjN=iydm6mbn6kMngoMK6!IiUpLo1bfy s&\H֫:=yp^z8=?{cJxaijAnTkhj{7f$ endstream endobj 68 0 obj << /Length 1338 /Filter /FlateDecode >> stream xZQs6~W icE-پ)/L/M܀)؎1)L%eeKLvە 44`e8Ճ2 OPU4lf!răe99czrs4 / d5T)2R qr(r8~'99`n/R# `"1qdv((mr2,y?b ՟H6¸AM=VEh ;+R.uzI /" ~6x'iyrYX(?!2FL< *1hFhF"]2%oůYXؾCM-j.;FjyDds{JDL#((*~ "҅a2tq;~ t= JҤgh,ss*Fm)NyG7ngEM^(1- oV[ּ ͓x#>*jWnV"Z7Ȳ5uV"lZ5"6)]YmOYEd4]EFdgC}q@ų0(f ;i!7|EZ o?̻7ɏ vA6P [' <Ń2fF(hh@y]un0Hp>hV=P@\'|L6H +>jO?UXF4ۛfXrSyNT8aqFh疋͏߻`aœ6g-kҥQFa!VQFRTrX MfWO cds' 0_+b_~]߄u-SL!úx*EM*E8'Vٴ#dzzwYZ1NDFf? md3p$˺)woW3&jEOoU{*/,g`ebģϴp&\? endstream endobj 72 0 obj << /Length 472 /Filter /FlateDecode >> stream xVo0~篰Z$}(i6u0Ѭ/UQj"6Pm`F $ !rO}Ο)JE]n:<|OH1JP(~F3,_MYRsbƈ/h`п64 oax# J"6ݷd$-xךؘۨ;|S=-v4SsXAs:}23ˉ}n0 ,ZMpwS&;mqt'W?RL4U/L:ٲZU%۱ZwqZf`/]Z9adio`jʕ!@Ɏdl n*y|}g*.bM%\A3iCVCK7<_hВ&{(N;> stream xVO0~h0ik >P``bs,$mDƸ}JQ(TJʉkrdKy$ $$cZm`nz-HyFaw|J^%0F^#}pk;wnsqo{Ws}o}yZiZR&ILIU5b40PN@$uO_zngۥ=~Zغyw@3fadYd4Yla 3!yçKI!b0i> stream xVO0_Ѩ kajt3 Ȓ 0h t1|0/WwWwW@rsZVpLJDTܷww; #G0M:#"T0:ce"iʯ S,(WYz'"EIHnP s$^ twδ0%6/]<5Z%GǒZ'c7W&A@x1DruVš7]oUc1ŤP*e qɪUU|TUn7sDt*jKkXAN x翍wٍD5_~r{qUQtq^E&c:ҳ7Jn*%LFAZ/Z$].o>xy|bIie KdzÊG䁅$2v!EXz*[NWS endstream endobj 85 0 obj << /Length 486 /Filter /FlateDecode >> stream xV_o0ϧ=dp}6>`j.R*B)uP* Gj(e;g|ĄUt |S:ȑKbqbLX$'ӷT6#VjH.olq0Ɇ08N%&`R6mٜiQWF8RIa cSQZ C[YؖP&*U uQ"nw;|fcozh]3W74S?wޗၫa*ѫ"ei&1Rd@?53}*V6F5"k&K @NC47GrD*JhNW*)^^%~1]\?XſWp:;=-ND3NKa>6:bz#8S*m´jnv#@?h5ucHM4w@kSsNg>`m!0.^0 #k( endstream endobj 90 0 obj << /Length 2052 /Filter /FlateDecode >> stream xZr6}Wp>Ю 3c'*jf<Hʋ-O'R,N//" { Pue9'=ߊH]nVZuk4_|q~#tPJ"_|׷Ķ>{}_φ^}g//g~9śo^:'R4YZ+Ms"I,q6_$31O*IJ")oġu, wF5߮A* w~J><>;PBߔطؔc)i<ƦН&;J:6͓R+H X`LKuAS]t tP5tQSD,o0Da=зǺkC\}vpr&~0L z<˅gQ7|&R&d] 78~۴ WFLG!qA r{fq|1ŵ~[G|j|S_'YC+ [~w75e=CEC?)vvN (9x̷; 1C4fyYq)fw_Mj $CNj5z.b\2 p ##Jz=LYhrkDwE)E1Q+Ȥv.]z=bt Ϫ'ƕ4Uʊ,kkf'2 ӌTT9KTKS;rȆ%^7 iNJ'dɁ 9x;Yv KR\\wHzJQS}l6Pw{JaL#4+0 %vn>i eP+r!6h7[h-c*Vu0)L8!*2g.*O d;sjhHx)PtqUqƚV%Pyuw5ȹ |#I(XGyrϴAP48 $":X'j/gfX(b W:.&nkrØt~~&<)ѻuGC_`lXBh`@1! C8v"u_uH\_r/<ptMr_dU2+MͥլL򹽀k^$E ]蘢\9)6)zD\ I& )BG58""$)m+_%!$̗lQ{&H>j稳}L߼}}[CVBQɧ1^3#ݥdœm8=JBƉ`aѠUfR(0b;Qh6Bl۲kyU-טq+O}|;ަٱ0~`\ n40+o9@f ZE۞cr YJ(Q-ͫ3 IEs=|޵Vfg@u](!&:r+sr}*#> stream xZms6_Li qlM$Tbw(F~ xg) =8͕z-v=<ʘČPμtra`<$qzUDi(ĒKz)NBɼF ]7}+lDT ݩ[''ԎmSPk!AH86{-QYck)$LZHƽ.c$Ž{Иy Z=k4"8!iQe\Q*dDsRA?YN]YT:m71ɟ撓HF"Y^.7.8 hKȫ jl̶ |+-`](0-p X=- A 7eRo:؍o&H-2NL,nt&Iy2Y&h {}Q&L^3Pj6qoL6m(ڱɼ"bͼ昿h`kV֧~|[- =ͦ(;ы<7V-n=Z˝%_| \Mt_ .ɱGS&IP%A(d2'㴣BaT3Ǥ3{>t.,EGÔ]tdsymTU'aKםkv.g9ACݷ X1{z䩮1w#,$ o"^Σ[AMܓK/`=~U`wpq'c7 ݊'fo~N7 #űHL[pzdge};㵞mE輦?=|ؒʔ)\̬4R@kq~_aA0kQ]|ozyV L1+s,7xW).4S䤘Lb 4V6 $Qm#|b Ʃ"V_^s aIF9R+(KP"D9(G 2r(GZi >qZIv pg*@oƵpq9 \xЀ;idђ)VCb/AUB{[;6s+CqK֡,!(bj7.HrOGY(=xuv_9t2]5Q%H#"˼?SeᎦqDkVK+SYzHc&g"-mYn{RL/Q 9;<" s <ݷ4 ڒdYa5"5,#W1>{k^.%pQ;pSlq_HE>n?ĄA{?D:2\YINҲw: D41q~ 3SΡPKK3(^73]D5e||͂LٟgV2J&c]B2C ~4S\.NuD2kDpTmA:[9H@ pk  Wܙ0T #B8T6tGXؕJn Hk-2vMc0rI&ԡɀ+=dL ֘-<Y"Dn}~Wo}g[kA ".#~'2|ø,mj't {5K71%ufK Tj@:\&G,r1pVKySLUWѡ!wdZ⎘Gp"ͤ5jpm)`([$FKQ ¨*LDsMmτzK0g/m,AK+T'7F0&Rq[EQS<҅| {V+ɦ*4QI? H\1%!Ἣ*򞆔d!eӬ?֥ɜ16Hn͙˭Qn{N60]0]Wàa7x4ꭔ^)]Gi̍4gR>kT endstream endobj 120 0 obj << /Length 1113 /Filter /FlateDecode >> stream x}VM8ёo;H{z[ {C4dz? =B"]z"}#9Y gFG NS%$h~Eq<*ogTuL !~!=IÃHxc^-DdF4 E(&FE*a`'"i3|0v*PdV/}b뭞uO0jk5I"p7ǼɶEv,wy|ؕU䥬<@!pR9eQ7UY@&/ SԳJ --? \Ϲ*҃) hɳɟ 2-sZb"X뛪g҄X)##aF(9c\Rr~VsOK&4%);իvAڄcer0Yl8*+>c^IVX$)YݶC Ɂ`mn9.o8֟r|=3 &q, :ZZΤE \_kݠeҼ}Tݝ8QNj@ٛG-6 "ի /ǣ~P7Bۚ.F~Q^z${4pjC P!ĥ{&#\ !tV243pC]p ,[ >nV6-(T P:M,#;_v=ʋF\2`Ԅ*mu O׋E^*eLv}qyLCȚ?1)̴zr%W[rk`w4~R@O@@բ+MH?N_][ '8q)0. ecKO B. Mȸ&g}AUl/nzIWER%TV\A!8hE`/muөeS8i?4uNw&I[m2V@XDD%K(X{VEHO,Rr9bđ)H]H'ه̹|E|$@zVags endstream endobj 2 0 obj << /Type /ObjStm /N 100 /First 813 /Length 2830 /Filter /FlateDecode >> stream xZmo_A_%pMru.h 61 Ym%$HrrjHrM>rp8p8ԮdY%s,(I 5c%9d`I$Q1=Zܣb tF*h1_j i; aN/]콅4 !L L)0)8:ͬac XS= R`jRЀOGfqB-ܣg60m XiK : LAp@H:>1Z!vSI@-@{ 4 !1$Ƃ?~Ɓ?4=%PV-œ6 FXz'FPƈ XΑ钅lh**9M4! /DN D{*aCt9raw <S9&uJ# s$|tP'͇a1Nӂ_/ol9O<{~;w8"/%qRZ=ML{o8l_1Ks;ȇ k+iGJ\V޽+H!DNxpܹ@=,JGnimg{O)S_R,t\=2e/R_Eq‘9T! '3OV~Wj8Y%Qxf*R8HVf͇˙k gM=G!υ&a%nk$Gei"InfR . BvD2+oY]6bjH Ї谰J,ֻ.R. x}^[䅴["`:e҃P~XjH{y 7 ^";;|7xr#q%x9mxY˜qcF27\ LtH=YG 7CŶ@w CG:`¢`آF PO7UnIɪ[錳N#sbJ'|pW)'ז<q6DB4i$u Z 9%X\VuGJגz97xxToR8}cYE#D@Q 9)QpJ\ziL°!S2ҋV  Yzdq"]ĂZ%jKe%~t-Ez59^tp^22?z??KcheM񴤵CZN}^Tf~eiE*ޠE 2bEE)bFb)uieF݈$:v1;M<E0HRs"^P,\\גQ>Mg!|QͧδJ%4Cњky+)(Z}:שoK<|KzZr!9dN!":VVcbؚ{^OΦr-ΈZѺZQBڌ1 , EXj Gm ۦ:Ţ EoV sYK!I׀BWV p*LPf\j,[ղ 9veCfi&vSϕsvA|dgY._/PʢJ-ڸbtl;JY(eMXYŽMt endstream endobj 140 0 obj << /Length1 1949 /Length2 22717 /Length3 0 /Length 23943 /Filter /FlateDecode >> stream xڴzeT\-w\wwwwwww'Cp51?jmd rptr; {?gfK&# +!*L/x zꀀNnNY' , dd0qߛTd,dkcO-d_]muydlv * {g }[ff`L܌SKigk05v405y{:M=O00rzoA+R 5YMLdmޛόW.QgkkY,[XXJ8[8Z[8]տNM/26y?D*aߗşgbe/{/YL["_F" #[c ``llOV66qI @ӻ `j 9 BD#.ȿ'3A_߈`og/`0d05d0X|cߐ== }X:[?ҿ߾wVt7d~L?Ld ,ܴߧ]o: ނnzf.6{IXXD? `bfbhkdR#R8]Ek#ڏdJb! ̟y2%)XQ~V>l2GoעRni \IA x+ABj'ylL!6Ʈ$q ͗ @hM/Vk*{I}Wrh0Qmȓt K9Q%{T>՗ ۍ j*]]-|h-A V ˥#o:eD Iۻkkd:aR$4L#"; N  QmOOqKg)4}jq5,PV".\;w9!V0HA!}:̤B lIw84dۼHA}*zʨE}A~xȠo=cy"&d"2XGn}ֆd NX¼ac0UZa+<>5b!x˹p#47bT71ɾKXn0>tpq{#"xb'vcDiEkwEۼg ¸$QvT*M c+оtuWP]_W#cXuoTY7(, Xi*jkޚ2 Y`J'!Ծ-k{k2[ df^cX٤8pXĜL؇xFECjNh%Fpt( |c'/hMώs`Eʹt;5_L3`a뎻6ocF|pylf*"ERDa86MD|iKZy̚xM.`|'TH$e'H`a9{nxCQEVZ%<;RtZǞ&1w782lSx!sCqFwT-^?PAo?B0#}9Ӊ^ Pep5A.'my㒢,?NKYP+36R m~&f-RR:KnM7r}7kصr#BŦzFIaK=EnQmxMa %tX46 64%˳Y(/?95 ShacV,*:߯02En-LMn[[tGd RF r{hߊL&0X# Z aV KS/;e-E=~u f:l>"Trѯ{w>ށpei'!TZqa]\M3nhkEϓm v$f,$`W녧sg15^Dpf{it0J 2E6<#n΋ҋ}kElA Rn߭ x> mj}Їw:|NzBh"QS= qEIc b4f@[4ٝRkBucOSΐX l G%?cJCn*o3jW@8sL`d0C1e՟h5r)fm4B_r*H_H$w Y+ڶ{ܚ0,;Z&KOF՜,finڹ\ڹ-N]EKUTQ½<;huPd:{ªT.[$R ag4fiP68Pay+ zBTşqFާmB-;flՇNH[EvU ^d&m&Ó&O6G8Ȉ~AGI*뇣k6b#LҚ#.btP\w~5JT\uQJ~L(1`f?HyclM'QbB^(;qIcR'㳚#e ֹHua\54% ѵ%4}&\ZO7`-[VSEB H@ *KB6C;m\u-&$Z7$\Wx&@H5$%O!Dzw9Bh_2ׯBԪRٰne6j!]ÍM=/@IW?f t/1Iȥ}5~yN R?!DøbJF&T4`B rK 7Nbu#|nP?@~lY&=2.2 3 p#$; ١ YhP`䎯{3;T1*[tXBҨ9H}َѠ;Yk{"/cW£zY^P,2e.GPmqځaj(GeBی2CdH{X(1Vhfp/|ܸ~" yvS$aZ5׍,os~>\&] x:t\ a04Bv^N' $$j0{o/OlUC˶ȴvt8=K,+o!FPJfIZoeXVбbCXeP*G:x\.YC"~Y_G 'aV1V{J6JZ$)"\.z69adiSUZb#qZ?٨Vø xF?67A{g|N<Ȃɾo2wa;'=5?pu[猪Ux*cI6ZD5% ]]/_cl>~0wj1LM7*u2x8'ʾo8(5h7xHvղy8Ŷ0-{QsTW f0UAѧ,IoWh>f\a[KOKs-Z^7c@Bk54٫}ߥ({CڧE2ro1KjB'cIIZJ]ȍedF8Y֩ةp2: {GeRTrxckO?Tl˅2|ϕX\z|3j8-` 3>9`S^_7ܶ_ݰ߁ y.1VpC|6:n򙄙Xr,tvd4HA_ VSu8%8;/=ێEf4K4^wx2G3 {Sy=!,C3; +rueaWg qZ$H zqbD.5)nnkD%n?X)h9gLɳF%Lm悋/Ȉ(W_毢K8wutw}?X9F;rR>1mY,XC^ؠRBˆ0"|K=N3ʉ4C|\~I,s {<8o{V^ˈ3KĽ \}3z >)X^+Rݹ">4/&`}~41XhʈkChCyAM+ ,0#"-eYЗ;Clί7O.A%nR>MևA@seȖnELBTvv Mr5߆k$>oQ߾5hHL?qsj^{ X,WXB<΄gr H֯Q #.ScW cٳ ؏cu=;iR4-5JCt /kǘ_Eud7 dw~M/-pW,j̗-˻#E|ݳobi!a}K^T1۪z6Xoˠ6uS>5Ԛ 6#nL:j=}"'|p s{8B}u!t~0|(S@N h,ќ#~r|s5"-e WPRɟ 9· v)mx^0~`A8mF8Q4!N{Y:zn4a 30Gμ͉ 0tUgmJzgLEVфS"';,-TΫm[!ikn_e8XQxK$ 034E6qHo I'0,^B[♌fv|?S0=#"J,(a5UnN˿鰶_reNkb0?V|TO0N}OL8b ٺ$dv,~G_L䂅bRTafѕ{U]9z+g0Js jIEB =n7M&?@n3O+uUOSv ^e=؂܈Hư)Yiw7X<~םidZy Y)*9tP_vP*5M~T.y5ӀU9Bv?*63w\0``ZT:B>An26£*ؕ+* zc)uӯ:-[1v}P( &!*'i˱ 7A:MMVȌǥ쓡zRIT0 xFc~NxABEUM:ɳ>p v#:Aa. n0&QgI>}[T6CstAX5In|Fܗ#qVE^`lW|woRO3_bd{YKLc %6m#r:pIK8%N#.ѻ7>1H)A8p զtp ? l<~4Ք+xlٱ`av+|ly3bdvN-.ڏ^O$? vuqe:okX7#"#w:T)u [g2ӲXD ˩x ~br-Hiu#PVĹ(suaYmph4EcɎ6ʻf?5 |k~>cϋh yH⣗c3KU]-/ kkkL(E!XQׁr Y.3-+.0vcg[(sj{ӆ1gڞ!DnikE?+<)sաSY_g +cwj~TXQʲ;}͍3_Aسc0D4҈hvfb)8!`of@"9(SQ'H,r9ۚ'C6ֳuqHʫgzw*׍tl188*k0f %'j 4(#٤#I_+a 8޸hHoU~v\+})1CSۦ:ƒ-^RX|Vh)TE,Ջ &T6x~ki%gKjq|])R$Lq+R/JׄÛ׮s& Lh_nv;'03V}]Zq(<&3KD#e5q.?(+\MqGtpIw '5&0cTxScυNr5UZDXU>@=χa ;qiVJʁK!;^>*?CnRwk{ݸ={$^p+dyƮϣnvIp@ )5/?0m9oVka]E0XOZoϥM9;7u<:`$9LOH{ޒf}uęM0;pX6-+0Fm5_l.gls/,x%E ynv+ >KyNN諵ѥ|pp,Ԭ99D 1a'<:ؑCBW4@ jUkWIܑ&0}Ɛ4~Q>m-Q9cͅS]148aGW ;,+R<'^^`Jsna!.p%D\H6%LKhR?":ffrDƘ`1ߍ1xu X z{Tҙ#D]cΩ\+F|{jYJBjY搀fz C Y-D8^RiK_ &F?gC񴍣XUMr |`nYQ1_®A\ͨ9[+M5it0ZT0.,`81b:zԣ>kS7RWN5:[-a3-I Q\YW(مߖ]kсۙb7|UK9@]q֣`y$ͶPH.MœeEX}7i: _Ws ĞbkC\@󯒾h:kZ E0/ϭI.}7֊&)N]bo8Ilb:S[: FJ3~${I( " FMNB{ꤺX+=ՀB|Or B(>ףZ-\G NJ%t$V2D1B20Sqbϭo)-}¯hUg[J,Hړͽ4@&^0v/'2naliz ړL* iWF!wRdK|լ5NNoyhwF#7ބEL1 ݛNxDӦq[2'es8Rs'; {k>Ý*kZ}W!g[c׽sx :UdSWrA)gkjBWc\X}c ̍3KJ40;a܃k#s E_~X;c{/EօRRxKڇBXVzh7 UbְzihljηsC-9Hɮe3')C\[Vpy(:Z( s䈹{nrC7SYvw3rO9̴썝;S|[y.!d XMdA*z[ @365bK b Iga]:x/ 4]J{\Ds;%tŜ5,c)%?, @ECmr2v|E Dz䵾L4-!SO>l`>z@nSh';#Dqdz<\'.?aB ©hO\HE?QoJv%iFY{ .,7D)V׿S4"s3?޺)p1}L#us*| jH%\o؅q]fp2eCV?)B&3Q4APL4vB{ O\.h{0yN 5z^صiJǣ2w(,bA(]Ėkt̴[I/͚VaQf'f=n;M)iP9M{UnXE>(>ǴKSF@4g#\\#{<(C0T~9g3=gwR+}#'FsyLO/)eagC08'}YdMa E".B7,ٓ-v'UJGç.LJdT^|)FGSaDjg#@G>i*,283U CT~X͈"eTm"&|p|V9W.vO/%v7lXjlϘؼ%4M7ǰb7nh6HZ 0u~[?X4FHykչ#WW^!7t6#y9He9F܋xQ7(Z̴=|=]}tKE3l;[`6\ ۹s}{vA$, pnd:G%9uD k ܄Rx0St"u8tf{}MU=ӣV_0!A};Cc^YCt f a_]L)Xd9hܠs(6P# u.'l"S>@egcoVp.C ycg1e2A:/JU/Dt*^|Pv'?.x>s;JtTx֓s*EFo%AQ5i_SWxR21R@Tr~~kpޞ1!r9)Gux4S0@P|^ \)3OGܙ=|ǿ/Ep Y i1U e`Ei7FEq0f ԮRFg,c”>aRsBNBqe㐱J"Gb*Pu\IMj. ^fJ&Xu Rs_W*q#C/߹yqef(+i7'dn\2 XqGƀQB|L%pq1s%hE6H lUadIuoD7¡ԟ 2SǢP0V[0a6Qb!GR'.WP6h\Z 6#WXs1QaGvf[綽q:fh1؂c({՚)EO,]sCTahMIrƸӯoLT%@b/yg3baM_i L) (@Kxg~'?$IP QG,1} "?$v Ke%E- E1;:3ߚ _)B[W4nye`9iW4"צ٠1zJBVjct9/vݧˬ !Z;e.6q N#i?^0/˳׀Ŀ=g*Fq\> MǺ\ -MaѕΈ3Hn*8vR@ x":@xO=ϖIWK(cyb V Xrvp ߑ丹wȄ5,r*FQ/,3zq2>zk!nLưD $oK[)]eXgl/kT}dmXH?}|Mxфvm qUFl4w1A=w)%H&|pۦ!A9' &W$)Ӟhc:e4oM"eq _9Mݡ ‚]ur)}5DRL`"C^0zІ ׈8R RV!o?J"M/0RG)!ܖzO5jĨU qo@Us50{77Xoqǐ_+") wjKFYw4"H(ZdG쮭/`Ss^Jx38n,}VMQ`ɄH5術 3]7" M0(##ʞg{(bkB ԙel2@ ]v:M)NH`v:RW*ml բ*[d,OZ* nIv ϕ(Gϙ.?%#~ rptdǖP,5G=lZ7oh@cM-|@뵲iTgg7}DvSfWH|$,ϾHR%jBZDZV.XQ>viďR._|44{0 Q"_"Z|5fA׸B$KJёZƳ#zh #REF!iK ZSah?TPO#2ey϶nVGɣ/ Dʕz!*i1aPvAfg)g}<6>)z7OAUC=1 QIШ}ZQASuKLѪ?ORY5erO#gĕ]2IyP~bT?6`@yMfb`kا[?|R_cCm?իԟrM77AʘXV ׼!MYeA}eH$:K$G{s%,4ߛL+UM$e_B웞e"vYU2z9>CiW?M3$U! W{X-S gA)Gڜ;ERMxMP8M+Ҹf(p푫x#D)YnN|h*LJyHD\A i>v?۞|^yv嵗r|x&[W8&t ^~gQJ4l%+p6?T6=;Eg_B2% 9W% =@O!6zj׎2^vc2)yT|~ei= -՛{ӼN,!M=(&sD󽗂I8pS\S<. ^?0S܅i/'+}asq',jQ856h:בqVWt|v5AJ@`IhXM.xc~ 'Bf$וy܎i[w{.j`d@vd)rvXy>v9M~.[&~}V:a3RY .^+E07wJ}ivGV5vK IZcŁM:#檟qCMI0mYV wDDZi_50LV;Jr=DEDzS1Ġ4IJC$4+Fi=xhCa.h2>c n3LLY˞lԲ=)rn xCŶj2|VUPw ^g'-~j@Meb띣o*r+Y Ǩ4kևpj=O*DϢFۑ|z`@"F[]"ȈOlyT 4G7 6la뺢@`>~b,jcT=oP$L ܚi|R5x6 7}ȁ^\72V8*!jV%ź;GPlXߙ]]@ѝ,2D c? ,7^P_^_%Vv?.{Vֶ`[oPZ'Wo&d9]]ЎR)7CWf:-űV܅DA6 ƀ{hRJm$(. -Ehve5GpWm5L4Z`VEr6;߆5oҋ:i ѩ喝v`t#r8#"/<ҩV>[ bo;oux=(bۄG:/P3&q@~AgNjN$45 qٲ^V=CT h/Hs-v?e$z F '/n}Q0UlU69|Rۙg=ݼ0)kLVbaq‘3Yq֭ÒSLIP>Q-r9醺wA۬ks6 m>3ɿ'}3 J>0f- NJřQڎǖҍRcc j߸c(k&w_M-o`oqFSW6|MH+n}vCк^s&^#>MdNֽfP39F\U` ,eY;>B˂;EL|՘f ȮZsX@'&6h”lhi⺎ FGV&{3As_(!M~:nlz*-J=(x4P}杂uC P5K֛y{.Ļ~H|xht 8(r7zt%*ȉ\-{F;:?Bx_㡜Xu-Q*6ЕE vNU OF!՞̔'-e.nlX?z\Ql׿ e7Z&ˡiGQQ@[\bG#w9lNb}!B,ĝotBqLn48LpL~^*TTm K : = ĕ_Sva3]Kj.LJUKWT'8 (da#+x }^R+JR/cSrImG$U ma@S;ϊ pK97O%T[EďP=Kq0Wc&ڈB=„2X%K2LV|+_1a—x\?Iݏ\]G.T; l}&S,H}7/@Td2=\b8Z .x竮MnAZcTdHOv28 q! :dd9-=0]A2y<ߍP߷uu>_"ehR]nAf9#d%ݢ98MHR&ͤT$,bFICuhQ(S(1 KvFh!f\ #H]&07:nW|齇 r W:"@>0:f"à6(0RWkT!NSX@UM"&5kwMW rn'@<64?Yݴa@]U*,xK, )'Ɯn?hﾉ;"$u L=/P.2]0OY nzR?va(ڷ~fO{MKO ٿۛNzĴ2\ hq§7NQ}|L';OI jWWhE=J\8x 11.Zp[5ۿ_7,QukXW-x{/J;q$E%PNs=EÚQϔŷ>pUb}4KyQ@r iW5֤̀:p^u^Uc0ueؽrf@1+B(%89--F0GLm"_? [\km$ ItYJ㬰mce=QՓl *G[7ٓ™.D4ѳVV9<6R-A[?oNtW'> UU|46;pT(|6tab }qFjڠ`0 %MQ1o#Am@ s9Sp!6u+xqp'\.ҟ$A]FT$?E*ԛ]y+{9 Cmҳ[%zipgo7Ud "ڶHP"㋫$=o˹-U]76N-޾vegӕկ~Tojbܘ܈0"YG,l @,r5B!QtrH+b;$SW:mbqagguj=^ e^OSœ_I.S]H7,rJY&L7Dw;蠁hŵ)~ ~Gnz ps;~4[RJq?V3$wHIx-3K.3yTT[eO:6xv4.D8M"F'@Vj!ρ;?5[F,K-"ՀGx!5 (H wbcSzr~Mv3U¥P=\-3s*>íDQe~!2la722oaϨePm/sw'F` ~{jq[a{\dލXuaW?T fuwER%EEybr;<>%eDw((2OF\ba@+߶tX%'\pI[;0]ٻ9 V-Vq>9U/\=5܁~ ɫR,Y.gE|z^p:}˺Ěka?EWcwv=P̩VJZS:nH`27#H%QH Dٙ_)7z%,WrE /𴫿FowB0V|:##FiԾ` q:2[Pk+.O7ZKֹ-T@exz˞ycڄay/u2"r^(xQEW,FC&[ <~ nnJ\/16=dn7Gpal:T4KXsMjĶ\=`:zXE82uktEz#= wUЉHBxGǓ,֬c,\e1oS[WOuz򼿛|=9.(Óbv ].#i}.2Ep}~2.6`jY.W%e0jfl`n>ZRj1xV!슱TUB+$ЮbLH .ml<+ pnl3zaxa2/\]8ZF0ZUn*9~0?MKMPnUa5(bYJ*],Ep4{V郅~/-.P?jo%<ԊU~ VmQk/S6]x8k2(3[x7Eգ$Bl_m² PF̃2c!'ڟ,|x=0L2X6pxBRO q3Kut\GB3V%ҶHzLU[hv󄇈p sBycc&۹r]YP4ϸ(נ b4VOBVKPT䐆{$HPD羏Q2_i%F1ͯL R8 680un^~j]z4e3gzK J!%YoLVbƭ$T}*Qhlep?M*%M(7w^FBUsv0G}E\}%D!^.P=ܦ&7?x79$L`8"Uv c%SX|'-^V2iWI,P> f^9 2(5" n?j#ƭ &zE{{ȗYdj5:?F_ˑ4C@OވR$EOا^zR7[qy{7#Nc-CXƞRmv*GHp.Fq="+r{#S;P2y'`~fL?8[EK a]v{sΖPjtgE#sN2t`|VT$ Mb[х60f0\Ǎ7=QF o HL@,>Ѱ{˃}:|3=vet ;U;<=Qب U*#҆ٝ4K/m í薌 nށn 3P=Q~員kn>8`Wk\׃RߧG@!:1>1zШ0Yk HAzIc9OشΣ#QܡP%em$?Z|Åٌo&Tc~3 hV'i$?l'DhH) MvR<9,HrkGItsk|*7AbزN.qy> stream xڴzeTۺ5ŽKp ݭ[[n] w(}={3o0B2Zɛ*2e5&Qs)Hdf+MlLb`;s3++"3v(%:XYyR `P{9Z2ŕ rvѽ-\`gbSO3@bk 0q02+0oFk-` 2- m@JUICY#kH1>*K@) 5?Ao-o?}+H(KY\p9Xi_ܨߘj ՑÃŕlh?u+k 57qZUϒ@.?I9Ffw7Aip mL\ʕWV؛X;L]M\\ sݜP_/u1ەx8xc6yf`kWU,@ػY3kl 2jLos`RMǁ?D?xX@^H%o]ۜ\^,j[[--L͑E $bL,AV 4b/1FvXع-@oO>.& 矎D@n6 _e,ߘMJCv^s""M tS4@;?+D *l"i 2Wv5Teq5ymE2iGvo};tY& 7޴hfrqpq [* 3% `l6NNM ϿD`av\`g? V_ "o `,RH#HEo`Q7x,#^ٿ l6Yߚ|ca`7x#ehg7?S7:vo\7f(|c|cF?ۜ]hp rKKj`[?BL\=X߶3 '?>Lo`b9(sGٿ7/s@ O<،?&[xDd9 /ql"Rd!އMrpIl4jgmPlʉ+s-BT ё|Mflr"/--$Cq޶XW4rBO&,g; E4vwX1&=K3E8#ݝh:ėKg}~g}; ?WawwFd]YL0ئ1\7_V ;a/[V EĠ A.2E :{IknH0:#SӁ € 9֯ÏԾg&ns=TLor""z2 XpС6#<3M^Ȑ a_љdкi{##soF͐ET0fpD{9gX}m9y8p9baUFyފJ L@?}K3*ji`9,JcОXvQGI)WH,R'0:$'M*wZ`2^z>Kt3ڍ"5*F%C-},27n~h( (4!ip8:VO ׆)LO8չaMG?{E9JAw)H^x~>N\z@4΍bn1@"D^Ĩ::.a,j֪g|GڽgDY[gHfRﻓ즪fF#7*9Q =M 11;/2*~/^HmЕ:\%xIT!셆(Jz`KĤ?@ ek'X(Kdax )sԥ~j aY*g.!x"tu@DtƵ_$DlVO\vZ!#o_0 vCWI³iljb~Y+H<4\H!, }e$p,*?(H)~yLsTGAN63/v\ Wx I\.7lDn8qd3[;|30 `~p~kMoz $4~Tv*z>4jfXy?sն䎸uF2°Y@{JkgC_28~?2@5I~r6h7ҟ! tȦ eU#\;TЋ70D'LC.Ί'к β A}#*D%'CJ"T[3u q5F䅰+6ƾ ؒb.6v5+Ю4;%@ǥs@uQvxQjAgߓq~tUm٤W蝀`,y]D'J*V*3 BI8ROuVo7`og;F}iBռem>M/*VlDzs'MV׵ޥxc)lI8i8VCO[eߓ*5lVN3!濤; klL\[ԇ.`Ş^>,p;مNiF؄?ItIТn+&-q,mۗP`!W];CFnI ;V,dvu'%F#^Vr #rF}{" l(R_5 $C_G^}e\`~P4KɣoEfە7+7TUY#:j"q}PA&.la+撛BPLnĖpH}c SN|EcD*I+ɱ\#zk6gjIy( E5aMJvUGfDQƒ,|i:B[z-I|d2jS$fE6bv-wWg*sOk:} 7u I*ȂeFdBT)r7n-c+zܿlho N!${2oESqK$mIx6Dj چ7x7^؏ދE2?aEu{ n#!¥?/@$m\ўgzJRШdsɹtS&?Srw:6c Q0+i4Ϛ>&uZFX7dxi(\Da;%5%;hEBKh}&vYdmc]\v˚Eg"kғ)Z\/M[\oT)AViRԁu_H.,c+y.?ލ;r`pWzsD A&Jɝ%VB%:޿ iZ-=[mV>pI\=sݳZhx3qw_X_Ulpàq6j[~\dZg7ܓ).ͩPp,AvG^M1+9B8oܔAg%'dJ!/ XiU8!4#Ih y j O9Ycd;hz{E~퇵of.w*PKm:-]ƲUC ThFgxuۗ?(>#D=~+BeAj,riXq)b39JwX҃F ŷXD: >yysv3Bɷ$K"E&Dd*:ToLQ+.|hJJo١f\@{ "링"\v5'ҹ&S~"PޅalŎ]i\:hСΞڃk&I1 Wg w4Aqt(_Yr-T΋6bp rC)&r-j 7KH _IPb}$vy=Y1Y읹) t cYFʋDTԧSv J:;j-ļ%@! .wJlCaüA^QwXT $Q#xh~@P";$"i|%sJsErDy4$I`kRs<拫V<+ӇQD{o JV?k^"wwfIkՌ=ANGZNMlP/ afʄiO8ũAp@@S(wӘ3 6eYxWqz?E2Er2f:)A) IMAM-9 ҷm ?Jg[ˎy>O Se Xȩ PnW2,NeT 6Y{5ȯA]>r(' 1ؕ}zCNlSy2I(gD{el1wwKZX}^11p0V}qnѭefnb\%tU} Dz m3/_Cc{p4 q,JRn6h|=dx PoI4Ѝ5]%b U˜ rMWE k]TW"|$u$Nwj/}yKICf&Y9Ud*m^x?#C6agrM j~{g s%Q]Re$jOUt@wݣjC UG?;Dd+WgO|P"p%;Lmlx._ja'یv[R)xN;Y8BX#H{^gf{ [&ITTO!6[Mn=jbKJG|ՉrdVrI)FsL(eq {G89D Y< x/ l :ԒM(s8 iAnYTX3P5"8f ,jfϔ'sHY1"}@_',K9Snq|C8t.x'vT3fJWXg4q1*}ʷIiJM0,7t&;p֎ #bJ6嬶(qGt j\jlaN+H9#_7~ѧW]yѣQ"Yg']a>0~zX?}X*Klf#䤩*e^ǎХhn k-՚VfW~4Z/krOWZ u!)ם#@1VP]~>2@DVg1ifOZl{Cv;|,X>l!%v+}_+9 q3C )p\WJH.Q;(^ug/]V2}Y9 \)Ӫ7dDW,7~c [g59/.̾lnOcKcS?DFvN *8.@NF()uujza*3Hޡ^ 0qݠC=۶QFs{gkjҒ  2;r#azN L(TeK.x sW0t;yƼsBC (]$[Y-Lf6:'^HA( ip /֞B2WTҗl0 C{E| WDiaξxwO;c6_;/|Qv?S2¼lcc|F ׆w<\ w-LLaȟZ{OIxYT>ld3j}湓_<wrl3t|H^7gr^y{˜ddװQSȪ LJ,a~zrG1,?ʒ:F&cbl5i]Úy*kr2kA@*rEh!ZW#L~eLWvQ߼)m50Vwy\3lVaU3!,O'oL5i*fmO^s,i իQlyg'oք_(NR<|Ev@\g\-^A7Ot?iaׅyۯۯRiT(A}n$R?R*U6llP;i(3!GP6jU7f3E)TAu 돵P NՇ;4Ԏ⃿J(ٽWN?aׅl aܐ1MD6y%#Pgyob6NaǟZ$VzJI 0qΞtpnS4߷zذbY@zȴ?hZe@%wS~ͰNh[J%" n dfvӃ3ystIܮ>t#ea3̢Ӳ5o +Fȍ/wȫLpASQ7|fvi% ՂϵHkLQK}Et\ 2X<2EL٣"iy%(Cm_(:¦6=0ʰC.0I*ɹqw-ݿt`+P ѻ' q kҒ){RDjNG:}j1V۹>voLU0CTvp1 VzC+}Tr`ߕNW+{!ԉNF'_Bz{*J>:EV4*zJy ?/a!rQ!%frVqrrQ,()C:Uͭqה xlUeb|UL:4#=ȘϽw<5~ERhwsΔ}cȮh@/ g1|1?\d ,4Hf ˼^}~%r$?ٻ{غA;4S=gs=^k:^S5㋺=-Bz15ߥ./I9918*pTr>'J°:`5fuUhhc0'sᜆ:7Y=FЩǍ@]K]HKN[]܎!m8N+=vճI2CZߕit!&~\;/i1hmB"PgxE֒-lyɻDTllҘ$f#k6mj~lW|7ּ=NhW&`׷Uіg4J?8OvK*ħw-^搃}%!h0dR^3xpky,vݺ\ytV~oQY"R$ 0웦Ih<߷vòjic4~ኵUP_u }[ QRl@gLSIx+U]BnQhĄل'ltKL}W܉>@|tDexTx.(/^'m{\*3wb&eO+[!P5ߡ/2Tk^B(O5xC/̅A@THbX $y4g kFS5r@. (kcvS\ȴR-]٩N)nƐݫ"1'6DLԪɂJ@0Ec({1G{ƒTוP)/72ISEl_%Sl.x_׻OS"U@2 0(hl$YYcfH!ȯ1iB2Zp>5&?.h\{T#3\D5C[ʫq*մ =qWǩDt>^G;'dN]EvB谖qasbMi:Hp~Hrt3w-y$HfcKxRȩ7+r;+(l,Ӱ8͝|R]~h۞4(aH/TUB:ݒ_7OPpi|׺nxw _֩_Kx)#9V\Zoǻ4 ^ X" #EG]n-GË.q&/bє9e}NSq9+4.->C7l[" %Y1MU9\֖nvhw.ATTY5owhA\r{5++)Hkra`%Z ~Heb|!SO>&z? X`j̨N_}_OL!!βk6&׷o8tR-n{lb"3_9c\ZxQ*]ԱEqf}.MNtsc_; eTKRo,rQWw/a1nbsP?dȳ'^0?0N~ՀFPM2}X5W˖8P|(NbK2sV}ֵQ)B#*$lDOGvVglF%LCoTϵ`ȇvD)[vk' 2VgEf&[~ʳx_WBD,8jTfɡ=!>w351v-j}1ߝbĐtӘVKAWѨR+86ިh^i,1E/̼PPn<Uqqw|D773_ i@OnV~6{C t CAk\a%UiA[z")ŔJ[Ln0a3 _61^F:<"8W{ȫU XXe$жjE+ !Y(VN"|13ңI]u.(FU[iv+VmGi)J;<Gc1,plv+( Q5 rfi|n~&4"x3#w^WLi B͜nDg5rtn7:L6%9MqEo'E *y~)Cg\2]PLv4hxwB9TdžM]+XD^x~L+ .% ~ToID>;djp<Z9&tBլU^fՇ|bpE-4ffmϘy߸NcWDJ[ B4ȂC;$p_ cLy۔֍]xѹ ،D+4(9pjM.;u,"D,R^@}g߉O#PlϨ@M:}FBM-ТvBHR'+,Kvr-_Zo>&l6v<)f_(hys [m3!G&_l(L"mK͙eN-"H n+VuA+qЏ^sU?t9ddCx^UUd%9D63h-[1]˦RZq>#,}̟͎~5#':1zW<͐7wZ /v%7Bl ):d6nGv'4HdJaU}8@fv5 unA*}'UV I$d|G=YCZgv?VL%(\9V-[[xtjf3t ⩕R>}\&`"5;L[@:"fJ d[yŴRN5OS"#9I<5O\v`kD`㧺P\.?%,[ J)pCTvrx)w22p@qH[ޏPܨךA5_TQ) zލRyʹ1L ["'OXǕ&>oN:љeqV9x+L1rrʕ̱QN9W:MjqTRK2b=/Fyt߮b*TheNִ 8=!~a N;IU4ER)$v1lëSgy$P Un3߫ }?ua," #.tEy<R{x "_-fjt,֖6Xc|Dy o^/=P NA}/!nzŶ3D!e[g U:<ӡgmPbNgFEt wg'QQ 43AAXF}'DIzDAGh]*#yCWP(UggMAzՀ2ȤYUᅤ`6{0qdN_yݦ4I0n1`}B2v%jn7puZ5F5W'S6],Tr@h(|?TA^lABWqߏ=>|tS:l}jz|}M( /Ӫ9Rҟ%g|Zݧтif~9t|[A3c[@]vB s- K-+=), Y(mzvCk{Qe;ҬwD}W)f+o>:vW3,h&U PLl%ҿ >HsUxcob99M89ӆ;e o :T苶9E7i1:tL ȭ:y`/n|_Kn"0ALNQ["O@ Q1IC事rrY<(#'E,I[{|[~ *9%ҵwCE٭F|3q,OpiBG-s"sz"EϿ!x.NԴ+ 8f)E.ળb-r.͊)mp-{o뤐Aku$ZF4)|;>sxfNi/"}q0I2rW⋄X}coiHP B =/8q&m')ey*97XHPsS=^XK wǞjl:69Ix >?w g1p3?~a5#`%}~ݵ5+.ȋZD^+iL@:\vJ7}]Kn0gglh林ߧ΃@A@޷.ZׂJlu*#,m%K`nië[x J>\k^S!?<GRKخvbF}.X:W6ܟVa,]l}OH~&!䪛F 3" xJrÐIHVz_N"c@5Ta+Onv@?A?\K%$Sme?My ĢL#*Gpǒ,O@jUU>@s0 Z1j %!_R#O ۾A& *ט-є)ĠO< Nq>u _~1L!+vOjvWTYg˄P4TB }|տW = cE*`ЌB|Worr63sLܦw64Rd({+MI Z*{zuQE@q~yiYFd6Ccs J!!_j9v )F7'{3|CUG8vzkh^!ǐ~nߏ:SۇI=DñXkPL]SRɡ _d2uNRoL(:`.bZcF6h{@9U1 tí7%J'٢ k Az$HYRwlm1. z`gM` fJBGX̺f+1mq{u [LwZJu~'"ڍC[-xI(K:5tn֨X+g8eR;|3_HfE20[=Jn>u,:X` Vyjջ0įʷ{쯔@AEsśU\}3^te|BK'qq"`YLK)f9>O3hAN N{DNw3qe uѺgBn1]ߴԏѲd'ol?$wǡ}(:s@AXC'ghZ~>fo8=f7rXv^}L8dw0FcK0)5 +Y  J˹DkDos9=Kt_{ZAnߗ,\|82Y!j4^ݛZ5m 姷\nst;Y`Q#r;hSߟwr7cQ4a# M޴I=+rjշ^FVՙh"> w }Rtz04QI>BIm..AG ?JCei^of35\πk}c+|@;-g4Ų'4<6>r&zqmTلrI!Q!K;o9mOLUDh6w{O\ g oO:^c<)M#gHbum]޲=֕σP#SGBހ1kZwCe}}_* Vckb?].ep}d6@-҂v| 7'gtsт,ݷd+{ LP#L 4 C9zq#YU%d\t9͝q{Ɵ m$gQ)8|nHdŚl?1"o C*MPp([C^#9Xq<J*Jd6ƠHȘQu,{imƬ0FGS1Ӧ Ke$G,wI0ś@J6J*YcUƙzp!F'gjo`3 ޿*黁kBEJ}h̑\cn ÿ=T D|/ft6Iw.X8k%G͗bigO2Z\x]׃C-  y ,>v :`nE*i8 [싉'oT"Qjg,i"p?7O$(t쵢c չ"88t]S"-,;e >46g[޾\1"( QTU-WA~/_LzF? Ɣ..tX/QQHQ&ٓۺFhoX^bvU ԰+2Q䲇{Mf{d5Apt71evz`?ISlEml<̤_hFSAy.sZN 3m(߀]7I鈋\QN`Yl\Zڄ?苡[, _b:/őG :kL93iX99bͻ7F[$yAAٞ§NOUBک;EDU 7oE2n(WcsS{ȀOBD]PSU6jn UEV|Fu | 9ON1W D_,cg"6|2Ok#WIk4+?Gg#z]D,ay#gj =eՀ`&׺}X*?hv~h $Ԍ?XdⳌ+X}n@nbF^h% VRǓ2|{-;\) m9"qWλ`=rѬ@ c$>}#=+T:϶]Fv+@ I(HU;cBf_@k!}/ ++)JXf},4%lkVf%'LjLT ·k0p iAm/VHW`q~НV*,ϟwǪbɷ%:C`Z^{t!HB\e,DD12rT5 qI@,ˌqV]fEU),/5ѐZ=:P#vJهޤSh'Pɫn kǩ3 ʍ[OWK<_@'*ST` Q>>;8y.mxHg̩yC߉?, } Kmc5c .h4uVռo^jwrшhz pB&LVb'JKX1Q4_ZT~A^ǠXkj>FhsK*,>f@\UMV(Cv@CO8}8DźhP?io,Zw(i4if .P- d'u@1]+6dOsXPk+.O7ZKՇ!C z تWRE%_X-q՘_C\ 2m+PI'€y~ŇAHv;?) Xb'ryC&tbI4K-_M tz&>=",Y(G!V`M (OtnVs͵P/ύvJtgK6mzi #plF5#06 *QVtۛa+ *zϕYoe\_\gn xj}]FdBWNXoCX)ӪZD#M(?(@AO]'اGn|8F%tK"*}4@u"3AI.:P@V[jNhTs7FكuAq8Du]>2tD?|EV &p?Bbe(2v=&h2V>i0 VxїeAkvvl`}ėYsM ~6:q`7UG0^uO(E :BWNumxg/EӕQz׻Fc-C#'Mhgu:_znӎs!&˴z1t#ƭu7+sCƗ9ܜ;AA-u>48<8"+fsެ-c\oՕ9g2"e 69c[VvK]4",t)͑!] H既d2ŶHsN+pӋ`NEb͕D$u.|`N\.̓ +UI(Њ€=Bx FI#p@(7ʒh:tVvx3,^2.N+NeE\(?h<ނЅ |]FU s!R4^% i%rHDpĿW `7X5lO\GM]cIBvSi ꙳DY翂Y6nw_+d7lϔ?wXAc6odW#CM{#,g71}XlPHhJ~vU`W<[Y1LUL#%]8r䅋bJ]?Ba[1Pp{W:ukN,ssSM*bd.󤃎'vW{R~8liVMd{F. Fiq!pIiN|K0*ҪZ3ړ5#+z[{qn 2󬁓uF3L1sf٭&+@U$t.GهB8EOR?|/yf0 ?r/ MVoR>y)ԵJUy9ZRƚ7~1=@x;nML\Ǻ!|=4'ԸC$&i2`gLڣht2ϱ\潳շ'I )m%ux%m7M%c"/.oŹ.w,qx(JJI~!0rU_Q0H`hik6T ҜCZjW)_è5zr dUYE ham9!hŜn)&Qho2 |$h[V9T2EDWCRVPә)ʚG _:/KBhU3gM֬>Q1UuTxas8 XBn.)Y\_r*imJ]ĨRɡ%'WuB2Qk831o3bX>uluviaSƹn1w$Z%| :T(U4i1-aY8N@rBZ-T Mm,Jl UцSjGf[2@hQ ׅnrԸZ̸Sewu6򌖎`pջ؞%c} aPf1q/ 31NRqV!Ih15${G jr¹G,M-tj}k$"c Znjd/M4;_{*'XR3H[<Tb/VriY%cq r넸B^! \mel3]kfY.5,W.,h ˿=] ,M1nJhu)6J~2@!m!bu'!8W(nxػPHvL.ZgNn GuzIT72b0qr(+|alt]gzR^$΁m?GPN*}ඝ3_Tz͈l pʨ+6]Kc4PSlWm{E"E害(޶nO9ozO*Pps>^;#w@Kv:hBW#G@kտY%~$%&TS;G3*Ƚhfy#C]X'1ڝ9.ESMTJUnV e͙855%U7}hl;"4Z -bglAaGBv,? HPq W]$6M3D'$Q ćQܿ'IwHΏ"SDRZe .;?B2 V@oN #i[H䕎Kݞ)?T1u,bVɒ{/` >w1ּ vPUfD`um0230$9"4|\(U|_/'o" yuS SjPSh?t QtRnjd3hvB 4'& uWe3.I/ ~hL[C ˡl2nB U[[ݴ?~yi<*nsv1%o6KІpyޅ<~G;:7)fz_کZM CTؓ^ _ <4Hɤ2(忯LQ@)`*\t:lD(Sˡ‚Xm$P6 mk endstream endobj 144 0 obj << /Length1 2914 /Length2 33550 /Length3 0 /Length 35165 /Filter /FlateDecode >> stream xڴuX>LHKݝ=НHwItt#!)J7>}+uk>\MYl;1 5fl@kw Zhf v2s xln\H?25@t8-e@gP1B@Gk[G =$Ebkm'3J%X f`OW{[%@Emt`G9 d[mMi M&= Hjji2U@&Z@Gk&$w4;5@Wm 5H :77'VVOOOkwW75~Z6O=" _~ @ $"%$ bw71nkp?JjjJ3[G7$` A @wuRCVf73stwK\_+[7{{fMY\E^FZSY 2x`:,n^nD'.$cs C*h) vpvE--D'77l{Gaehi[{Kw'VmG[gwCLl@7 zYذnϼ66Cu;@@[+ psqO p: S] ]3tUz9;@+dVd$9iKR1sM;- _>[W[/Ϳ] 2 d[1i>R B?_fv.A W"1/UEZU[A?qҎ`K[Gk7 2 _v`[+# prwX]o(7U_*X% >`7eAV?*qX .A(A.oAkAA ! HafoI4 )|;b7eaTT .?1lAH!_u!KYA! =3W?;@ l =aT;WAH_? adt+b BVbl_!t@v?!~2d!qs2\Z YjCN@[]\C, ;rI;v:sCV tϹk/!E\m,Ci Q BTs@zfjv[zȊ=žM%_/6?!|.b]\lRtsZB^H Q6ss2`܈;3.%$^\ysAZrj~9^@ ` PEp,Ux" gMHoQEeRKuCqAk-)?-ŶФGtXe+/UtR(}nMl%hJwr`^QVy̲7㸀1;';^pczſ3̘*;ta@ǔ>Ŗs^[>@0r@!" "&"yf)FNA'vOJiœZZ~.}ܻnO% 룑n&%) H.8Rt7vӾ(mwSR.ѢSVdi4Ԡah Vk cXLe;X}326DH:.Ħ]=_z|=<---hĺs)u dA27>IkK5m2*ȫϩ9mp252Kr meEǝ7( %F1jm /&Tyү6!5Jz{я3I?rS\%&-Ui0_T뒨iv6zݟHtJ wDqEhLd#IvSp)-o" 1AA=z7jm>Er%u̝)]sH BT<e뭛Ť ms(eEH0/Rؽ'9u߿\MNl$oS嗜@)#(2ծ쾜]lf@ :6w mJ̼|bj_w v? #@7s/K`h:N2vm8SLDvn#=y7wpM#ԐC m.e &Lz;>."Ԧ3 J߬8nzn3Aapǂ 8˿o-%HBDcz'OѰX,.] 賭/n~ Tf>n hpyi 99?KVL);#y!F=d „}_+r&~|Cz%C1vA<OԍwMRδ]34cj}pԑhɂ?=F"崯loycDFhu#B/Ks ƧLwOE`0nI2X\n0% ,>T@+ZxBPؽtP||(9@[W!ζ^\aI_@J GۢS2N`Ԙ=S_kn>g.knJCƸ FD[|JR6#A PZA ţ?.@lG;hVAG16I>Gكq+i\(kOt R2r&sG0!o7ͦ'n#*3m\ĬXj!u7wygqh\9Ӷ j yyʼS*C6Ɖ\0 #;Y,Ƶ+J-2ݥ"SP[Q/h,X|k,<Ռc qyzFUƲr2!~nG/F_l$v,\̄xEh8a"DĚ[M: as@]ei.dF{퍊Nq.MuAG/GCX=R݊پKQgWd 丱:۵ 0zc^ gIyæk6&,V@w$ɦ8ҬV$\8 8k/\f2]}`:/@W"͟'eY&^9b=r'4BQ\^.oӣ-2u s[H/9%tqh}m=չ@jZJ<*z1, ҙN7F2)4~ Ocm^>1eͲQebxZ8}m蔎W-X$Dun Qy+,g}rSFo i3OJ6{3ɞ c.6kIxƁ45 Cb)*P$sF s0gr5vpV)HG<) v%MHe;TKq2IB.i'{љ uo88d8{K OG0ph|}l(0sk5PcWbf6׭y%'b s4d3r[i0C]J(Xh|oy o&r%anaVW*ňH扗Kb^R1)[.;nO aFpӖP_8I#X"L( ` ĞWI ךn{2%^G1{Zd.k7\|LbOLo1Di+ Yz29:+>/m"Aa"$m0gP[6̋[IQu4zG{=qϋM9w>kC dqQ y r惭`[4(`wU"th Җ^~ʌD~o0ya1C47+vȘe2n^ڼ59E+b%O9 hL-K0RfspCF}3[4EFRq D۠%p`5G?DpŠ샵F a΍*%0*qpޣ~6 UG|33`}Qu?y.d,rWz&Sq\j ,ζ2M|?l xfMW,.ӖEѽ%U3éš,t'j鑛:xu$j啥DR*8\_/L9</%=Y[GF p>*N%/t[]nd1PFYՍ%Qe=3Ën"BV2'ϗ֏ؠrhj&  jkl0"~u, |rb>tZϫ~i H'_Zc$ u,io/V"q̴R9}{1:~`FEU(m,+yUZA/f`cE@LyxL+Fu$! yk,>m͛,dzFؕU~k:SRj_`ijòl8I`t ;HtRe'S&G:@%H>ʠG%++)F 2090+d{`D{^UFvx>]c}Ms>Bk _c!X<$AP0)9jZh:^?Cx e alivtX3oP %I]Y)C%(w?qHD|iX~|B:[@9}CŠ iB~8㉰0[- !9ܣ~í^)q7bf ~D{hC 6~(w>P<Ļ́MLAp,_pafDx7[ցk[g%9J䓥|$QP2Vw0GIJH2mhwAnfxUg0;։A5mwKS_Ӑej[v$nQHq'va;S-fy!OO[hU;Xj;fRtpk+v H6jr$煼"gg.aeES&6OymXfgq O3XكTKα}߻/V4tS;+E[o⻓28&h A7+fH)Hh4W^Kp_^ٌX* e*]xm$̯˞^DU#LSƂ/ޭY~hL/._ȾbɍS$֤1)RH~)Lo_;]aq\ 1n"'Gq LudFp+(z9Hc0NX4ҍpscPynD鼚ّ-<$o~09gx O_Asf*9JG`zemxc YlsrRRLjRG[H+30V:[^# >!b%i lFNAqMFw'OVU-SW Ņ۬kYtB* E;-8ܕYmr*bG89#؃D^,~PΨ}m@t#d[.%o6MF!-@uxt~.K^kuX5m#PM6(G}LMOH5u+wlՇ:1wRӟ)U]"3]%P1G5X&pd9(fdn&u=TJcjFGʒ풥`"z쬓›"g)yrD3ҕ LC. J[ k1Usi (>8; o0r"u*MIN=wtw21o##G Z1QwiA-G=7?]CX.6ƅ@蚫Dʔ9 {D 2#ƢQ $*~GU!e`8fXCnx&d%s-f|,#R=ΤJ(2G9D'0@Xnh>t`,.85X>k$tjہ|썯{J{y@*Qhv7ٶ}4|q' k4ǧ: F`u遥 H/0{NH725/^L@K)huq+:_+x?xR7^[ I !}U=~}1=65*I,'D ŝfOޓ1rN{r+7!V7[|+l v&UֻOԢjni̤"aIMf׿3I8䭊7;RSN"bP( )P[OC~[ߕFAu"Na\mOaG0h3r4j Q.w=SUFtc4ok2޺TducrzYUZx*( VHƯ#k-}OEju˽/¼NkSy䒰L /VyR _noy —3ӗJdk\}2k[^Y;7D~Ä1`TЊ"~*W)PK?*1/FtM%3~^_EӺ_2Ngh<|Mg\h|P&GVWGs˴= b0sqwՊZ] TOed3f(ZmjT쀓`=Wm _7@ha);}UYqx`ugƶ\/!j>X㺙1g~܆^uo`%_5aP(,Q:smiÐJG!/!q[uϛ;5=vV,|˫1`aZQ ',~j^g&E K(|f=l]3e/q]:x2}߀ЛJu _iS#ۃYZ*qKsIQxO|Uv>GpE> ZVg*S`Âd'j]cOv_twĘ}9zT~=*X$5A;c?$KdE{0Rx铚³j(J(AdovUO-be ^Ж@>*Ҋ4kWtrmOTvÛl+ q;wxBRplz'xJTjWA54ck:g!OR,4%}c=%r E4#]Xvl/ $5 gj _EPmyeWF(s^_psH\~5ObUɓ\Ӄ\n^v/?_ŘtlM1-EHó{!'ԉbүGϷn)"VQ LaV-Jo:gPu5i~Qǚ!OFz$d" cg`v}wb0wSqmɂ+ru˂1O_z a޽RO޼6Y'^i8&\7oJ*~{rIC?N6VC2tmbK{v,1󍔼D=E9Ov,2;`eS *|ٲJS~JoEG$(gMӟJo ޣ]grJ'e}{􀎂~ix`i/?Qv~) K]ӑ$.a!,ڨbNaS/]DK5/8UInoIsH]q^V?mN&$f:P|,k+hvLqeMnި.(ohX15( '׮+Ӡo fTlD;ٮn8j%d0|e =UgMV(86~2 ;HzT}[: Jp#M.w#  1PDo"ƶ,<ɽG 7$ijN.t0 B٥+?kGG.o4E{oɾӣ`ޙDd0|f0XA'{Gb1pXz22l,k ˄NY'BWȈpA{60ZjkMA@ U~!={c9GYFep\xpZ˴rxwd- Ff<,R,'3I̊>Ѯ+e"rQYD,Hʩ[@Y:HixW~ bS:ŸwLϡ>X)jctXҕdI+|{S1伮42+rKdx)ވv.r練^%<}[7qV; S_0Pd>c]2i|!w"TUE4%ۯƍ؞6grl=CԬJ#ry)NN':9b HO#}8jH5"<&myKX< e $danxdBALi32E v8輒)t- 3Y|<\Í,|`!0o*n `OsXy^Q<^^-^JVvxh?ӾOSu Qkh+Lbe0YR+(?uR n@4"hQAVb,_^ U-؎)vf#PAĬpH]}ٶAUoa*3vr▭A T],n >3z}+Lu)Aܒ70GrDZz~ Y|B]2yӘ{ :Y `sx20ONS%L{./r'q/ B_v7Pb{ּ^?{V2 ;lKGJ90NyR*f'{ǔsb Ъ^J8: ư)܉"T =8`ǦCM{yM g/) *Rr ״ ԉʶ [cu!8Z6jmz)1p i;H% 7yxQKa#=-P~.( QaS+]B'3T/"z&SI捾¸Z,Ý ܁7V21SKO}  ER_`h-N/y/JO%ho幇vTYsBN5YlYw0_/OD.uDר A:'l ֫Z_*VSvi@ >1~r툡QͭPe95#z#۶ /CMCoz(q^Mld`( pWÎYVk F\G0d4*#{_3 Q5-btePcc7\0už8p֢CQmΫX9r0z7BR$!|_?87(1v U5\[(40Qib/Le|ͽNE|$e$4)&R[1/?teTrc|+t-׵Ks0p 6& :-d*y O̍1k8ޞTuNk|»5dHcϺV)#=!ѓrlĖ$K袶bdqӬVʮ@wR2Z&6I9s4Bjt/4fGI(cՄ*%(oȞÞMp_.1Vylqm;KKboǍf7 w8H>IEpl(8BvoXzq$l3b/OT-WZ"!TSv4xCƣT'/L!T5]ꀦ&g'ޔ m1xm#a42O}}wU-H'_ἲ1r1_b}QR|UNG?@3eՏKj}'Z) DmeVtvL)Pfq}a!1 ׋~I%M`܆|k;1E$w5vH\8cB"Y.=ksYTڴxotSc$I·  %? H KQL+XGTzzJ)"ᨍF j(n=4Ax0,WR>'SAOgp;K胭БpG5׌, I6e<$W.oIإ֗Iɻ&4I+e\ڜ2^ ȉ3ȢYS{Zts|m6Bu_Si7UA[Pb~a"?._5Đw%j]hJ/|I[EƠd}dg`JЙTM4I #c)k9gjzul㩯.o`{?}EkTQW\kW#pitiHJ;J7+ߖ<:+!w'xf=I(Y˪@r~P1܄4몚 \wHFw$䌀@"Gpo6΀_Wڮ&}>Wf`^'=5x>pK=SatWuWe2fnJwEHZ8V&rh;y)`\T(T=xIvM<"ԗXscʪ'y=/bIvV߸Q}lY+lԬk#m>B+rT&O\hpS9˞|hf:^hC1`7f|u }TޔĤA>D*`݃h:f0)I6)9h:hz'H(ܝHܘw~W(;'D#TU-UAiuǴwLZڢ2PDSYs4Hy^{F]sBtMs/u5loX}{ۿuńѠ<w|#N&p NIiEx/d.N~"lM (OMk uBu7uݠWOҶC#b>x˫Hqf)&njLо}7gAp+ȭ24$7Jd&>{az|{u)CԮ7}rYh/)";tHLUX07',?_VI0\;²` AiT+%x#MQ]<]M(lAViSokHkVc%gAIFK)p3Wi?t9;;P@Iz":N7RrvtzI'oPe K>+;GDq8b40d؄/ ?)V {+NE)˕ p ,Rԃ0?krÖ)'Z^WA.MP 5J5/ ËRM|~ujs;@:\+!X<a L:8tП8bdB9X@$.?׮9jKŽ j L6kuMxGH6Aԭ(`y ٰ꺳MQ)lZ3ZH,1Iv 'rǠC |'k=X!|u|q[(ڲ,۶m۶m۶ʶm۶mvٯu`vf#G[JZ+II-L{ Zgf+w@d:.#W7>d2 z1v׌6lxdm:3Q 3[ Q,Uܼ Ƈ<Dի:_$~7M)+ٍ f9 , rp9=zjgw.bkf6~N=|P!JGfMP~B'V|8TSlwhftlD\i9L '漃ΒP$,/;a_TiS Ŧivs{d?;}T*N겊xr#j6h]aQI45hʕvMsbu]# gjtd!܇mzO3=( _甌 Z.BFK=U(YbD#~e#"?4zS\CJjGZ*:MJlvA͋zn|ZOJ@&zќ4JmT㕶o' $VYIp-ne[^C,iy9Ә;fCÐUnUhƋ[$Z ' \VKuhNԀ(skfZ/j1gm9^.2VQ)mVa oػy\,8i[\n~v4lі\X7bgNkP,ꕾm'[nLE_ܖ#tE~ô<s7\_;"R d BBMu'cG5)Ic^BN`ʳm~n@4 Lˢg MyZ-alc WJ觔6Z!W͞H&zD%%:Xk8mmahWWA]>߁D]w\ 4'RZŞS0aGeB9[$,{b㵆fS ϓI`߄ Ss"'v9ecՀ2&")پ,:)hp5Y# qȹX ѯQpp€̰SI'6^=bæi H=y#|4|sEZv5[\*eHHbqdǿIs֝gy2o[2<{lpO3 @@=⚞ܨV2.V oVYb3F2V#f{ _2w|/k6-D~3C63s׮2R05-NѲٮ)иZ E!:7RfEU )b^8K+*̊Xj#世}<#I#;J\x0X\3 ;(w[wƲkl쵃ДZdjhF!tgCq]yo;6['v}(I`,tiCAhsBB@6KdanX>pe̸^+pzuC]#1۝Miqcbu >Fׄ^#25Wc5r?o:Ƭ%*"7бDG6`⮤s6*,͐t@>[ BhbsRk6!,V[NP zM\2=eF*]\,L#rtn3Mw@Փ[2Y$ 931*ݦ*b$R8Xr"Ksr# 3E} x(eR?Eol8kSjO;]=Em^[2?w3Yov։l2|$-Lp&5r;i=@7L*"402yqtؙB:GTj+rU`af)2 x[.ݟ%vo8bu?iR*w%6:&]kQ_CVB5KXx"}\.Mcv{;v dpyө4^X,`8&&]Y"E&R:HM/0\/ 6̇<'B<93>Cb @'<~Ȃ0K۶<|ĵ%>S?>k@ L8ϨUCrߓ)ɺxmR-HOnx=:~@г]+'Ԁ{&}0vԙS)T?Qaz\*EYYu79oxpk;͑"m`M) q~u}{KQi;*W3\m\qO\#0ƭ*dK+zu|=}zR[lS2u/H v;A\`O=wZ57EWB&&xldǪ40oxh2~7ӵ6}CZUb}̎ϕ)i$^g,*WC1/ȇZi:w lp =oC=҈ lܤ2p?m@ H; <_4h>ӞᨈvR߫nq Hu=FNv9h FG$QЂܜ@(鴖vEjDo4RV\c!x 0`4l$ 6\2:P ^~:<us' #*Fx̠k jհ:ũ8[1 }Bȸ Ixުj`EZ/nL†F`y[uM4uRFaPD1>mH9V_|D?{{ Ħ}F t\*l`vGcTehW#Jq֛8 2`%i{"ٟ,M cٵWi62%YU*)k)anku}(0 . qb8 lwZ7*|\0g3nYTW'ti}oz|C1n3WP#@ӞIiCѦ%n-A%N84{- @;Hqpx'7m}@mM|K& t }`%>%w7Le#{ FK9 !Er!B{/&^.z) mLf2݃7[- t< BxoqmbK=_M&HJ]N ZH'';$g.;ZQ(@?B6^؎ wIé)tg.}' ((;`2W}S }b!O]x&$E.C+mq{^޴5oh씯?9| `s +%A4aeIMfR1FfZ N6d"Yڙ5Z`gp NՎ71 *U 7C:R5}ܶ'OnB6 y1_eE8䲩7BElen\J4Ԑ@nKP@إ A%I⯿f.WO>vTN`iɧ̀SP7t$HqZ?C֐:?L{˕2BGFrhv̀>:BWz%c7ÏcJD8oXTYsfn ]AA#Ow27 ;#32.o@%DݍG&`L,r?K80"]'P%hcJ! `GTfr)dJh9,rXY׸C/v^ub6S J?}3б?pë3)o92r5LH98Par=l胙."A"#0T f̼϶iONFB/3412G^צ_jtt0sǭD#&ظ O3 R\[T*" 5 ;eh5Jgͷg?(’rnN>5kHsh9Fny.i\/*D#zP w#.ev ޝ76}#!h9c{7z |+0#H9 HaKVVx<J *HٖCj}N.aO<1ES"[*]"=g/w [VS9kO yb94H}3Mp M;P>#;'_-Sk.4/yYű;M qǿx+" j?hn=j(#r9>v&wYAy:)]RJ{Dk[E>86| L2E-# kFy7ؖ1f@pg]Ka0MY|HX|.5mj(g8n`Fc^DIQW-Xvd(C16jۊ~vq ƍ&z BE5ӁS~moQp_L+1}qv=&-+gO/1w*UY[%jrD윿4B;Ѻؙ*̼_"x1WoO8h)UK.p> *~0[l {Gw_QD< \cc`7>_r0{vPcQVQ<_IoQ%?,BPe]y!k(|`gP!4iDn܊ _;a 9 WpD_pȚ'O; 1ـ;hӽNK$\,ķr{qdY 6%n YaZgvv">:r"|E88[ zH!"I{!bcUYCEv֌lٲ-O;R4L}vݰ:{<ܫ4ǝ1@hZ:5>qÔT׈cxhشl(M;Vpg?E]@yǧiJ/x֪qSA6' W2d}YB}K^c—xKNc?_?n hQ7g43IPG/%2c~/ HERׇu&A t& 1`<:K(FLe:&A,3Di]F- *H'%;t 9hZo~}< zƚ{c%p5:,]@{}uo1M?fFڢ s7fS*R@V0) Gt<|{WiF5BDPj 5SHḃh#AyR5YBNؘ݉E$!-&WʫSC̟ā^2X\*Xft# Hlئi8# lSh5Ui'$Fh0D\1'ŝdŋiN\TY0n/(+ih, -CjiX݇Z:ewX`"©tDPSfj]T?.65!y[(ձc5t !/U_zjD\ BcV{^]LA\.~~2j>@u FyʒV>o^yRO bNI|e\W _9 g7y>tviM\gdgz3W?@ی9i09ɧw_ըb~vٚ%$ S3 QܸvxlmGC%B^W7peeċm/ .LBbyrHGRg:KyA__AF@ǂȉA=Ȟb*954PcJ{-4jԛGZuJ~!3M8Od ?.Z8.Ɏ/Ov,HGDCީF7G\./i3Oh[D'߰<|Q߆ݱW$F]9Bm^\UᬪA̔y9|R-z}6Kj~!kt?+!1[vzEO_#BF܈ڊ+g%a~|\*YԂ|t]*i$m3Ifi$V5GΎJlŝO;&ț%2}P\b.*`bASe&j&yIW ڤiF!y Pb,H*jx)TxZ# މ=$O4H}0lSlY#Eᱟb2(3mGLS6ek]Jr}Y,ZGm<F&^Z"<)߅VQ4bI mZ[H ԴtgE)Mx;yQK^d]8ܰZBŜz]7zUca0ŋ~lPZ?yI#5{6PS;j܏ख़Viw[ZPA?/4%P$hVyx0Cc9g!Pˈ2VSп(M+\DJnuOuP JcS޴;6ߏԄ=_e7@QkuPgO䣚}t \íQ c]ʧ ,si`=0U\RVǂ90[AJU1D kͯ3u,bk{wyIiTˆWp/@ ڮkwjgFAچzRoN[A BpjtP.a2cp#S\ĚQ0gnh2Zቄ#Kv9vhEESѪ8#L L4H( <}}ESTlpe?ěC<8ʟ>_8iV7͓6KTtCD6ÂWD{CUOMVSjLK!pԚn b]z#D~2츐 ǚ4呙-_X%D y D Ezɲ R B<@; 7-QkɩCx3W`5QKUI!mYeu\p>spZvV=j1ńq=`Pa\l] ݔgnK=-jade 10ʕIku5pI=z| Bh~x^OK/IpP#CC: S׊}Y(>_<K[a?tc Ρeb!VE')Uv&gygǡ-h;naVCA,= Nv5WX]4&c AʮQ9Ns)\Jΐ$d>͗:Λ.M}XH1ܐ(y53b* u @W6aEzrs?xFa p\Û+q'W/: u6sb{iȁGZy)e=[`~:L8!HFy!3^l֋ A,NML^d}yVDĘq]VC#S쇦A>NA TF)OtY2GUsy|,R4z3pjC7k#n:1$DM>X%zG`"Aϛtf*x 1'95@AVnI9&#uT8;cac;IP/fH%GtS2Qٶbd#b=߹+K듕 #4m͖(YY/mG`ozrvl/Pqq^XJaҏ&oR9U$T-i6PS1y$OvVīj^ K̩SQOX=žVSj<ƮΑz{0A8MN[=į5g#+^" +&s"sm+Hgϋ: $ 據,>n CPppmVHFźQ qX%:.m(3mbUdKv#S2IzF"~Uڇ L rQ$ұ6W0"vp*Tcߎ*!tJw-nMq1 !id Pp*"0GR)c } Tn1)~Go۸¸Jf&G&+|!Y2ؾ]]G?Q:kXWBV~҃%2pپ讷GF}tƩI6Ǜ~TrwZY7 6FX)l> *%abI?8q)=;c:{;[L*ZVl ݺ@ɄsF4NtѶ4Qn^.NgvQ2KL%0ad.f>ԏXZ6AI5#nerws¦шֶn,EfnnJÎ,>o}7wX6"LFB&v3XHdpjѬVuf1ڶk߹DXeᵽSVYXg%2 xOKZƫ'N$)Su\z~&?DFfh;n2?<7hQ% (vSaWg}w9A&>Acx>r^' [Ovr\Lnhڟ|u\Ԇ8L(ƸV"Xd_UEE'n0ŗ쫨=|Ȅ8Tn`esD4=S/*[cH+)c!OЙ3!xXz`ج=:6"̵3G]wqa.R168%{FCnF2Ygĥ'-&t V4YԽ6WTR6X*]mQ6J%p0_sWTN :$ns/ 6F1k>[Tn|"O# :.2]BA(0k$-Ov(:?杖}HJĸ24cC_)tGyB#!ʨq\tgN4T?^;ۜpÄqy$WI}@ŭq UlDzL~ng{[L7vxt ,W͠KID[.(FM㮽ز„Cp&9d%&nLJKľѹDPV"@e $#ߛ}z7gav*<$W{k4:FݺH+.:9ȸca 4FE'=8_ sI"#:s*cKRpUۄj!|R)ݍ>$]J%Z,bx;+[X瓃)ѓ5pL>s71fYXh@=F381 m4S Y>/_2hGP:<"@vZx`hK7rYP6/8Up iPGFn|Hn|0 ]$Gq;F%h>|0L7T{՚3m вT:mm)Θ cR[):H  +/ wjB+ˁoBvq-Z#RauDB#d*WYiqmAC1^|Q7&,f0=d$.SQ.|>ePH߯}03@qT)tƖ01. bry_2!":*ĀJaZ|r,k+jWa&=eBM56Ѐ+QǍuH(9?f$%zVJlAQ=뚀"jѤ43=Ƞ0e[8!ɖf 9]45т;2lYD)ZlEW(o^&<*@]9e$}9-_&Ҝ'd RrN]N&b;"U\5C-jR8v*RIh)vh3 |#Ud }NV7JsHl!hI"ZJesGGrN{U$.G!H-퓍 5BB Ϟ rP4CGBw5%nCibMtoVp.z ,(,iGl cۍk%POgxTʯulxb52 @#߹.;Bgk L?ɨ>z)vYl@h^iUʷad%aARJ0[p ?HwَB_!ى=̠ٯ1PıԲ:]Tճ{(.3~ͅ!¯<]%-L"D +#L'}}h1FoI%ѐhԯqlyCIҸ I2L}Ṅ0x3U,7+'f]/uvߡJ/iwy2BD4__'R/e@>ϑ"P5xZyQm<Iܝ)ID=-XԑsHb}jчaͣ*Qz2rp|qMjH`Bti3''=Hoقg))lINlz;X\x!tRrgUn dI\R]GbZ:}wEjٹ;P+\ ""kwkdXbr}>ʪ"R;J-,^#ŹbYVIwx2$I<4ԅf6m?x_vɨ G#8{ гC֛7& ٽ#-B*ۻ|3P5mxZӭogV$=$Ih CBu]*+-3g# xhA)fkw## kr Rˡf gfYPEˈL*%úga%}`Aјc:qL \9A6wiɀj((8q,nַTټbPJbV tDh!˫&59co9?[V征ض8sp5Gw'b/JJ&UQ v(“b(ֺ+"\J&V)ąq,WMf%p\6rJe[yPZ6a27h'_6l?@F4QMs!wGkHn+h/F;RYT74)9hphvn&,U04۴ 0{_M]6nnfCB,_ s]>UBɠ[,d~zC7LX7Ÿ\S9aڧRlIQM8>{׽@ŏ‰^?|<BÕӁmU$"l^f.D1Vݏ* dyɵ&ޘ\˥9+n5M+Z8YO8c˼ MCPvS^77Yiőhn+r$F|rFǫ6i2EqִFA>0qDDY-uHsl1ėҲ4G"zI3!%Ix/1SD6sjZ!k55v*]6C+T^tZP| nQ"?3<(`o[r'l1>B| aN"H6B?&&ie"wŚigg"HM9OrDY+k79b!W7!NHE{yơTH͆(zF[P'ȸh蒅:oj7FH4u'ë/Ӫ /I9Z%q }J{7p伊gahX8ۼ0ʖ *b[~Nx6^%ϓj}l؜Á-gz0u/gjbGf_ZWh]6|@QZJFuZ$lP|U]( )UV{?95[i0J0Yn|6_ǿoZoДp* ;jt~r?,Qzeb|/grh).4'_Lb9:M92(ca;e0pF顥T1!t<@0(]LǧF E嫏'D]b.h R{ ֏QMһ.jY̕@؉xp1"H/v-WN~њ繚>!ݭZmve ұɱYDPULOLѹjLQB{V};QfR@Ȝza܊ف@ypյ5P럮P;{CҩE3ׁ|Hf UERQ W ?=Ҡm:0!kK_DMF޳xl{VfeYz6ʯԚ+-Mo[.kp+ %#@#ox(Ln҅53i,4JI2 {!̢c6[Zt/ g0.3m$Wn2+ E|-3|D8GቫYl4oԜ2D65D %tee̻<2'y坨˒uq11B.p0J>PjmF NJ9Q {t3WS8 d+f%~f_|Z;"Fd.OأEbA?a'&ig4|ڣqӥW!!TP2~UèmA+$vV0v1yIJ}'o.x;hZ& :'g[eKd|`mԘS/)_alK,S:VWaXWmZLm EeY1+\ [R1tf +Q^;k˥|$xg[_%»"ǻ|gU?d M*X)X"wCDqۧ0ˡ ܇֗HQ dUL,'%Z<@ yVmG'0oV #IJvӕ̧rHx`%Zт>fq3-&ʤr 5q'.81ܰgt!9.sb9>^] ,IwĩŅT9*Es<1AyQ1kpDGa0Nk T/_ն B:d֛ֈuQ [)(19dVC9&d9֞(nfgə}r(|n& Π>´IE7g՚:C#a("׳7e^YtB:yϲ+3Tz!2$%#i> stream xڴeT][5 nٸawwww%4;!#s 6}h_c9 rbEzaSޙ + 3gfW6p5r001:9[ŌxΖ&&nxrg3{3w) gl` 2 (AFw{(O Vz?d0Lln +)@A t{7Zc3K#[sj PSWV|VVPSTfx/t.*jb3u:g5??U[UIWVRgf fOF 7Ts'_ T?5mGK#_;#+{g3{#{@g#g/)D]_ӿ/uzٻ<1|l= WE3?gfeMNX^JB\E^]xr38;,`z(5Ĭ t`¶{8̭MŁQLJM,̜L3G%㟆叙}>^@- djpvr1?<3']R@L?"RSSQ. np73?3@#;+[  ?lNvFIX*Z9XkK9_X2R}?V|47ll_ߧ/QUUFIY8q{`dvvM ppq('Q/`F\F7Q߈7z7b0q47x4ro}P&7df&f? ?{?;+@?;w&vCf&!ٿ?＀O hwrN@w.]ۻ Y޹y=_Sv_˅oA/1Ӱ2}1#D]}30߿7h@RG݋@c5&zOo?K`fnf4 Nk +^4UMpR%)9Ձ-Eb&XEQ M /$ƴ}hM1U6CSgP ̒[/">+6ՖFP=~axCJ%-o[+v+cnpEs_B[w~{Ĉ6^5, ۍ_o&4G+Lk5b$5Wo)1I#,*ÕUT+h&p1Uƅ(o7яFŠG:Q7wI}g KψFHtFyS*lH8CX%57qeIl MlE~ ɝ2 ܓ1KԈ03GTP)K}!B䘦S` !FeZ~&Z"g5ِL}-l^%'BQbkV`g9N, lFqSH H/}YӚW};m~%mE:d#qص*nͷfrO- ̻xA+Tʣfjdcc. :Q/=+@MNЊYt )r; Qֳ'/הEuUooZɇ;GF|C*$124O"CcjmrҁgGZ- j&T75Gl D".KNWi3d0\LencU) J]DGuOplC,gf4&MIޤ=aIKJG//Pp}5'HGo3Q)C}%73VSH= 8TQ`~/آR0O!+qs!~d@5󝩯?. _އ}2^Yp"Ts:w z< ~Y$ݧ>P`7q>V;X Q]>b@0Jp#ܛt?'OPT+X_X{BB2,A#*Vnwu#؃`ĂȰ7D |iy6bdZp۲qx2s`Z ;A]|m]/:Ƭ,.QzVx9W[-A[79aO@2>R1tHd4m$o;,/}.,#̕QD$06SH&v3:Ţj!:h &% ,A,섋{bL _dM?8қ k E %֫XөsHE O_"`Bә`Ә~H5_"-SIZ Do`8%^ycAn-!^'eaq ·|ݲOyL;MFe.7L[%/d7 W3Hu-`NQꠝGK8c)s*A=+NlZS]u;Iy5FPIj>8xbsbJrT1#Es҃z1(%4; 㓡2'QɊ7pDKUf{PK2 cKB.L("I ve\mpm2N ,/j!Oa{csȈf<:>L2BC,}n`\L"{d<;am(3Nj w-aJ["| /v/F__0#?wFl9MR\JYbc 8< 0KKNXk^؅!G=EVo\0BK|tbU2U2 GLzs|dӋmr:C'>Nqo1@wQ| !YNlt JHƙYzѥS@ZcG:+ZuxUul2\GfU_x򄭎t\u(cN],~vBxECg[(QYi?/A鉢aI^ p0S}a8h'Lƛ?\$+u$In0tC, ',M>}6#KK4V{t.RɇJ!jޢqɿ{@I\ YƋ؟e=*ׯ8wX'>Kdg{[ :S^t+՞2]WZkO?? Τ4KJ- #jO+pD{?=UjlukM_S ȶa P;Q0nϯ mn>QJYwW "%/ݸE\Ў{PEs)7QO e!N{d7 $MlUycM`!?NҢ<{Hy`&+[^,pJ${Ul˵(n9doN.M澾% s6J!w. ܦO'ː)lB֏BےϋH֍c0srS|{'1} -S;d)ͥ#Õ|AٜV`gW$Y "zĬrQxAO\~ǻ zME֒uB(yb\_K -AQL~Yg~AE#RUY(#v-*X[#uBCLQ;(/,/BEV#QA%<9ͮ Lª[q1So$ vXO+Ƶ|f/ocM2A/6ĥ=8yme,>vLLոo|غ$l@jo 6\dT"Le׬qFͼW«sމ}gRenq1<ɐRsx{\:~n]e`|2E MeRho;غFlH!J<}K \a:/8Dwra>PeQ"J~|&/UtR)ۺ!{M|ύ7>6VP*` e^&ϷU*y.ɠ`z~'8 A;̈́zHq$[9|-ǁXiJ+w)Y?^؊.&tc:AAUiC Nef+|p:5$TS$\ jl]K~;VM+MKgU'}j?X{BscưC./GChc0bdvb`543gS-{ E G/_R{J6I|4~Q1 U/=JHxVi:^C2}@U@AO'#պ:(]omZ6y3.8* aI!" qaIv5]'NI'8DVt,X7!Fˌ5=.M?9XrwRq4[N#H:'~ m5TԸpo1bؒ9+qK kK=.!ٹ°!d fCVzBK^5UKV=9CRO͇g l|Yb#R%H PQP饌M8,q½& `j~cP_Xj+=eDH`A"wqlѮ_D ㋸)=h t$ʽbͳ;Z§ ;`>WlK6./rIdeHOG^z{y˥NI18xKoA@|a5U@iulFOo %2Qt]t(i7.xS,C'ܕڼ W!. Ey ƴi;+- @Z3Ðx/ΚhVd2+l"p$GW[~$Sɇ($wE| J~Cvd"wQ+u|lD6K%^#Uzl"?fB#?n̑QL&I2]y(.kQDU-!Hʢdnof8 RHg[RKINe!dn<%@kDD\ LL]6TJh2X[ ;!bL]aOͤq e#NXgm]钝[ڶd^ɣ.ZeAsʪk\W8iO,Kd9_*yYm^3E$J =!RYc`)~Q2#֩+|wkj۴#H_2\:ŵSp'Zo_]7Y70 5(ҝ'DLn}e;4(2NJr9hHcfEHא-D;o<%HNOK$-&9bTܹσq rXkcw* qX\~k<' gV)ijX(sqvGVCq{B pU`C}2j_fF\sp}.;THQ}n(KXM4E#v>;'w!/$jd(5PeY-`-6r=v{Jpcb?'tNe)/P FM{M@;Ios)H.^#M7;MMlp5rMu2.QkVP7 -bnS-%r@~](LGk۫.2]8 4͙(3y};gU[Y^fuT@<2.Υ)Zm$M}2mH<cʙw#C\F>׽d8O/ @цC N[0ITPHN/IK]c/%Г}?x 6bx ^:Ӧ|nK؊# Eqs!OqS+h7"`>Ѕm7-{xvz.Bc7} hEWy̕6D_N1x)RFq_{I zH@dG %~N'Zfs J>n.Fb4_@O-K"dɘnXGZۘ!SLi9, >|W 鰛z<4#,,tM+Hi#,ڨ?1>}Kʆd +cM~Fx\[%q0zts bt |1x)9 a9Jl?vē>_ݝgGLaV oCyzvX;MP47R]#$3Sl{HNEP<,}GL>ݟgM}P dz 6 =qgOa R}Yǿn^jf9*蹌?圶[Rr+ 19¸}B na2P .j`wL+A !qBp?[ƈWق 05e4,+N_-F#( Kܘ{wƂtth #7{܃ SXnݞ-{;4u.nJ3;3cXb-7+mV0ԉ{N?I߂cB(NB-Tĉ\.L?y񌙮V_T$ `L-E/|9N2KO?f]2*g@-v:KIsh(va_1i"iMiy>QI6ޓ @Wӹilwb,)]i|?Tg|M*TٻS|g e}k5VLYjw ~$e@Yw2j} Ի VZKSpF{IE-vZAo uk1DM}a<Ϗ-]YYU0F6 @WӸ[ҏcze'XB{ <L:9m`\6ъ,(FYu=NЃ'*sQ s4|%YpG_ EuyڃN,RcPAwPgl4/8Io rb[Z}ݥ~xduIjy:W+sNj` %s} ;Q2S- ;R' 1x?ܯDxHJ2> mJ&QkZ8-%E=SbL{&K_xf, d C7y`S2^FXhsMɉpWBiL [DR4}sR->]~ st3O.9j3-OxO* m8Z9䂡oDۧx 4q1IJJߍ_b4BkVCd Y Ƃ`7K75u3L \$'6^rjj5 (;WKUqn1Cą=YBcu0b]O;*>O~7ۙqF'ʧ׹U2^mCLSZ3rA64=exczҤV%Uu#7KQ|G5{K4j[(ᓂڄ5?~i蕟]ƼeNA@g3)as= !υ`[/u5}` pt)Kag ol_\S5AC]v&PpT80<>+pLw RC F(J1V{<( *;~pd(B%k7ܑ^}8gn. I7QK'3gr8xC8dk3iS(%\TEU^U bN;씖Tٱ2j-fƘξB`NxQ=\WĢ0 %|(^LcC+ž.L20Ms''XB.b!cV` ioӏ?(h=w>55CZRv'EJ!,b<,mjFMnYeB"5TUm>РƐ]:ACӱ$$pN@xlCoHQs"pHwe̡a;,\&BV`B7Ps75LV~?i :QӒַ+R =pC">tY1onjσUC_< ve{2Ű9 ׳f6X4X8jqx4;ʨRUOi7iG!SY av&SˍeUeC֕%qA;Vp#LnX|LHH|ixl)aQ ]Av;sY. '#,rN/I]n_(u|VRVD깝#¢JR oy3 AK*ei&UI/D#x(,.:vҨp׽lNJ[{av1dfFO EH̳Ay_RΨɱYj* ZRI9~ 4%T: b2} HIʿ!O4W~2c'p!K`{fG}=";|چ 7kyеܜX!43x+\hݡv~Zm7:wOprSuy@6OE-O?y}ͷ[}}Ņ.E:9( |qx3z!OPIç" ^Qiݱ$Y"|DWt$8r`=;5#Dأ>V'hcu--ol#q p`99刡 |.ojOED)WmHk!23gJ>mr`)\T1{[\Vjy ζ'a+CU#VNY!oCrtyA~J\݉se| lbg ZFQޔP+^hlA*Ն_;N`7_n +~ s%XOcڀiRlL4OLPM =<\1A܀𚸴~ ZE9?O| ,ró$ 6':apdTW[9t=h*р9*v;A,Pc\MLwu_F`u*rF=̒=LљR5lL9k\i *(JRX dl¾;YL+H@TlGJ{bd[j$ǙxX(D!<^\q[SŲE.2DN\ kjfYcHdH5PM"GE,Nn ˣA: T1\GsOv; pؿ%FrIDb S(^pTܷI5FB.!OϾrOH`q0OW7[C#h[Up8B|5yIw ?L"H+d] t̃A velqFbseujB"&F3)uvudTcީ~9At5h矗W}R~p1.PT -YD)0.9]n\&!6d&M\b`Tu,H8a ,7a~/rjKG[k?f&x>b9שWp XJ~\6܇/1{"rB3E2t=wɔshSGQfd[:!3dSصȝO ]=twy)"q%Q+æh6D4!лs0g?:%5mGd%Y#m*SRt'][k$"3͚JaނqDZj_8. *l..P* \i⣡} 絠b /*\P \mkʩxu?=}9̸5凩AH}YGh+91T3MwE9НGއ82clJ˦5yy<4VM PKZBba˜XfT6a ūNJ}P5o./ӈMjMHmmO׺ɿfo:]|~f+=Th>`"n ~Sq t2sE~h۪Su"Q)*qNY(Udߘgiy,ŋE薐=@ s`_;8ng.ts~ ~76v4=۔# r? fş$X}e|'L1IXo.1>tkj@ƷqxMx7|<&RBkkV+eClYf Rl7u-:s^JVČU<@6yX{&#ڊ"t]ɾKyKy>;Ӛ=lG.N iR_ڑ &kƗT8߾mf]nup4|5*le@BtC^v$͚B[wc;T[{OaL/]¥T`4o%Aڕ_a0_z {N/O:%;a-sK(%P[A11X#V*rj.@ AXgA42٧GZǻzMZ܈}TU3`_/f| xWegn B;c|$F R)sy|T+fX/6]fa =2:,tꬺc>=@`ESRj>2`С_>(G1_{Cs$-x9(C}>WZV>,ys -8TȓsDR=ߠ5MraQp&:<1#5淣E[Srϛ :.7l9'|uINH")~2@fq-yN5폽ϲL?2 3={X7~Ol 1[,a(u5 (1 ,뉓ÿB|g7\a e%,C,@#BJIH`S2h[m_APZXtU\'@b#f@)]!vSߌneѴ%w`m*LbQpa6 eg$pcFU _%}64&?2UPnK!\ 3'1vTO"&F]e ^`vAQEXi"AB KUu)K'm;SJ*C2o^Ҽ.whUbpIa0W3W texV 2q`H(z0đf^aax\3NƐm(@I%~ׅ:2P^9U: Q,Pa'^c'l,K!,FozHʤFh~{|TL뮥Z˂qt8FBf@Ae92K M9@GU?R μgn$|xք. 5ljň恐O0fmy,ӱ&W;T=0G9 t=WW)9-[lUM/ev=ɍ[A!؊g42|F43Dd+ڗ[/1'Ia6Qed 21@i&'_5l`<mx_/oD;؍ÐԻ^IᓭH$gv1nOA/ ~bT(L6X;3ܫ#+d l{}U>. ǫս& VS -mqLon~^ &׳o4\nL4E 0c7ط5>..\=;eqLuWTICPc|5_N BG^^#x1Mu̝盹Iv&lCK !4pdeFm1FǤg pXH-أP>ZM)Ɉ|zpO NmA\g(i/Srb)xkaYB3 _ 41*VE WVgO%.2QrkfXւu"֥[sW6J_m 6hERqec GV&J{4 y㖀uj`Xؠ8Oh |䵝)Iص4ŔH я6RfȰ I$bB4?"mA,0!_nhof̌i3QR-y8W /۫ͯ4:6t<.;tosmw")sAKٛg_.(xM!s{{ f> |ܹmx+\uffj[ґjqacqM7aΒnuP?/<ڇkmrtO=QؚӱJ qJXɲ;VLZiPެ"Fb'nn~Fʝw32 cnqazUvZ"N#UJ␴4Α奁~P= |gOr* 7H>cOr閝rW cn*I5WI<RU|r됨5QZ ϫn.2Q;0HIU5ut{11IO)%v%cr'ǞV 9jtjvLN[MJ <0Br(co[N&/_cӴG_,"M&6$ȰY * z횲¾[p'$~=8:r:/Hiר[?ϮQ W7rΊ( "[ѲfE]߭nO B'{jW[:!o\%U$b^/)6/c$.:ؼ9ҫtWctE$Ռxe N5[DS{i8s FpIy#>,#tor0*m@[bO.e4/Sc׹^Lݢ^X7MƭJ%h/UcB)VUP\Oh5ODEɕv<#~̤c6%ۓ*B?&-k?+Bݙ@Ng)mP]@,XaT,4A! Wn4Mb;d[Ba:` ycE -f| 8Ӛmg gKMCG>B'.xrg-q"%7o%B ] N Qb} sw%˯Gt?zhn]U@0c%!{o[ h!;+ ~- ǬOA"2I@ܖ 9TKMႉjlul}M/-dO>AsMEo-_8%֜LBVj!ρ;?樈XH*s ̳.MdpGqK.VLMeKes\NX-'`Ӂ}"i2ۚ yggbP=ৎlx8`ӵ(] b I$_qbz=L}χvdFǀ`=~`@μ!QEymNx5TQR=i&`t(a0gZҧC%FI֏Hb#a5 &C^lZ$V ֽT#0@ 4Dk$3cKNet\>9Ms0^r,r|9$X"ϱ$nzGA_S`FvPEEo=W~%V!N値/5MaO`e\U)"H3?V.RbDH|$&9ؿ8#{˱Yj M 1c1#_T 'Ќ[ 样zKbXeDG'*W݇ȉ7CR$8( WzIR%gqš.NM<9lȰ.6~r ="=DiU7_UI>hb:m=lb83ۅ>,omq;FowF7ߎ9GN:8-FY()լ rg̭Ɔ2FY=ypqF9 ಷEXhp#Ku3x=P@Ajj:/&էwxLΠ"#AO/2z'b dF؟ӿR:9[)BuHZYV^ N'?@_ତl{Mr$Uö0R#b#V].Ԑ؆һ3fofv\WIo}Љ)`mZLsk 9xF}ዊ3Aƙ3`ry?j&FB4_$iSdY7)~Ԇ948.84{ h` 6k=Frbi &'^sAfeרHO}9[!)I"%*pFtԔf6.,/ҍ oV2y#_!_桳`XTJl  l.e.7|u*;s+1r+[ J蠎(I4Jh ;hO9PkslL.DBg031:`8'V(KvToO40ccBWT{AzJD Ě'Wo<\8)"1}s۞,= f+ (-"oEZ™x 6\װơ(KɒW; MVډkl*Kq.cRgV ֨,; r^x:{YA*cx*uj^:1Pbf|&T^m_b6D5!(J^;PҚ˄Zn oK#4 R$~+ȏ>Ɏbtv Kv?ޖ >oX }CTg\ꔦ8(N܇n uEN_ĩk*cuJxπ/ Aik c=[<|)Nνw~o@9@*Ơ.Ø;zd,ML?PykzA=MG }eo4| \w{d\N~>qKW<ba|pߙx!Cw9~Wʅn2( "$}7x^(akzX?̓XKiǽ9dCcE= LlZ{@Uh`\`{ŋAjZ񯳡Β2pU9 HBoe;^k7 蒨[0~ieqPʶ s~nM#> BF.ľEN$g;~dڵ+;OsG!8RxX( -v{!w<v8^]ٗ,m7t}RީҦ+*ױSa&rGX[d2aӁ3( d T<^:Nό]A2\bf?6ܻSRI xrE"fD?_mwX;aǢbU`[7tu&'.&#kpW Y b\Mu6MSHĸhMAWUD7<=ΙWQX}ӋypK`>Q88fʁwIOEt+>9$~x >Q|*ukR[]1a>$<|!D4<(cWJT!3bU,c&WEqi߷J-&r,{q90W2Xi&Dےkj8-[We:`N*[2&oF% ߔ>Cِߤ\Eѫ—Usҕ:Qξ|x>jt>v@p2eXof ^ AUf1C"I $ lQM#G2׉=-Wlkt@$Z2`|,>r xFZ '%l 8U(n 9)p{P<yRehMlԮ$negu"|ZĜ_:2 2 e^>A;+sV}`0k. XC jD2ȚQZu{dޒ1ZyLg z;'=Q ˷"bэGm۔EwzNRd52hQP`pfc'z熠"Me0 U*;*RЭ=K $@Wd˳LR5VqPې|'?..F"Mw Uhƻ +#~h5UiMguMoAVDN]d]+* P"@U:QA=ME$Zp햾Gbļmj^>tb}VK ƢW)]܍*̢3F-7-Nxm~v(6(&"lvDQ(*$2\Ft`s:c.`=i*\Ƨ~8<#}.G@)ȵdqBky3&>m6S~rew*2p&ueH. NC"7/|k6\B(@GV ,`1 ޴tQ<x:Y.& >P",Jy([|&H$8_j1rwzj$U@K|C9ݖdžx},jfrq)jJ]yvA{zyѺd^-BXjXf3p.[ =9Ō:OLL#0)<}Yp4%olem-E uZS#,Wz0K?l63jR5j.0I璱 #EGRyRxj97#"~l㢏 D^+Y, A?/>m>Q{ew- 'r]=ZZkÙ`낂^7J~ޗ@mg l}z#@g,dEunNCʊ4N62&7tZR09#P'>c}!s fPc8XC}yrFLZve.n7-D,&h?BZSPh|#ի5zm\Np"̈Tz<9cBc֒V[S+~rꢩ2E#  U|IMܮr.\lT BW{trY(Ʀڿ;%rhSm ?^ ҳY88f~RXBEuTNI Ov#F|C%YuXQ aGR^~ӽ$%i7N6l;#lq\ûa 5dQ+ӊ zKOFY*IԮA<(аRvyY^aC^زW@)|{4Hq`\zmo\\MN#z72b0 > Z<>FaZ$zHM\($Fǡ`LB,wjbl&PS`~n9eDJ_bKaC(,Gh!V~ N# ZxW¸YQ]zÒ)k(]=@`LgB@>?s  |5Aڇ/ M+j4hE1d OSS2]?l[\*$U+eûc];dȗ+ԧ<ΜʲaiJI ,:%F3|۟2p=PpHUd}I"D5pwiS9Խy6WD$A<ɪ QХ2F=LkC#J8HUa۝j;~NF{'9&9ߔ"mI)|>xc艱?q%b]S8k owE,"c['4t/eiYh:>xO`ZU~f(!a>]<#c9dtXykl{ W)Cd3wa`ny/i cPgYUFC* F#A&ئT2j IWTc3f(#NH٠')1Ց!E]˒ -bݬp ٮ`,!RsS5w7'Ҹ,rtQ;Ux ._@1P^dV ) cZs݃Ѣc7;uĴStIsÕ5!+Є#&Y#&ŦjI`߷musX4ҿ-JC:RjdQiKҺ:`!L^ǹs'MBz9E` qzOX -( ]8(*$䳿~ڴE381ZҚYQK|;em1Ӽ Usesג[ WkEDOMl|}N n p^F8&ts@_8|бKt6uZ˯v:NBE +6spHcz8\ Iϐ]V5Ojjdu2Xe_g w' !|?̙SxaM֧4KT4IeN~k|=#IgF{뙥I`W~7;.X"E&9P:B6;wUԯm7:8n b7FdFiO#ۡN[z RuVi ۨ4L84kf_$?(+@.`2;5" ֙X}(c-*fE siч4}pF/@-<׉ܑF%sD i.ɀtMr|{ "\;Y`7 UR%sys!-*[]-6F\=~ҚW`f͖2Y$l_'S,٭G?E}?BWc%7X%(1n^yptm|/8T̳c.JR˷# gM9sΰduӀbK#ȵ{bՑfuzRߘz̢9h79ْatQak*X8yI{6+ `e2ciz)J Z)mg5r{rRF拨9N^YXߜmIjvLF=XJ26ϳx Qr( _EGYžYJxS$˔zyS>bu.ۘWӓzua4N׵86Niҡ? t! jFS*#Zħg;ܟ~%_KI-ߒD8(meP`DH^[p-֤E!v7 TԢV A(C2O4Q\3Hntx@DŽKARzO&7Z~%C  xt.vo?j{aPg"ܶdR_FC;AImO3b"8'ap{b\ic\Ax!+hMx>(˝!$0IK#9K7)eD];A.H" mPԟYQ+ dzhk=t:=) N2C;<9fV֝ѥjNˣP&,'CorNtDڃMu4:rxQ⻬Z̸2D4Pr'ɟW禡2VgW,\^7 Hۂ U_o lĹgYz+v U ^MTtx/]VN4sL| ŵ>ƶ{4s endstream endobj 148 0 obj << /Length1 2042 /Length2 24107 /Length3 0 /Length 25333 /Filter /FlateDecode >> stream xڴuTۺ=Sܽww(VPwwwwoqt{ a>:׳Zo%:(932v VnF5#RhlI;ΖeSܷ^DJ4t|sL<@gc { +/vrf41vzsAV [8O vF?d1MmnN6Vc@I v{3Zh 6huji5eMuZ.`")UҐҚ57 %7>o%5D5tU$Y p::Yi_ܨޘj͍ٞə hdo? K+'58-*gW V@O_NQ%ٝ ԴW8 6N*(쌭@@1- `hF/@v9oSLwM '+'gṶl;3+_6EQ%Y)Iu F7o19; `y$Llg $ v`m\5AV.@Y 3!m:X@ԒOÿ6/{= ce|@r2v]>^t'BdY:I U]d߮_G휚A39"M4%bkdlf߁vV_!?livƶrrX9Zk:_da |ۖLv?|o4\J M_&+$A`3+ `h6NN뛰̀;]}`G?`c qFf["v߈l70/|mo +[? ? ![_K{K?˽٬ YV`ַ˽Qd{oߺ9e8طΖ+rwv#፞ ߐ'_ԃʟ;@aVwvQ4vvrvRYo (d-&vb`ayS13xSKM)?B}%'K`)yNpt,}l%—&  hOVM Pؾ7%O\*Jdk1i+.eqL7i&hvGb\&4º̲6`;b/-NA:cFw.wukajmS* #tV ^;**OU]Q)W+$qO\na&oݘ#c&d-Ho&)=KWQ5U51w>!F @^F|QS,ө<@4-ajerM;5H֣Ʉח@b;c>DXTrpg*95lzݼ(dk_x+ -zNTul-|ۉKT>WaGk0]h/8ӭHqv|Fu3:V ^zDcݥuܭB/F:(;POX|S_yR CR[?=A?=3GAhdIC^ 5?]*sO{y43pR]YL9VEW X15lAoGB;+i;ʃ~rYlr*m)oW?J,}95n ?N XǰsXOl=q\ٻB:_Wv)N9m([DJEڌL PBAiЁ)LFfRQ!gA61 53Nvw:9KFl)`Ӝ ߯rdz ) Dľh"xHnޮtd C$dK\EogAI\*S<""oX$2O63 9s\r$bRġ+vފUWK\*?. )9Z悙5DmK 4֢|V7H5D!rbjMXf2SK9#^"FBmju `+dA;\SlBEUDe,Ӧאַ נQl7~%f4i;}s).7['gRI@qe΂AqFO8'BJa\ݻƝNǟgy3Ĵd콓{~n]C@h~Bcp=a3i;v{N;Js{?sWl%b-H=9?;3'nWR`D}&ERK\b?^2= Z,BO'x4Fr,Orp~+c6AN<೻a' Xr YsB`Clbd#WHP3}b7+!hlH~Uqj&  4,h"9BA:<ƀdA'K!#Gm%)XըiZG? -f2.-eW#Vq@Ѽh#'\0]M9w\Y#G?q&bzhgi;iBEd.Wq9[Qq+kEeOtpo j3!mH**w@K͈N;%IX{ܯ|X\wnGEcO%DG߫0UqQgc~j&ь"C&?qKGxFS^읭a&0qY=g]~r ⴢ3nW?UD gV%U `M^Wǂ *&q$Vzvm$O[!vakssC] .VÜ_͊iiEŵy8y-`.MgW 2ѵEW]%޶DTKn@ޤ`@er+Mu+/mE]ӮT9di3Ɗvh>8SQꄢ`c"nmo*}e֨EWޥ_^}N$NjQ۩erJڑ'Ko%=|[#1^jwM`: (awj\I!FkcXqɛX^$m*ձrfe#Ziay/RĠ+!XP^s`ٻvܫGx~r2"Om虊P2/"ig«<Exaq~x:8˦)狇ߌ 0t!H_l|ڦ~HzFM`1- fUj'퉒 }juVvz@=&b)^UŻ9X(9& _U\Gmi8n^*||jۺ:=+mXPP)jwe\``?ɭe>DIDAY'0AToůvikIrJt?0Ng݋vCZŮBSPsSN|yL1`[^e.sn\(IjXi׬Dh-.=1.CLjgMx~=Npx$rՆDMJ7SDVȌ$S*6~ ~VcښtN["= D##+`5< Kr]FSQoAE5{8 ;)BuGH2;lxĴFg05(= {R>ↂt+PчSMqY6߀ #Z R')`i=DtfotL5pie$(ŋ{$ n͏T+cZ N40fOyqo3&pbpbwLaam*|D2׿q ƀw%IE#O/6i#W³{}`YQm¶X2 䟚f1fm CYw]+1 pI)2FC&ŋu• ~?\EKi6in=´[:H&(:JRu4@2x\7!EYh DOw5*$ 8038yֿXh %[%o1Wl]Vʨp~3j !sן3gk܄gIbOtr28(d* Ԅl0n%7u?PȽGurl>;o)ŠUqm v.&_v{[uDD:/WČ0o&F_O v8UQD}%Pf+(ci{o&l;}دMI"Oe+9(JF~ۅM&Auk.|OE3XlB$12ax`T5Ƃ%xzp 3HĺWat8c W~'-dD#4@9o/'*H`iE{5ۆOƇ@}Oil֌H9J]%+*uH9Tg0oCsIxrj)᳢@$&~F̰EMw1n #Vh8˶!֐wUNS%nᓇcy7b9q;eR*Mس> FEke&O^'PRa > aZ#%(@rF`fSQ7F5KWWȤ~- Ym¢i Q:EBwE]3GfgüPdRs!,TFEZe`dDB?nΟE^b,d50G{]xOAby LE+!^ Nܻ#b)mp12hCkO[a/`n(@u-aT (NBƪ}P1Х:2VEPe]4 3cU ܯ{YkΨǸ~ oNXƼU/Qc{',RS1\Fuc=@4'QsW/ȱ $$T,m!)v4m{KYןm<)qkntJZmeNJk+^4, \ujF2g:en gkNJPWŀk*^y_Z*:x4ۀ2~;\h6"h^Xn -¹~&ZQb  T}ILKW#b"|tϰx3%!x2Gp_?8b&Z>%{TOv,YDGh  z(NZSP&|&-sj]bAs2 [8dfOp923Oь B)=_P逸m=n4Ewz3'8DV*G*RKTc⊆:;>'XnPNk:$uNӾ5;4e"K^aט&[m1+$KE忻J5+#kTnV"aso_D\/wT1ו!0c7ݾxb|d]Nx6o:Xߛ/?Ēb|AȮl  vjTMn]as&JrQ۶ܪIovq5K#DeB1_H\]~W?9x}x+;MZZ|.N;=y]dyXP:cW1ڠZ/lX _:=SĎ-yh?QDc0ߎMfyYZp++Xpwjŋ<u>-֭g" z,Y_[M(s?40!8U6_u~܁TwXkc 9ʘ^`(nNӨAX܂Wڈ 98vntmf't}9UDD*;Uk}GVڢ4VzP G9G+U&=w2uK7 NuJSYH`T}neIҠ *̑;8l\ !fV /UNE ~ ԙ* /LKmMUR #r) }R^vrʚI|E+4 hۧ4m9KGPY\tWĹ_?IDYw `ɣױ꛾ôaU.^NZNU@Mp+4bcԅ被%QֈQЀ@nhML}m- i4l!EMSkɓo.AjfȂ 1Q*pc^|Lp/ ` }B0mgErH6s"VR~Zu;2^PG+)^Ҵi=lQ\2drW|V @xKQ&Gm.ZdI'v`bǕNdߴ"ohN <̪yx ,u]?q7v۲ɸ{npzGL cOaD϶ p }b{Sݑ`jtT{3E9+~XȬ1PK¿T)Gr(+Ӛ]NZQ%hi|uX䮐NťrYE KȘ y? /'C{i-Ra< _`+k]\@:.g0T7|VA+#x7(sxok^YW!6~L$I_&0&+5B@|–:8"1K ̸ͯP;sQVװzHRI!~N;} ~G)Xd8A[H뉜ΙG Z1(> {08WO{SZ*\sl C}NH݈}TiJhm>'[N#F&fK8/&'iJ SԘ<7ĚՐVkwA_VuMDzh?}$N \ޚGMB;]#R9\5xCgf@5f| 4?mD~䇱!Ⱦԝ az@LF*ٔ_ӽ%[I$@Xܥ|[a ; ͠'u{PPh},Z$SP[5#ƈP`8lޏYjrGGQZ,w> s0 <?/ٕ]%R*o XO)Tibr:z~O~^ޙЫRR8;b$.33#I\ 3Oe8|V2,zO07Lz`'i%&!? Rx&Wf\I)5{+M^!2/Pd14AC%$m &~g)Et#;6YFz7c̼RO ͮdۻJ@ng,o&P^NjrdYvLh$ F{ G< 5. j/ksp+y #!q?tX]ڹ!Q#(*)ݤߣݿP𨋘@>yB5&P5M !{TS^ ARZJLn?Ә֟xNόc%7p8˒"qǴ?vk@oJ/vf}j`Դ_߳0o9fWtVIʉg0@#U9d;X6:ZC3z Q~fm&H˳5WK6֌@6@mbX^O׾EYm5u g{$<2e!^Y43 {ZΞ$!lipTdpJ]br\j)tyt⃡~!u41Dc  =_{h,`{E=W+tW, Ariܽ'{^IڐCőRAPax01!6Φrf?8EmJM M~=eHO.(B,}7ԩIbWR/9.;?,N WFÀcGO14m-J)iV>᧪٤~K YtO6 _xd@vj*w5$N *AB\:~M#:ͽH WiE L>IbХw+?+%s\oVu3$Eo ]ոña6uXJ9߭57m'*$/5̺z݈8>rdBlk(&g tƒ)8>! Jz$ڎBP%@Y)AOeV]H3鶶EFOS51@0=_:ƽ9B!F ґCg$JdeΥz{Mt)pE,# K]j]96OyfCzF󑋻Ǡ_N7㦰IԅHq%[ݕM$MA=炋ۤQB;|,6ל(\1L&M3ϟ Cnhw@-~8__ lhHbҸv>xɽ%5fKD n9)eV/M3^B^09|ԹA$&2FҀxUz &$rթr c4GNLƦm[*[H)tw# 9D^ȂĆ\mJדE]siG G~ԃg4o!Ѹ =QI>!=5"U+6ҷMp!yjmv'QAk~v|ܣD*6&ьKa|(R;/K?Ɣ\!'RA.D$#4MGMX ISJN-[œi[eqsٜ-*#ح{[>}79H]бuYfVBby}q>|vເ#|.>#vACl!,syA yÅ|Y=ˊ̠Iy ܓpٱV1|X' ϶_|1G&K%ʯF7Pd~]R+(%!]ʓ֣qM\| 'خ1Ք)atDvgS\MNr=%+;-߽-8[ S:ӇSHܔKjI^Fz[H!8$A71[w{Q?G(絛0xӲ|C <㤸KM(`AJ+pA\Kxɘ8"QTcr*_T O|Yԥ T@edž )1!YZ{9с K5.3Vb%V{;z?dӱ:ϴ`Z+ uW~q[A 5.<]k:$ն?C\ \~ŀ?Y뒎?%G3Q˧.u5V^D۝g[M:!Yx6QV*a2kBeY0)!/Of 463}5ODuf0^#EKiHრ|@Ppk&"jǟ fGNDV0Q .AK#5YR6Q#eK2AiFK  6&h%[(!~;@P6VlXL[ׇ \;4F7,JmlR=3E0cR)rK 7_S}"1s6]^p:s 'Lk.8^I6 [zz/ryh>sph%1g0xyۙB&S/PU7hKM{҄† o KV2=Og]JǭmX\yں+ J֪YDѢlaq3j*I:DnǓ[ }ڥXQ} =eÏ247EzjXG!MWV\)BsQ6d24QTӚGκ 7e&Cfнa!(@_CHd~MeսNT9M Ɓo%`NS D cN@Xi~xi|+*fy@u$ە[oѥ'~sjAksWV$-H!Lhhq"`PILY/+i_d,|]͊O䥁N;Ł&pS"<)KNC˼x> _]p9X*]'{%/hQD5c^p'Kנw`w*ʕQƣc/A֟  ((kzt$~p#x H_=<܋4WO_C-o2X ޅC语L<^7|u)p'՚ۋe\ϫU}]*uuQ%ʔ]OMg 喧I?oʛuL4+y=* d"ö*Uhݯwƾ)^I[ +^F'9nZjF'CFG .jŲp큱 *rOk*x/ޝ;⾻]!it٠ pTJע*X,겗&TSh*tpM0!dC#yOYZֹMl]L*]Skl=*ᦔWt2ZhFqjt6 yB")wrw^XKySܮq@ n70ͥw'΁|6:vdzdg(CU]|O (_*ۉn?J:#(%&}:Gc_~jX)|Kou!ҤG!N4M#SmFjuK"qbWZ|]M,Y{nWb;ͽT}'4:Ft'M$c.FtZ,T9@3 MnȈ-<LVdD67Gۣ /M&S^c Q%j瓱r *>3șNhQC{V_b?P!9l?_6|Fqcj{О;ބD))}xt$XZA-[;ޖm!VQ9B'0LV8E ]߅uKŅ~uCBF9|eX;f+2 )X! )}9")1ojouaFÄZ~VnR"1rTc xU 8'Y2nwz؈vĝd*d'ݗ}KXҙ< \f:D̖/B㯋E)JuqYv`w| #MlfǺn26͇,Xhos=#+űNJos g]~zգtܪE&["(Cr0 v w;h01_@ n!*f3iX ar|YB^.G@cib2e~I G4i|6>hhPehݪz鹢RzK)1Cnzg ̳]dvXMn<9A3{!F{S>G4۲0N/kdAVDŸzfh$ij=1"86N#swb+t%E>j9بJ2˨ -Ԩ%ZHS7?&3[BANUq9~f9a-%BELP?α-}J^YIK֎cZb^4ղ~f|p5&[IJvWJ- qYuq>?& s4p7WNrk]`hэ3VP΋v1Oa< h|Հz<PʚvRQRExSh"es)eu|). _{JD{- \Ƨz<}hn?6oAQKwSZ<!Ǖ\#_!zޑe WnUk-k*TqȆqqZCmS7Λ&k^aO` e0׉ vLQe$!bՃ,4,8{]?X [o"zIG7OYF !ݲ , @w= AB\̮QyHLߙS." j(\  ',W]aUU\jY[&5ol)[:tk&' Ys pޯjoH33P+X{qa uu)g̏Ш:"Y+B 3c |u]5S%iZM ?+0Vl گ] 8 gԿ\@@ >[^U~>I<#Ev|9f:_r4YGv\ގ,vb",\1bŃxdo!;!x]9FD)mowé,C`[z@#E**!Ч Zaz-v؞W;Hn5/7rc\xLO ]l1 ܼXPryet= T1w*oā;~X(/<% ֞~g6pr)[?BDCR|_sh&f6Ui'c$1;TIe690b҆-+g KLIza)!@HгuȸGߢ|yKa?yROw@dB%u_4ߥWy 9BZY`%OI1J,If𞮙*`r=Qޚ]=EAM@ ENOr64!3"|uaS7@!qBgY\B[op-l ǚ_}8 !jb$#6pU>6wz00z|H{L\-H'*I>l憜moX%Qsl?ɫR,YM\a!JoMkVP Ytu)c -龍%okUܨvdxڭˇ^_GR+: /ADo;N: ,wR)o$XB{4U<4U1tceę4,oE &ARz__ ̈Z Mt[}zd Kdr&1m UlYm_xV1N b'4OT?T bk39CC Fo?. rR>ri؞`d8N.:ȟ-$.k'5Bv6?;*\ʱ\xxr|4*U.Rwr/~s_4 `.Ϣhqw܉x΂W$ө UHe IK6-)䙆0Вkzي zc^+%Qή?KH0"~˨vZ|܆ aWt,&3]U7{!OXg*L1V!SP9HN S?ՒD1 R=I"U aV^Q@ E?Z8'!nk4z2@PW{=qPSԬ+잧2ǬqL=ȝxQ|r$h۴r7v~g~Jm#}<@yՕ"{'}L"z uA[Xf*<~=)& ܮu CFjxJ4jOoxobBi&E=M>T,)c[z?Bsu pӮ&3"b!|hγ1^|o}MVf (!*^"}Vqa^o ţ E"hw 椻e&:7,&<,=BIofeH_x`t)) y)EI?ޓ1ngK6Ō6`vKe ǝ7D_w5SQ0;Q'վGL)vJrRY6uθf#BfDFݨ톈0e 0BmCf ^;?ga+ ;4h< x`-%'azF;sTF_%X2/Y!G[H{cVV9HJ' 6E6) ;usXX%zD`JR-%U,դNhGJ ( DD1AH:IfAzm: `Yх=vh 8MHɨM)=(Fc qwHFM=L DnkzBo]_N-@ %r/v Uq|/g$t"oV~E,\ҦΨfP=2pv =[$﹈CbP70q}/8boŚ@}DFZlI 2ij+E,pYhn4]8^ee!|+rJb=08 *$ 4Gx}M#f.=&=IW?@z'dV)Ԑq6e0QuZ|R|\#([VQ|F?(&A[Y4 jVlB5Zq?v.,U׶R=!ŭ=}:? 9gwJcaW ]DŲ拕ɟ],v<0,Uɷ11;AF+#E0k9"[O~8ݩA W3j,]D.vgH j&=d2#GrM3hoiMQ@yo0׷Do~ ar?1>I4S=<Gri>ȢF9Kޫ5yp0u]Nh$[=oBoDn i4_D>jd 2k @hݲ[фy|V8\t\I%`E/my{gHjwrYz4v[n cĶvvFFKg3}~cc:6SLLT,]PP =%Dq9Y_^\-)\{ Bޣ+ Jk+cod$Iso5ʽ~ngs dqMΠ0Vp?p2eX`5$c1!@0}^ (3g- $/é{Mre TD /hp,53LzR!qO7m/5{xe-3:rs#TN&)a :BiUݬុ6NRxFKllI_ତl{Mr% ݄ &~^@Ę,ԑs;pܳؗ}# jIzK/t ϏfϪ֤Ccnދn>##9qI3ʿa?n` ˭yJa&;M/d2O9GSkdLbx;NUrnY <{;h jjX' wTw:/Ko[HU^Am\huSt2Sz-o35Y?Z ~-l)r}@G]D ]Im˖ccYA6K3Tag=`f*X֙Xm,꞊MSDk.7(9(!#-vjTC!.wfTإ;KW5j?oDxg3&5\1sCp&0=S͘ofބYf{oR{MPOٱ^/eb?bdr{6,փ.[sIUj}u;:} :9JUɞ>p=64)xWMޮPd[*Mdya@]6/i["$bF+x#ɤ0eyA1CAV80 l}\6""bG`qZ3;BvN)Ć?|qjb'h=.ufby970`-6~h,?UZ;0 KN 0mD l JP:Y6mIDN4#V*ê>1B\:_Zu`s_vkv6݊ď DLbgaҏsz著WoΞ7|>R534AtgM!;]vm5@FT¬vt8 zhXG虄'vnx7E~RT 'q1(Jj] Xl72ŏ;TʒT~Bxlb4ۋBs{]`AeK2,Fy Ϊ:Vge66;4x +-~ǮH0Q7&Vg˶B+l3XғՆD_ ٛD;)OL&[JW˼Y/it?w_$p̮ۡj.kœal dqD+)_DGטࢌ?5(d ܫ!b _tG-epFNKh"coSRDݒUk8TKOq#eYh;mƓ1OR{j!|T"F\h^]L[6L0@>`-Oqjl|9*GB'))ЩK- ZT0YZ.Zs$I<8VL^X֏qCPx/@ fMҽ 'KeJ^~@۞q ^$^qa F63(j #BqDzJBUfm0F CB : )wd>(<$YaeLci-uUvzeK''(^g2:4=\CNL ji,s0'W*v}7#?  `^TxXm ;Ps`#?dETmiWk/i>hZ,0<*6C㛶3evo c|6~CK+ Rc)% Im-*%cZ~y] ί-

~@@s uwXN"|x)֘j13C{$zޝ@o, !cxpUg6\EeY9TgX;K˱èK 'x1aUL'c& h {FP:XGQ`ƕ]b2~Sd6YHלqФdjL #ğ:RJ,eiS2isLg'Nf~L) m_IZ֬إ?J^%D$1,RVmlA{bcxy {Iyy*%FpgfgtVsT|J?J 1jP3\YBo% rI}I׺Ͻp%|Kre7NAg3䓋,X$%+JY }|O}62 3_`ZŬ C&)udEu "!=nx3>|#L,(B_iXRAxw WQV| [Eq3d?ʵڻPk!"-Desq6^ $ P87&%@ ')VOL~=oVI;VLI&廭=Ιl|0Pл-a"Nj>^[pܦ*?A5FӳFԁogx9LUIu:pJ &'El'%Šh&Lz[Wĕd zc hE$:6L'/L>/*D ֧:@E P0b'xh<< DɈ> stream xڴuT[= '4-h tpwwww`]{;̽3ł]Uj:uk5*Hڑ 'bcfafvZb #Zp9ߨVff d4A@57[ Pqpd0:Af`kElf'ßLآ603l\ތ`5d4ؘ@ZuU U* [bU'[[".EQP4RjKi+I0Y w)?(ߔF5іŅތ/}j`i'kv:Ϧ kͿVo|##rZ+Ws_\9%%9lZ:~A&T9!o[t-=.c@k'濗mlcvptWFl Ϟɋ(HKJ1Ƚ 5[w]OD\ `a0 ՛j?ލk kkn 61y'[&ukHZL@fr56gSicfc~k-h@g O#. m _٥Mm<2)FJvJMl-& S&ǷO-I'KK[q@+GO&Xj{+`W__D,AvFf6Ny+˷}?~񽍦5 ֎d+Llm`no`ym_`bq|l6v$/ `qL#v'7zxLA&+IoVAoVA?-'o `2U0xX淢&,&?ӿ![6S?odjoXIM?4ۿ.IKV% { g[lAeay[?ۊbycV􏎾&LNoKvGnokt +yu1} VuiM>#Dhva~Xo?O P}-*j6D *ڀG{[qt+,#A 2@ ٹ /Z@8hJJDO"`q`.妾9AJ|J* pfC2jLR ߉Y\H 4( drD]窙O.`(+׮S1SI71OD-u'=ylkICtdT=ޏ`=6kW4jh?d?3TPF, &Klٌx.;>l9DL8ah:ǯ9g~yҳ>z}!f;)g)RI-^L/{l>͘ ^+ܗ\ZJ޶dC|<׌"Ac rrԥXҞˈ F[wQVhyX Bk_Uct,` ^jKSF_/eSȎvՆжA)K`P; KQa?zO7h\Fz[;\XJ>=wCDO3i=g{w=vnGτ؟Y(3E8D,\4 :+ jFKT勊x' &P=ۭ[1|`^g0UPGP~SYA NFc8*̯2\ovU6! o*OL[Ŕ5s/Xa0G4\׫*tK;Mtf9ÉVy+.!v4ت$;qxcUƄ9ﯣG/˕Z_žV:xi䶢d 5my5nFfor~H޽!K!-R5]-oW(ʃv-r%N7Y;<}*}/X9 ,r}`Ԁܴ}Ԑy |@uF ;#j:W]Jfbe0/˽3>dTq"ҶMm K5T1<3_Ï"D"dJVn8М6r\*5LJkLdow<eA檗VH=;0"]9ӲG˽.  Xkg)WH;=e/ x6Q dZknHF$9a RBzzqp)l]΅ydS;KS9u^2Vgh1]Xs7m H)f^vBFB5QngW,{vʤ%FjNFOsuPMt#mܗUIaCZuL+. ͷAc(+rg{KS@'1.ω)D qjWl2J-Ɯ!0e5O,'Jc$*~H ϖC"I7d9FT|xjMM:(Mg`/Cā Ho)a"Z^B@-qXP9r!t_q ch1wrFi>w6μ+gY1FPv;;񣏲gc|@d/zALg6J\*EeG(4"L|_,cGe#sm 6q#d;S oev0_N!fk 3mhƏfLr\AT8&x:%'BR8( w۰Du p!amTOþ$1ױeq>=G 9mD}HQMkGP9UQ_F lhLE`?ֻ)fKT4ʊ n\;8~ 4j2L yc23%K>" 3o}Pbr ѪVP7l_([\WNuvk,j)׮zS{h@n3~r@0 hR#W%¥y,〫!1uߍܬT%֣;iR9 , DR5 cZ4CgpNm7va&gUE¼hF1裡lG~(Lݮu}6l sE-\H.UFWp#&D-N+ct6FxPCVoT8NhݥIMI$[3@LWz56yj6tQuܰ >QS<(&Q}pPnh:55" ۘF]iq: j^tJ/urEӻ]0 ъva=ܑ7o] 5rಫJmK(_0lU֐ɋ*DXѸ9:tޝedKEgG>?0_6)Dž rʂ9;lӳaxU}蘞:ס~GgX50mbQM_D~oVJu˄ !A|?Znޙ(u"D9TFBw {-4J~/#2"FYd7[6' gEKh V>χF-jY7s4\f) #p X[5g)YBUI #6\_.7V\{d0!ˎRn]cZ2od1`-!hz1)k@@@Oh%OSy"C,p KO1@Z.-|;j1;Sfz1t3\cd~Uw]&m}NN&~ U&urLtB,q]rtr,kq7]Vf!:lbAXU9ni*5\khtJħn~M =Ii]$c7 0Lx+l~o!<dϮL5Ei n;U!|ھ7b.txC| N;S:@󢳭mξzr \m)BJ ]K xR6]7Gm"}H%+y 7/(8Y}.]MJ4 NfGUk- ,$z(Ĉ lxmO;OTuoNSC/` E I.*ԞX[&{}mwQeQD& PG7q˲irv}8[Rb^$.k1@K@#oI(I5DxqOǥ&ʆSYq'~ȫď850z$A3:pRf_/JYalO<<|Un?" B<Ըi5^!X=N#]wUb!ыyAhnO]ʈܗn*.F/`28:B9G): &mQaEUp1`Vտ 8͍ {DΈhf&GDĝ62 q ?)ڀTJ~-:/򗖿$~$܏okbLx7O*p 0zG>~ό"C ~vIc=I"'V5չm'E;/UFl Muc9'0~]K/ ~=iPXElzO-xew, OkwfG&ѹ=R"[P,iZ 3/4!g&D 1xM&&(~2:zWQJ?AM"kai;[ZM7~t[MCۈϝ!4 ]V\nrxJh "d׵IvX7O㾕 w!҆MLsU*[j`Rft1j;~IL}wq$9(fj . Go4RNmerEm8uM{Volii456lm?/ˠҴNBt*vztƗ%0Ѓ jx6y&\kl~;e>wD;2Tk:1>WIb ?|~C8V ͠ n,[4Lժℽ](K'uI{D#)ڷ 8FrIgbқ*la$fė݌^a 'ex'~9+5ʖ#vpa)n)Sy}r33)i'@.1` Go=Cbct&|0;I8NAu&܏vrWl]4Yb;6JNea)#]fS~XPL*#CssSk\kԥZ-Xy:W-1~2h@@1R JtuYUqI-z.FojrZs48->GV7uUk 6nPÜe%.>xcӚ{VKl1cg^غΰ6Y A5l7z~`S8f%+/{ *Cۻ'/'m4u^ Kۦ\PFVv!/s+n5p@I̋CO<,;Ks9'Yz!p9W:򧑀U'm̆Ng~pWiQCzK'&ERݤoYh.^hm1.6Ҥ$qi06lG645% t3Y^i%lk(Rv_vNouYt54 4 VlݢEa^ +bթ$RqK!#\3KG ?EĢ%UR!(:Ɛ8&YNN+$ڻvBzl~ A"4lq? {Gz>}Y yح@ bAǷ R/ڮ9㉇ . OJBn^7%=fN #41]匯ut2Gqv7:$uNxhT!R>e6-&͕/)GXzO*˟/jDu[IKmiuxW+}a}Yq/WI2wle[iӳlɧBpẋJkeLc*P_?RA_~S> pziA@S7DEޠνܲUumrIEͫGdzu3v5큥ݤ"^ϘAQ} Ȝfh{XZD(֐V C&C& LVӰ2$G*H\;L0 [['S=/KՋ,3KXP烂YcVSվA< !謹[Sh<CnʍM?^4fY.D2%K"u$q=?{](,=ّ6ršJp<\b6Ov5D2ώMee" Ceح1A<@W7x3/?6?E Rqf5[5o-iB`c6 =8Y,GxIe$EQO:4z9]ј_T ;r`qd5K;i?ذKG6*8sT0ZPFoq|ٜXfdk'&H<~uܷ',]ߑޭ,nHDƮF0b`dOy+PM Yw ɭ. 4 F><7"W:xmKk*Ui7NIF{g2)IGB1hlk_p,-ؓ|u1[ DtVrMNH|<wyhΰ eǮwy`C[j 8( Qr?6ڪaCra]y34> jU ǃaN[ Ɠw 6lvl/B0E864o%#nGM܃&ȭd<;54.B\#}7qq)?=&qrUoz UM"E6G 4AlCo$@S>7Bie0z'{ZEKv8( R `J9"gZҩ-vh b*1l(r j5SJ]FD) niTSey7XrMF,j`.c]jWai04"``M\܃n:՜C̠vO -\;߈;bb15S܅ŧ<gV&|ht^rF[^)a1 dJJ;K2zEx'%<:m'D!Y_2]T/.# ]o *^ꓕ[A#PC3i/Br= ߞ'Iw[oAH'8[ֺ~Kv;系`{zp{̭VΦ#6zھFz!:ԧ$'ݒNH/a/vlt%cw`BH-u˦oӍ`qj n[mRQj>Sw 'e@fƇSd9wOMʙnTP j "=p h{7-7}udzCz3fhZY?6?vWlb&z>tֳS5Q}$yaqVpKt/ƾ$رiQ ݄uŲᦄW1_a'1Y:o g{)_ҁa7b)vƠi4S^ \?EQv9 ;AN]c:XٽvNdv4[oM:JH5Om ' YiW`׏$kВ\tΉHhdW߳³d..H0jg,wPo/ G٤=hQaL nfobCb _oE<ѨE- 1B!~f$ߞ7.O|$rS!&ϧ#zZaq;P@Ntj97XKl3QVe0^}dEH ޘ RKDV!=LdErHj>c[pȄiә\ >sUfdM/jM,M`}g!fJMk_)) kἻ&Лnɭ rr]{B*K/Aཨv10ʤ~vz%QDҰTfu<^r$g@'«f" LC;.2A4ѹ܅'ÜD% 76XAZ xӶuW WN-f.I \ߺ. L&mwQV ^}[_[۫jeٿ`<79|t![~ċhԏ # E"hr ח ;m>%+LZhܮ7IXOҵsȔz\;Dh'$7vdL3h޿\ M#>Jlx@zVb%~N u`MCIZi?'Vm0q Im-SM8N0RBln\#(|X+ >),}Ÿ_y˰!>hMɯhsLǘ,]Хw=XF6[D|xk31v2_K8f7943=_Zڏ^c56Hip(3w kfIߖLxy˨^C\xϾP~ @S'!/.kSWNF? 8O-̷R3F>H|by3;@y d^MQ#=^3,S^͖80KK_V>jf(B]]3e|K }DžXDy@1@4|fu^a #; G@gF᡼6elheHRNvcw[ dIf*U|"aS2P 1QMԞ~B. =+^XV`v9sGk odcee<-ũ^_s+}7[wd6J~~e.?hsː6eUYPq ͆eqY# I\*G|jF.7c.9]nw9sIgF_*A4j[Z>hJ&Mu[?z[:Up"$ڵBW0u~ D_ٽJvYPxrrN?݅M l*F; !$gokȎe+J.'kA,QqQNڋᕫ:f _1m0,vmC'¸XsZʟyEaOdjuN쑥Dk˘<¾Q%ii:6rL$+]fvTctHP=WȲM*O& hxqhPX/I;P FbX^yϣ?hnh+"u]ȾHv˶̂7AWsJ~=:~ [h YDF?'p*}p$Byo1p5>Te=o޿rXmj;R9w&,7'?ֹXSK-|*^TJPi0'ʗ_fLiӃhcd2 ,k1tZ.'ݫRu;6pVU5nr$>=w%% p! )"US8(&"#x8M|?}OF~ lAҷc#yNo}Dr+ eDes{ۤ8i"Q2N2aq%sR6vP !7/u1 zITб U)pr Iw@%W:x9ݐ[[ϛϘV56nC4G8σו؀16A]ufA`efEҦ~a|ef)\ (߶$HlpaQvdLi4;E:QkM޽[01i/|up7ZR[u#(фgCpb6?f*^# oj#/:qg N8ӣ 5Ɠ&<vQ ]lq{Y*m8E=Xc'tۀaLC‘ fE{=leݳNOr#2) SIb?2$q"LXjU\8YD_~݉*-yʸcx90",(X+ ׎!]!k߉V t^Z#Ĵ=xŤݙ:kQk zNq~bH_RکMFFGV,cT,PM|HzatfGEڝSG8; dZ]2#4Ò@YVqԐ2Io|7T@Gh6OjUŮari@%EGOM(*ϩA5H _&ΧHB-=q'욅`NyZcj|{,Q4K lDzJ\USt.1 O\M9󣛰X<߳N[ƖV*cUeI6?B,OPi XG\" D伜D &x@p{w =RmS`Ke~7ѧ?yŕ#d?{r]xzpEv삺w_ϐȥo^ dk%tv%S?\]p>BfxG-[{wYvC>a3*m/ǚj/pat5IPؓ]e z]Г ߬: F< UN%V !S1xz/ ֟?+.mHŨV+lQ!=WNC$5#YW^0;`s^t{|?sUWhxS&3QthzϦ Me5XmQ>rta1o4FosWYӀ;U_%.ߥ-[4;> ^K9W$nJL/jiP |N w&?3jړ>ܒd`M|)EF<3װhE8͡fk"&iK/{MwliAd e$ xt]۩"%yoab4qZbڬIb f4i0(BP>MTJ+t!'S"BmC W,x䊄`YňP\ &]n|'ATa;ͧt@rDpUX/QQG,_\m`vY`x"޺ׯt Dn~:m-QX3Hwr^{"-P- ^Am VS kVO=W;@SpO1m:w+Q]y"1m sB؞b9+D4{snK{ܱ I{l SZPC} c:ϳ?UٓuE Si^?zjarRi=p$#Eh(H `a'Is> stream xڴUT[n]C ( )=5Opww Np _r>;j̵̹{(j,`S,.PRV; ,`;s'+;;7 3;H! A;?+;+'; @r~ej`)u`ibxu;z:[[ZA1XXD- Pق]l@s"2+@JZShښ29 Um5M`"- PђtrښZ>@ gKf<ܕe$d8]mK{up@o8 Z@XΖvӲvm ;_qu09 V%k3 菓,_|uz!ȟv2@ 򗯒h9^ !@ _dN/ ^rOKVfd t:x7] z?{f, +Ţ:r,8B< Y'!$pr_S\ loO_;{@:_ş:i;X;BAaf'_C6ڹ|-@_(.@7 #>u_ _,ѯJ@ex=`;O9M yNuSڃkKm"k2WY K8Xځ^/JQ{{υ`q4u9Z6vZ_V2f`skK'/ DaN78<pt,(& ؀#Wbfmmflj9[m`3 x^v7?y 'W>\>,_M,hgy:Z|m?kivuCkZUv9uG_5Vqhzmq|rۘ59*b b!k{9n?7_&l ҵ6XDq0ds8^׿WxKJ=Yy,\[gJ|_co@ 3PMJSXLT9<I%bxm8x~cZ[ Kz|ɸ$|T4S$hP(Sfs E^*ۑ~h*P0mr<ݤ!S_m?û =2~cMF50*}[[\|Y V Jf" q? ?+ B?޸jY9pYIB.pLߋ=MBM~kJLDnP^w o$bZ#հ<犥!>x ^TJF%f s譙緐jOh^LM P<KGN8%y^uq&=nT9HR)*h&6Y+hWLDlb: IL"^/40!9B u&;v"RC6WڰB'*@18%P$𠾧G-kPy C8D:9˝u 0GS=r/ou-ݗѱzXI 5BT}y|cdʛ!਺ZV@nɎlˢء4\Gvǽ1̛hl ;'=ے!/PIfCUH:C$ՇIPD,1Gnɹw ":f-R/w<*D\8.Hҷ"r~Sݏϖ%zDѺV;X3c{wރF@ٮx W̎F h&C>͂]ɀ=d:B,Z_Kb" Q l%JS=B,L?3MyfsIQik(CU2Rl4d7jB争*꥿ GtX&­FGdWo`TdjϾS)TB!J[ί=j5%EYh;*<hn'߰Npm?IQF3N 8b w{?fq ډ(j@q*@A@ƽc۱AU݁Gb|q6j5-Y_9"3?"N-"+[͛jjU BbrC2 j$kɃJ6korA,h[S3&boC4 ,q?"ź7BR$j' )ŹnPd]`c ܇:0H1N×Ueȇ P_@$\ TZf[׶4RʍXMF VQ,dxP]n2NrO KmN PI xWmEPc)F)9!Z8cB:_)]YK 7o#pbP Ovty[y4JD㾋NY7=̽έ ĽQ2 "]F,Gd ipnZdi|Ϧ|_FQ&qW =*q eas[@"LpGHoݝL̕Գ5BhP-)gQZ)"m) K:Ԇ4, =Y۪R3q!>E$upG,-@NT:-AҒ}+ 2sڢ+>m}AW8PY[ ^)^0 S|h;C6K'Sw7Lɪ,[g&:1Ec]CnN:]Ռ+6`2npnmd13@}sڋ.2ǩ[#.ub& "{ev-^."3}(u1EN,AT Q6XvmKyj p`-YĞq!ɼTB"y8=ヤK%`lh43|m92,My+S0as~V!tXYk1$n܉L4ZeYlCl_bJ)TkEz yеSI&13B [ o\-:=ͬ3bAY' ?r8ЄpBur0F4.:9<)ipoP-)/s|ƹ4[!#1Kcf:R@mT6Dz҄s~lWdhNPYrL>R+BvnWN%ҝ[b {m-.,:{Q"걙 SffD6~F_:+MFC46jwvj!c9E-3{NC{JKW%/xn_ &1YkNn}՜CF'wP项%ȴHrq( eTnJ^k6,m-M*pas,l!igR<<|f]?Iyc 1s}9\(s55CCQآxd*:/7|yT RA1 ԧ%D Mqc %Z%_ՙ}x_$gS7gdQᘉ3YүrW6/Q  LZ0gGa1ao =LfVH<ĝ@l 1h|/*˥e׌Hn8x.@QU-`6Q"#ݪYEStE] V6B"ːFdš*a Iا0av 7@+;AE$?l2YI;+OsaAߋm]4[ P*OW#}Ėtyr&9cLJf ٵNw*Z'5FϨYUC$ ""i&$uΓ\ 4efh8<ض%L8g輝ǘ5>kG'K*]J̄ox"rQ;5,zڿ4^3p'#T*r ۈFΏ–5T:U +8`O=@X>M22CTl)$mcB\N)(mQ'J~KH' 8ss>8FCK}rQ;FwӁ' ǐMG&F#n$zdd($;ppT7}mbUYw~Py`,m jiUԕs3B4i)jd6(V4 Ka]d%ELX]{b 6Nh0wN}E ꈀysT};s}0ӯߩ(`n#w *hZE[&~c62 ׊ACg1 W7_ hpKRqKH]b-6Gd[p10D 8'{ WZ yfF'f8ivk~o;{3NW|]\]RޢH$U³c 'MiN~{c`,&ѝ9{q 6xV0y#u<ND|^|`W-˜Xx{m)ߔfF:`00i0}M[<%vنt,\+&U| g-@)TTju!b?ˤ-p#6~#vCK27|6U3=2v!s]tYŷ *8dm۪9R*c7u~p)Ԟ^Sφ2jtK2T3 T*I#,G+@/hࠊ ^m:Dp@էCS5MC:`:$Gzmqhwˌ{㘴TQd?T0B\ld1Q(tx;_q/TJ? ~Ie빓[Tfʉ]胡n+xVbaa KqzCs̮uO˖^ѡx"Fe6Sdh;}6NXdizOZY1u7CK?|8̭039AX[P WA/l޲p9Oq8OYţZ(N'b[w#9!\Xʓ!@mn1:讨r4Yig::b<(#柩JrYoJqIg?c3),cMk? HtIgE?.EW6:~:@ H][cdнk$&^[YTuP#Z5$YGWLMYgdλi3D@C1);(t9AcŒ%~o=Itƛao D~.} wXD 3kő׃{ 0*'z_ k[óxTA$>x?J=etT,% &c$ч'l%;.IC7϶D$VUDd?$H+gwC(bM$k9ĕ HBIux׮Gh܅xrjb?>ĭGbRڠ%lYit;#mPW)f|T?>OPa_? vmwyJGS:<#/R1"3,g#~MS;E~`Vk™;m֒,Xݰ `A|_O7a'0'kW(V}z <HfE'6$va[ 3|.L\#FB6i_3ALC3 OӞ֡<}}!dx7M£ q\xo%-s ]$ń}I3_-U#NudG:t_ )TQ)RrfNL>$؊^4^'_ՐAjMI],P  ֊+PF!-2ؑnפy/C1ܼȻ\Bdͨ} _Jw) _vt/pKo}EjýS&~A̝>R{^*f\i>8qx07d`k4'*47;&{?Z-+U;DǩE8r[Ȯ-j86LrEO <9Ƈ^iWߢqjܶҼqѵv⣖Q=Ԕ sf~gʞM$Yοu9M3"(!]֤UޅeŽWnEBu)1XD>]LWu߼?ƋOQr؊7f|jāL7C2fSI1iߩxz;W@eYq]uBfgxܞQ?j} p#A(Qd;xaݓjnJZwqc+( mdao(^KbN_ ^Cf}fF[IH4q|~ ԃ(#in0*n*r q8H6Im:V /P dƳpZa j!GPCXX#ĥ{̱VWMcLG[S}MvH1f] *y cgF Wx(AdQ ^C7Z7J@9 ݿw i9-x|u\Bj49ioeRjq EGf<]E?"%3Q^g"$zNҬDVKU~!i8˲nJ w߽ E\muX mViHK"kZ*Hqa -Ur3|yr^A=$XBDD̥n՜1=41 z4q 1"uxf*`mogp:8t;{+/Xi;>;-(QD)?~(|B8ч2XF5. IWCÉȯK :E=ІGu'LTn׉6DX~TyE1;|6%K>;`s5`Dx`,*LkqDum}Uw~W L:Åp#3􉛘C$B@~Ƴ6'j>UJ:jo7_C,M] vMkIzZVp5vo72Dە]r7G'؞rUOBOF !śx2Яy" mj ~g˪$j-j9Mc̳rJwge0~}, 5y~doYaJ+c2(PZ$gǢ%~V<:SM %dKO>FΎdưy\('0HFD/?yamBw~ٳ6cfL9d&XLPm @I`S~Vz/uΤLO)_P/1d' |qhiYbds)?ѪzȘL)w9@nMEvS/6S-oCLѐ ȭcH$bR~72*%IHwZfwZӆ Z܉v6ElyR܊>.GQR6(I '|7"]տzMe-y'51spN!ĥdvuHƗtdxdׯ+.&ҭ,WX`WZyoLZ;JQ!#($$Ş©'kITɬBRCmmknpV lE[`(pg0g;"/ROnl6fء +-Mo5ymc%6_(b蟡\`e1 co4'>sXire h((EA}S^ĸE&Lorg\z,E{5͖@Xf==7z8U6JEލ! y$@0h9Klbhbp9.aMHMq@4',h<|Ze7xi^M!Ѵuh&)v<"GXU Pʉw = J{ZU<_Wt sOSk* odF|{4Ɓ!kqL]b)=%p& ]JQӛA-A*%+˭lxj ÅZ1*>+_R '!Oq+EhZOpH=ŢoŦc, v [)2IMO\mf'pɛy坄?7^7&X$eO6>G.rJi{adgbz"ԎazkUp؀0&xhֈMBRt"Ze[s>;޳Bי&VlYﶳ'VdwPat:6Rr5f6K+1 S{& A5 ӌ 0ʍfO0ٔaG_ /gyHRݞʒ ʣEW܇cݡPܢU<>{g}~VKYEt!D'S1b?]X SQ4:hw)._V G3y?1-Z(D-"KU: e^=wa.If@V$m,^n‡n3M|էu./h$_,^wPQ ]%ZxԾ]H]ej,^XSx# Fs)Cuh1yg]*&1h 0+'.L~9Um灰BzZ[4~,W|7:IS86I.^bպ_n1E&'1ƯӪtu>iLr-yRsGLj @ O/U^f$"6֒X5_5Z}1+>pUqcfq6UZ!.wj3UďKuDCw"`&39yRAZ~n@<dI7H t{D A ,6I1SpP ,}f& d7F/ty"@|M0!}z+c! %eq }vfI"k ƈT06^}B>vK94j sST`U18raД]f6t3 ܖ9oULMr%k0!]<9t>h4ќW%!x)>4,T<׮ofhVcN˺FROj9=(#ݬ[-Ўȍ^T~7fT/1M#SS. H02Q*x>3BzN^lvys_>h1θKRp'@3"<]/UG3datbQpnN`-n,3jkh'0FZ~l+jsM;נu>פۑ?b{6[U?q(;z~{}qZ%]|; Y ]خT}9Ϧ%i$+K/3}Zh.k΃YߑاQM'o?kD=[]pa0a'N ,éIde-6X9,4zi4 [  3l?Ԕmj^![ZuqhdKMxMZnb+m^f!BRTe!Sndy$&E:6O}'ds_8fMxa6P$csq0R3;4hQ4L'V/i=swD$Y|<wƕ`9̳o5+)1,Za? AJ x)qХ'~r1|.X,E=-/8hVA5=j2p{xKёN&WwF6. f-f IGqwB[1 ?0Z1WS*:Bzy&ӽFa:Eqb7((WZպSck4fca-YSgQ=$8hģrϓ1agh(B,ݸMC ܰ 5#n1Q'U>V^VHѦJuI ,Om{aC^s"N:o7R/@TâYYWBMY7p:q5;ydTKL|#_Klɳ94>&O;hh!F K">L?z'+mLDaC5<-Q=RY7lLϫ(no`Tѕ/*$i}^><3ѮʋƤ\eȨuw'hEE i}W"{;*sDjk>cSb3 {Z|=l0w!3DϜu};;.[ؽVV?{8_[`;MIQ7 Ƈ]uf?s*f ޳3[dԉ(W Y`I)m]P H1nKSIj" ([-# E.ံN 8۽VԃUl {)YSN]N"6tWөoOS>o~Nqs_E$`c}%%ӏ*J&ӜQr^N. 2,S4Id`^+}"qqHGVӞmPF;ɠ >M.kjv6 Iu:[*ȅ^^cfd(ؼ\g龩P= %֫lP( 6|V3--}ӛw j'xB-[(F>I2|饿VvK}g:=I+Sb~f$sF@vIκ&x _|u~(,jc;ގ9]Zo.`9dG"L3}M[{Dro/sWc}b-xH4@/aw7d=K /=K7N1sobltQ(G"H> Ȳ :Pwp7Q.[uMbG6:Lk >}Ɣ*ӆ3HD\3F5ƄK)X<ǐ ?b] ZafM)'T"F-rW"NVOmz ٫zs/N2\.{7^eŤ 7g%ο|RH=I-CM5[ݵu5`TD/+0pýW/c!(x aL0^&{u<_VVMx+M>'===yl\Y}ɦQkJM%Fq*+} gb!w^mr_\hPIO띭, N6ȀuB2@\[L> XC XEǹz07܂; xXj%dgܦXߞB^[Xb ,8@_!9:qxroAW^@9Vy68yT0h*:W6Ms73(Ss?e)x&R@ݧh_y_gW,.aoTN匇٧f{7U~jְ k7zyw/~1X)09ߌņpVgd;?; 'VzY?je4- =L'Llj>tq{nQE71ēqO13ky4yBKM(m*·9>)({<-)I_UO )c]_;tQ{Atk Aaorӣh <}IrXZ:7r!C?$D@gonC.}fu፛b(q+u2;rOR4s[Ԭ;{ xXM/}ޟǁSӆr0,F 6NgdrW|Ñ" JQp͒<3!6yNe:Oi㜶+-۶ӱ6:fǶmضmƉ:6܇yص֮lK=b'<*I P L2y$GȂD x2_)z$ LčGvy5Ut夅ĕ q41&m[Ə(qOp~w}ZU[OƟV;Jl ~32<2|^ή,2di_ ^@Rd ; &kt ϸ c)FBX0P`wB2KLTa`'5wL҇UNcQgvI_o0:K]Ɇoᐧ~=Wץ>n2I*Orx3dc33ws :"86֦78+ٔxWeEQdROx92a1!^-B?iycNՆ_lL/w"^-ҹܫl!$0FʈF%5n >bW˪;(ÛU Щ M2+td|_U).U@dzQsKL4}=VݘEHz=ʨFy +MaS]4lgP g;$e 0P$p'/g!}x2#?,kARbM%*Ka} e.WZ+Q[V jgICK٨1<*Rxܦ{~80}qbk"NL)L?TrBB9jU~",RJ(«! xdq!\jAg+a{.Ӿ(1Pb/ FZ<PNh#ܕuʻ1q7V]oXx⩳JX[~3-zv(Æa|H/DgDp)Lw_0)o0ߟ_z-h@m$| mVÀA5B;D^VSyX(Y[_ڦtPɀJ8Pk)3!0 $Y:#Չ]T3ԟOGLi-ll\_Jνv(DSߌSd("C+ 4wWPTz }eoPDhw C弒0;o"W[zCWW W4k&gR\J`i ',? L:N0'4 xZXE=*wZq+SoQxðsCa5_s[_g/-fC)؎Q n# "b o RpQ_>_9q䢶b%ʘww2{!yNTA)&j?  Dl`{T6MqevMT ^J~g+ޭWy-RtyIFVӡ;Cs?P\\>3\@ w`? 8l20^M@)l+=fBcUImǀ7~^A歋^[ ܙʾ/j[wTkW2[m1_ghR~~M"2)hO[S!ALSdO)b(<˗iVi'UrϠZ32TX잖#md-!lr{͚_BwL8dz95^V PXY5:ZSy U8tTiRY =)G~aέ%+;ޗ4 ;64m9jd,$99DkgaVW2{ Y+!mHqĸ#RQ.ykoZ[P2Q-/8;,'i;)XK}zb ;OzUwڎV*< ` ;߯?.R]nёfȇ׺Hl(3x^" vˤ'ā$n*;pBµDXC? F m Rx y"'rxҔ"r0Fզ4-y%䫎SsA |=.1BVLqSe&G>*Zsa!ES`ʃ(?K؁6SM4>)> ge 4dl&ݛnca!Ktz] I ~f^IAB ̇/.m%MCPW=,,u8g[{P#- .gM BzlS=\0DvYг1Aa_zFv!49gely43>?u^kNډ:|X< Mh뒕IbYF') 0PU>Wq*<$^e[Xê?r|_Ⱥ7 lnB0u2&NF*r{Y6cY ?ռV9[%NꊮWs#^LkaKs3SR3qKSMơ#ZGU\C䱨|-z%g/=BOA`h5xCT2.Rr'A8@o3'цK'g'D6\ >#\yjbXŖ9n((mqn6yG.LƖkO1%WZWVMB3> e{x>Wa&LxrW\v/v,jw9>:-<vTa=l3kz\hNnVkB~z U^WnR[udIXV;+-f($Y53uv.>3FZi Z *ȻOfi憤̤ D&BR ZuM;Z:~IHh] w 1D-ot3@9 Uf=x[z[ vB?tφ93$9z&!J`DMk39_(;cLse:5ۥ #q"ՅR~`@?5S咀|a|$w%X,-mv_yf^n|h@rm߷.1O9-f.1Ƅ]-'h^@ɖd!x)>|@)L{ Ac܉m36踙շX*siVLΐ77Sg[Cn=z]EOh2Ts36_B{ _:JĹVADH|bȩ%]IOfF`N6oѸĉBi˖Qz?aM2+YFd阿53 6U4ni}ACISQ cŨaP*u^ =莟њѱ=e~ Px Ez{*WQ!>17ҞGx)eк5 8JL"Pݟ\$;ֿXߧ&HB)OiU^n wL '/^v9:JbQ̚ Jو`B"2!ƅ1%+V}Ɍ7댅8|(tU,LYZGH\;J:XLȩ)~S$ɅG"0wmM &qލjdG;3N#קCk?}&'{ Eg8pBIk0魆o}?/zx 7X'FV-G!7eÖ7?vPe(?x` ϼñp&Kjzjv, @ dȺqBMpc qG➺l?#:vϾ[6”%OYєc>x$==J+ݤɃ%3K ^Qst^)/b-^5 Z  %I)$"F@F0`ΊXHԈBdt!]zJr&xesb='_ؓ]+Jƅ|Ϧ lv+P|>/ފ[>ҁG:U&[,Vs,qcJNqTI <-wR'~KI$Pl}\¼k>NORWL3׎W u&!j0F$lUMrT5~& LxL|#9 Di; Q10i),mvx[CdӢY1XZ@Dx-:<uq!eːćo6 ?q](ş{ @^oi a;}X\9W(YdIA>L*Ts p /X$/=Bs3ik"aE^ +TM+my6[]aYhn(mjLf oɪ ܢ](j;.MټN:؍9&!_Pv>8> E:,'pޢMKUЬ.?1?FakJ/(ӇR;93CMPf^ͻR"bu&esT, 稧V龣@f3܋N]`emtxTζA41j T5,vV ˂UM+vV:-xj*|m@*41ҙ@%BrL\5Ya;:FZ]QB~*[ܑ+V_5Z"}6#Hy;6CX9?;<`WƒĮ1dEp2T c[[ z{Z5%Meԥ*ʀ!^"k\.Jݽv?zx :{ӭj3ZCQ\HZ_B.lw"Pv +3U'6t hVVг4[ 7q DwR: m4ZAБGwA=ߛe=RοʉSIq%%GgM #Y_h}oIEuV4J-QN."E%(6L,vcxhjT"mdRGfCt[vPK X^ykW*O^=cy/io>yMR.OAݐ;NV:I,,X<J. M,W(LBe=pݖ%^y}BS  ӆ׿]G'fGٿ}]-;LCE[*~6Eϐۍte@OZpSxD'$w SE/+N>z =q55 ǂ¤E n+:8;n]K⤓h=Ur<.."I#E% FRa3$pevX6hy;Ŗf*Yj0"L 2i.۵Sv2Kp.ሿKVRx7wAQc:X,s'g;3e:Ƞ1.B:WhR$$0UB)Ñ_!nbYGH3[羟>-4CEwI9\ufDM5Pl3^I;܂2E ¥!Jf}ɴ{xD/xnŊPX:"iqk:FKaQt_\гJ|lEu ce.&pnT&H\BjobxBچ{-c dž\G"CfƹW |Bz 7Z>9DHZOW-rV瞿 aYY"िDe!G2EpI2Hntf1PV!;Xp#*.#1 !JVse~ cʈc)'U]=.Bf#ҁ"h3>͜}v&J 3b+ ۲f076uoXۨs{ѡoO]T$YhJbo3DŽB;@bSc'jX{}QXѕV^"'Qmaˍ;,FJV&pc3epڭL9t&$9%k&@%!/rxp}-}HXd7=Qof:Թ?Pƚ/ su6`r-QLm#A^`i"ʻhj0a `A6ØM8Y}3v=&A^OGpͶj P,O_} :jpX L'qJ̃$عQ֘'ƀq ?Bb.{/ 9if*%^0wGk1~dc;-PCN5ް<invՈ"1ok"*I/̿|oK4N^.cY{gThs Խ.z^Y)Z+kKd!cB* -jPQ1V8HVYeHT%_g.qʇ9_xZ6S}4I@Q =W5#JGz(l\][[ |VF4M\ nLpp0Vn9A:oKQMj+kL99E>4fwbgp௻QTir$4:mDO:D\HvA~o|doImq/Qj,_UK+S`^ F41i0O@OB+\cy6.LVku "ˆ]$ӢȞ-_6sY6 ,@帯ٲY޳`\ z<KGo~M/H5 { z|ݫ![QW}ʘKy4Tl<*o0`9pxA_HYt _L flwyC-e]`trϧG 89C" ݵ9! ݲ_ W<~R 9opw4f&H6+qcTLJqDbKX( '@W/if HhO ?U&!0xpv8Pg2~pKvG S"X +TO5UTO6^{OqO:2醺znh" ݲh՜K q`/(D5AVw#?!oe}hJ)dx\rY:#B9h`ɟc*+ M/HT1 ?ÍW7uRriq*i endstream endobj 154 0 obj << /Length1 2952 /Length2 30200 /Length3 0 /Length 31886 /Filter /FlateDecode >> stream xڴeX[5LtæC6n)ii)i99sk̹Q*1;%\Xy , @K7;R 4qvt7q\JfP33%@ ^@cPvtqe05q/,1G'/l 2eȚ:zZL EG@0ZY-@m@JUICYX?.bjRqEu P :ߒ^U+\AB]D]GY,w Wp39U@chdt d  ` w_M[\$^K ]!WN.@X+,7vp:8]M\\@_5o O.ޙ;f6޶ k;/.zfMADQFRBMAu*L}֣dd`y3>ƤOdnθ0{\ΩBu˪QwH9l%yg%YDzyEX.QvF}*IRVUT 08jB8շ! 9eӟ@3UN5 `S d{)ϯ%re{/@1X7Nӭs^1/QJ7e/ɤ9}]IR%D)R~*I7z}^!aC΂`}IJd;7qz$Qc4b2?ʩ\u3Q9:@SaP]+J՘r@tK4awjXJJqɬ ]%_+y5m΁O8T} L "h-t {\N? 3Jzzj,d pfI n_C59m'|iNGz3!'AW:f~I@ \%{"ҌW$]ξBbzJ_ JgޑRk6EUƳB\ "]!=(TʭZjӝ4E5pe9!|9b7S`JWmy/"PPhb) CﺿD&~gp-G=`\"̵ %̚kO8oiҋH" 13مO>AGScvL^qB7ar%me/?ŽeTG^'{ͳe`\AOȖbcUdб4aR evSѫײH0;,)ߕOk٨ مW~ VkTn%J8r8A25CU>6\vZ3h =Y$8 enniB6;wBJ!͇P BAV0ݙq\LcH7c[<0|To NP%6Lch6X \.2gĨDO9#ą%P#z ̱b猴' ZrĻ}Gx; m?ctɑ!q ;fT\qcF0xmjZȞP ;p% Z`Jľ Lr|ʕK)jM&@~Irݔ[6d7X5ihIfF:q;s~,N3lqFϻ_/r6KM{籫x]I 5툓=Ƚd>*ymMz+c{6i6m{BǸrcq;g?۟Zr hw Pv53WbyQKOY#Q+YK[L ˻!w-osfVBApi-J" FN2Xj{?̲#8T FfYwvh_&ʑXux$!SL}6jFmBk[Ļ5ݚN!76>b*.=x[o*>Lozt_3'xJ\pVig :J2& C|޹ch4]л$PTg%Y։IbƒXT-kpU|a){\}*>ˀ)J zŭ-~z..2"f m33yF)v=v4 U6`[ L4{`>914o03yeH*y!Z7Ib:,9v| h f. =ER(m˵5AG5l~˗[>6'%N<~j ? /-=2~E8>'jiڼh0r~.+ىjn맹vb簕mǯԞȔ Ю&"^#HR"6fı}LM zth˗aB{3g(<8a=g%>==R7\\j:[oze6(2ac|6:Bz"NG;E8FMpeʀcD%-y0QKSst:9aVQ&jU%LkC*3oP?=/ޑm4":zwQ!ddgd( Xex" 7=H4BFz6ժ0Wbw ^2"]5.}7ղ5(YG]tlQA((>$ @JìՅhyr*P&e=dBʴnqٮ&Akjzm6G;n96DŽE a5^v$>.D3/V?@R[((1PCUkV] (-x|_#~H{ 3jIeU SW\^|f{n}W~U~ؾ,#w:6~q!*Y1c/C4ķT!3 $Cu)G O䶽 6S5p~OMs_LOKw~ 2imw=ǰ914%K兣9:Hܟ.%$YtKyxNix!r9]#b f!$$ \c\.%+<:`5&𾂵F]%ZLSA\zl%ɧ iIԼ@A Q_jbzY#]|BzAMG7r I|? >?;و%%_@;Q/u+ 1 RӭQH!đ$Ul "7aa.EXJ.Qejb&-vKoy}} kR*Ŀ>A%2[K wŎ4r^KGX%|uYSiDa=UгGvª+mҖyt]R'ʐcJO "Hy>,hQ $#;8&YD:>\[qtFA7G3|l <`lʮۙCN;bu#${b_B5S]qV$x}vtQPNȦѮ!|e:>J*^!r3U z)]sQ"˙%Ewg3ciM6wZ<3PR[^(Ws_bkfFn1X"~%_ȚXYGd$;@Am˄=^LЧA`i+.ߛN0 ׇ>Qq?Y4Fg=/J;&YخG$IW{S@mK6tWQ*5\^]:;{ KlVi̻}8a<>X!a@\OYR{PnGIaCȶ qYK%LnEN~EYi,Dsq,li&mKpY5YJ!h7lF2w>Š$uK"\].Py(0^:giGkWkb zq`5g/W[\_dA*)i,Mma}^QvlM"˯WBCwZ;s sdr:&&}XcS7 =՛M,z5V@kArIFjϭ!G0MIq؈BYuC;RD4"Pɍ̂ow7PHڂҚ76$oE4Z}8ڣMhQ%qap㴊ŀM_dw]Hy5\5.{KD䞡ɭC}wjr32zR9̷\>*3clGnXL @!whn[^8E܍A3ޞ(ZO=Jį($*]!?rC旤{DP5>dWd Oޞoz1NJ _TloNn1Ԕu"^zaCuME^bpX_ 8_E ]o)?9BW%&pS "ZZ5T LDTcʳ.j끭ޜ%"n?8iRDz)7XeP2҄aZ7ɔ?Q"Hyҡ?a\ (Nۖ6B}g2yrUj ®J0FPQ<޷[k".: }%[D?Yok{4ɽL~KL2ü_HƂS*پrg2>k\YHn&N.&Җ%*֍c&k7l ౪ǓѶȟ.H]1o%M8y29kر\Д_[4lnyS8zw?k{}N<{?=[%*%q6Z:i)*wnpo&ӽECaz$H>%)Ʉ`:`JoJB{ dyxHy@#s7H拾'A]%.F"։ G]8ptl NL!ܸtGdG߇&:-0P{cpڝ}Gz#<?fg|њBgԴ[AU^H>4Dߞ=^162jUcw Cg[lal C/i(@DgK@ETXJLC]A3) TԱiB5b tStoMG'>#߼<+]21%b7LLd7ҵyS6h8_st_/KEŭyyQ1A4Gr\ҁho}{ 5]9jh;,J*5Uw›Vgu,?ݶڦԅ=GSaKA^Űgz(M eHUD  o04Q<|*mx]Hu9~>v|%tP˞ɞm]Gu 1#ƅ!pj5~>nPk^:=Α@Èx$a-=G_sU.3E^W" Ti9:hQ,U? z`޲/`zE|OЙ !SYT|k]xsmר*@^X$#hBf>jΎ:,?G I6Vj_MHZΌ)T|*{M?FE2i=|\!&= *껎޴Bd܏?b)3&:a*+IWsb f8v$6H\1q{;=1 J @eY+GOW [;jαd IS:2@yyA:ObtGH?(f4~xH(a=suZL"q&wbހ}{VOFI9TuyO:-5X\҇H"Vrfo-_5;Sf:x m{4NmLr.O4$!GBU< ݛF% 3n=6Oz#G+P}S3NDBff )},Oy4饇}Q!o!Et+!:: !ڥiz̀LC@?Rq20 u{z6JK~7,TM+oG' ehs|Mbxq'`:1;il&2ho90,o3g9\ nct+z׏Ǚ_ ~ZZp 3fԥfR }0"ʔyj)ѤVH#bv6a>|< U,7d0pנ:5 :gH`{Q3#D.b Yf S{Ezb caw_|His䶖u sh2$*ՊE?) Ț c;8qo5'l_1M]lx6t\Q'ˣ^, חl xvmHN 1|r1e^yOu>. (薍mIy"I'=Q^ϔ?w쩷y=bM (pYYA$vd}PRY`>woKxXIFdi6s+Wz>w7|id.Z-,%Xd׭Q#z-W}uΤCLcԈ#,N4{]{Oݵb湗 ۏIf_Ur"llkBLUZuy4wkR+X2"l;侸1~T Ĭh[:;K0)xo{50栏r ЧMx,g~2CBqt>O9&c!&Pijrdݎ>%?Lخe=.$aCKU9O5>Į?Ic~;| rAAX@f!4I=a Rҫ(zJSB-4U S}>]/#}%ْTq|M>*ty' Rj턒4No?zYt"!~Dڒd)u2.!)`gO>l}{,т׹ZXibBjvGߴS:{ fe`TqkqMnHm 4 r odaqwX"8'/#|!COILޙ(*KnL3"!~HbTjJ4[hUyp}}%mTR2H\g 9o\qn&(kN h2Vw!._5Q:,gOf"ߊ C4ȝ"fϟBrPW 2'd0b Oa1@:Ѐwr* Z3`Xb.vuF]sA 2PnR(Qs7TC&:o[~[ErAHbGjE+XބK_e>HK4C{krg1.F+acɷzlbt]qW{~ٌ6!R*F**b_bV7+X/~]:} #~?ͩz u^fAKrjČ Dd9ycMKOdHi ZsQqhF3PePnKwM PaX w7%3j P m-a_ x[ ҟV͈(kPTgQz=kAl2c#ۊǧ,FrѥV^jO*be%s"p|.5P^(2TXk7"yaa8E],Oc'mM5P[׬]G0ǵ*L*+':ŵX̒&(p':叶\An:󅭋%!V68ʎbi9!;&ZL J l53ψG^eg6S58 dɹnsWimcw#ЋZMVCԚ0 =v- yPUq[n{5-6;Une md"㘹 =yGdI8~boVMQ`2ҧ:8N QfRZ8'Ќ6[?׬\1 sl}5iu!" ߦ.?r#>S5RPS~&HEo;ყS\@h{ΩY"Zڀ&yڣFSK5/ĀHn55-m:uB\B0 4[T>h.{Ί ̛f֘%9vZW߻X:M` zsrȇ>;Oʗl ;:|mt:%8 I[M=S'HLX?C4Ov_z>&mL8jL8d(@6sJ˓G}ؚITtČ$p\6CME6`XS㙔JQʱCBygE9fQ./PaT^\̏>.8"pc 0}<8 \l<p2׾]N_[Y(w/i8~_xS2U%?p[ֺg4?b9-vO)q#]]W5Sw(9`&nWo`]'P]܍F#}*_Q&%Gk+l!LL'#LF8^]Y%]@asƐnΰCzB{-H?V~d9MNwNa<7,ۭq?6&,V>mlhFOʥ*|*D^^"QGe! `xo6~D*;9x CDԍ4 #`$Uo+z9a;֭9o. AgVJ!TGiJ.J0x禂"\'βh60@S]0{뎰=;$uy^<^pM=ץLy;WΣw,{ѕxgcrܚiDKI9 w3E龬=3! 1!w@=HI q9E% |oy۵H'HvaT}.ÇRJI[NKNȂVY=F8459)FL#+TljW,_ t趙>iQoyG ߭nZlrv$17BבR y7r-R֐2!kb]se',؇k0TAs)oBe{ 7CeO &q|EvMڅ]-HNִNm3g7tLj vQE'/=Ls׼ߢ+ k0BVғ9;>HE_2wZcn$\cHcmY] *EBϵ*':%+b&Ɨ~,m/RmD:3oBJ'eT^:Z6QC-3Va.@j*jZƜI#"G-SiIu喀ڇ&ܫ,#ĄAUquPA4Y9J8LMΠ0mNK(pzF6kVYS㖐g藙xYbL\mc~0`nsym6er`A0氧n OH?طݛ}:6_"Gj8KwBqC|Ix iDl-R?;+*wfŐ1KRyA鍲\XҲÈ1QxOwpOGְ@kG6榐ti/:f 'l9T&(>WRip,b#B&s1"EQH\U%حH%vQ:Va^xk\<C>) eGۈ xfF(+5°ઃLSKq~NDRusiYw.L%}K#\ cYh-QsMª׫R[cmpTy9R/%MMslbC[{PjFu2t綼R<~"I>- '2KKP& V0a~v+OȖqT(s<}7፿/]W+$:>']O'3ܽۆ&=GG7EXc܂AI z*K`tTG7ܗOc+JѽJX?<C IY]4Jh*5TUcyToXRL:9LpJHUdбwߦue6 I1TE|;#ɖf!oϧhHMICkPǽ`x{#W^t> SITzs.iOJqTޗ5e3|# Wв咇WB`waFEgœ;Us0,gj -CNLJʞ]->A#uɈ{zydw:SV5F ;_YVU px>nt@ :[XW7ahgpQOC"K;I=m%u:J ?|(_=֟۟UstfPLSDnRp-}[x>:eK2dѸEC tm$;i=eLA6E|?R4(d ZixJXN~QkI$Hj[\dCdIӗ~Q/yB-/PiՓ3ŻWcUuW3j/7+"Քq[hvXVd&weVZ֤Gi ?~'e~d?ȴ';%3xқk(SӖ:wŞGTayd,kDSc00eGr7nd ɼγ%P tVpazvV3Gќ nz?|#PkGn'j lm?09 `˯t>h76̶Ōz'\ć\#2ՒB^QH  hk/oc|  O)H≩}X)P ~lkƖg^V,t*"[- ԛ?;(6~điMi0rK:]mwO&_N9wЫO7 Q14l^ :KO)i EņdM4Ȅ@~X\k*E*HR,]`6F}r@Lq<ϒ#8C ʠ$@gqK;|4 O)7gN< BopLBm(|am8lo/ Pᶳ 7S`0T5ʛMV1)D9Ǖu5cm?h@iPe1`a$" y:L%WNNO*f46RĺvoVVHE T -(V|+c#Q *6h}_$ Ɍ 6R3弑j+F6!AuÅ Af%K>ˆwؚkah5 Hv@C\d#D*X]52b{\km?2!J5V~wȰJ_@sW歭EHC5t4bGWjSktM-F^cKy=Sն+*#K:/CG]u8k!$a35hѮPSM7L #Ik:]0Pfi3mծou<=0TZv8Hr?gf`C;ena#|(6C@Fua {cL5~*R+nHٯgb ;Ȟ,h^ &"Sʑv*ћJEnx~ h@W5t[Tv3Ol[ړc>wJ&S|5i`$qRR*뾱/9|iլAIa<_Gٱ<Ҥ\ѩtqvp;viiyh6dѻ }f-وD`E ` Ri|ٽhmf&WFW'TҢ'lw e#QX>q'm"eßuJR֌SKsŽ߅\msyُpGL”> Jw,/M,H_d;imMέJ )d%:Mf`7J6E&ju5ax=t: ?8Mjl N:I\g`ZV~$q6vY8j^Md~h!17N5Ъ3>EQ>΅lB`_Y^GuR*08 Yx! $]5㼮`z\C$ _4RLv=M[$tQӧߟoAw}edNg@w%FӋ c~0=kt=ǺqzqI8ʞ;i<7f%UA0ԅUZE u]bEk xuXHp vmH~+Ìgq.* ^Έ0c9c_*E9s:lh@MN;*ʪ0eYe[;KKbJ:C!*SdՎTK8d|BKy ~mbLokngPH\`Ļpis<-'cSmu(8-(g/ KEFǣ1ꇩ'$ʥꮯD˝]̂+jgjSݙ^n 3pcnv"kw [ 5Fr;~ŁF5.8ZG*e* 9i:JV'UGז>t; ]qΫ++$Gl4Z} =Swzc!2UV&B|eD;O ②&` $Շ:Og 4TB3UՎ-@X~49XG TOa""ܚ_ܩ 4"SDenΧd{6f;F#ymB[ڧ -1XBkw׭ lSBLj03>I҃=!L=--,_R2$>ɤ@5mr8V:AlJ,už \:%cmMcnfܱD`tu䐁ǐmf#3 YqMpUEyJi6UyWY^n q&~9 7nEXȏ3Y:s45\vg=maRyl2gnjL`1&Tː7~  P\}#fdEbӿY%~$Yaf#v^*1Ҕl{ Imjhm# o^ik(d޸8ȳ`WX/@7L9A /ﾍ(`0`Z1sǢ"_fmLa[fMupz4RAۧЬ!^%Ԝ.ꭓ#kdws,DFsyNÍKt@R >4WTehc=[NiuYE7vuP+>WZ0=Ť| fX$3cKNt9ktxE~Al˚q `H3QxWB8z CR4Whizm7;toۉ_w uTnS גw t6C(ݨB,h\ 8ˆԵ|J4[;g8O m #)pD:1!~R?H)W5Ѫ2`E/13-EDk;-N`RI+"i{ʀw꼼|z[@x% ci= Df3H&OZo\ho?&;ȈO8n v_jAiBKe)OUhS3lJJm51rv$=wdBv zKDxEQ\:81(f!"+71"?e ~7EsTeloP^t`lWCz B,^7p .]rKpzԣVrL,;&]\m-\Cf!lqP-HDtZȅ3á85VoTŋw]a\"Y0Y‰U]XmH=ڰW"kڕT-ENJm L\\ngY `DE_m{Qfg,l\G/X3IXt'ͥ޼=4%aTwQ3s= \_h"ڃoOǒKI ーni|r;HK &Z@a7bZ6ɶVu, 5W=5Q#g -V4;\yذ5/帇50^e\e%2ǎqJ`u;dYG-JV$'K@{$%ӯܟ()FVC1t)@=Rs9#k0d`DGi }[>xqaAeb> dV$Y/fD$@ 'y14PxMoC^CP`wN ;W ߽ g_a=H#}:e3$aK1wZLO`;v Wo۰' Cϛ$+Z R8cgBȼ|`1_D@ lUMi0(4PmGuwH]lHN oî^ZݓH՘>ڹjcL޶Ņefih3]M*C{$CBɮʭ;8Įdkߛ١-f')>h'_2T7#1D4rW/ VSIG%@ؗ1_T>8rnRu&.Ŷ 'p]=iZcqC-): HbXeTRht[X\GpȘ7lL=ԯ"G~|!`KM')秾!:%9+qy8%D| I q`B30QA@B{E|0(\?E%ۣ1]>ivk4PYJ-Z+UHkv^G4 7O||h΀Iд9ܕc8Q zMs qZQ]1(ǘx B15&qF9w\S@-K[To~R"L4?ELݏT!V1Ť,3zaGf)$ T_P FEי:AwMFa+tT/7c :j6Vqj d>Z(F*/<+EvlsHPx\^i_;qpXnRtf4A 6*_OQ\նQ)UWLlv0M~H BXC<`Ajtجɩ I,SU 24K]_Z7U] jBzw421!AlŹ J|I)`5&va<'q>${pz'w˥uEx({E7Tʌf =c3[ݳ,q7oXީuʧwK#n.ĩ_إ6bau (,r44xtw[o.{@!=hv,=\($zmjJ} x 1y=X@D;$)[EޒzC~~(s,9̽Wak +Z,˕K oc3W0YTLljN)ӜK=qS!rx7V5*wTI;B<+KJBl o }qW2"؍˩MJԶ)O* 1uTm'C,xiU[E¨6dpbܳ8}hUY(X-)z<j1a,r%efv(%zVf;H1ūkϧX/m(}'5-tp6cqϹ?q﷭ܼࣿB`tWdp0Ͼ 1$aR3A!.v%R&%2kqC;GSY@ԇ5P+J'~Lk)r:J?%Ktg, 8˞t7NZǏ&@c20WTqlzL~15yDK!E1: 'a׿m8IAh?W5ƽ4@ PB7EAdoMe0˥.ثiu?.UL;ujaɝ$YJ$+-F]:%.TV.bL/t Ϗff~%Q8Ҽrxެ`&#D^gY5ˇcK9 >, 4+|&Y%SxaHw (ۋ.z/x"/mqhkK1Ѻ,ꪙU28INƜ[+|e[\(pZG =}&ϺU(K"/ #&9' ]A/nr%JzEKdm?/ʵkvZY%[Үd B^o:50 ,SDڿ G|%+#JCs/ѣgj._w)_JJnWH4J*]@ȱaAfE] k)kҔ"Lѥ; ꊓo)5  {.Tihֶv yHT'اGn|8F%tKEUN)fe(E3:?v{Zl<-&nҹ qógoV3ü gI[tR#=+c3P9%W_ -ND>llv\2؈LGs~|>,2}Nnn( /pΣX'cZYv Fh sA\ dqz kwqr:#!!۸0C~-sv)r( @c 1vDu/0Kp9f_Kt2VK2a'@䏮XI|ָW/*I~cG7{\FKjn8\~;u!#P-A9K ZIʸJy٧qX=n ZK[LV[&Gƒp>x^vhpp +;׼g-0Hd8 Ci[IJ?!G3eS+-eԟ>MC^d>H}QFI]:4;;f)$lÇaqsYKhOXMXTY_۰pࣾ~pI ՜gDDZ*~>YrzV}4325 D?\yF; ,/_wfM=*h6a= XHs.)szղVZqkO;"kۄF9&YU \;|AVx,};6V݂qBRx'nu0L3k'{t*1W듢8u?fp["f8-u:pڢ]uzV,k;쉪[|4Լs9v)'@J #'C蛶 N7|7vFRdi6{$Z\\k#WIk]b9J$!y,Lxh|3AXR17Oe#Nd4\>3͐k&M /Y FA.ߐ:0TӖ^*U-\|>!岹0."*-j*ψR-waf+j|y8 :Z"0g69B88NE^(PH/&Mw ƱHx1ە v2,?RDϏj8,^ɞ^EYzm DqF.I+L%Ee wGݝ٧kA t(SlhprjƐ9䍄΍>LJg vIᵆ׌ @r{|0~Mw_k9hTn}T8ҷc M 4xcrpNТ/0]jx;Cin[D,M+Gs\8jv;+ yLKtVΊi]jw#Q:Ti|^SoUQlqIn u)c9` JVĒAAׂNޅmpe/a:Ld85.^nd^E1Is~W>k<|4!xH*+qSRF° z3 b_> Ծ۶.m@"-aO6GjII۝Oeo7HM>xc'os?;?XT cyX@O1gGV.ɼߗNFá`#"FD园 ? NUmtTZ(}<~RiAYmrF_` ( q&݉ M̗p-b.=bRzԙ12}5Li5NzB-݋T1^)K&>,z=DoX\ƄzADӟo2DpdD{x \f6ĦEI[PVd`3cD@⭜qbwŇ}_HB!.0a3%V:+!| 8[ LgbYɸ,x@,L>3¥'HiMzˋ`pvbI zԨ\-fh Io x0mkuG{}6k]Q1\@d 8hl¿uK2H5AIWEBʔuhejGI)miY"Aeg9ǀJ]JM> e%wxqB+տ#dUggdI3$JL|ZY(^`e#-R8):`b&c y_י!W\9U5O~` ʼnױ`Ym?.g{telleNN,2؈S?.Wt;mRk7 ᤮FE2څ lOV7GK\>56ˣD̚9>,cJ@Zg#:B*o. @\p & x>[o5,\kU t= NgcZ-h*63MP+DOɏˮ5a;<XCV)Y8}ZDL)0S=Px$fxk| c$q~g*ɂn %?=EDH7/lN #,TMpOn/$LO):tgyH*~ĞdMՈj5p:17nX23%<iKo~bLL4G۶@ŚXl4rxx|/<퐱,iPJ)>?Kc*j~BH˿B3sexkÌnK)`ToP̾ S ztwD €v9j^\Y|= -u+]V`ibxOò cW۝o%#Ʒ)?,0vǬ:jo]RYH2em$Χ@`;jV2LbTƮK45דnOyS!؊;fQˆ+OXA(riWME0WM'iva,8pt9$7 S5I\}'Jߒ`YtN8.@0(̣6&saSpYiE T")&-)r!*M}s ;*'Ա(M|.OCol>$fyz9_Ña#!TE\v_rRﹶmCCJ_ |]7l/mEm*[ଳދJOZ ;/ktpՌܸIdǁ(/6tSzf|T,XlTJT4M7dq (g7 A@"}2$eT˴P:?f]%[V2f!z?E&My'~Bsо"uސs |ʥ:H1S]L33!B~qIQE %3 ڛwSn~g1Cų5ߙz yy ʎR^X \PUEeq6kdzB'^LtiNɇ%)E 1~ɊU cP1~ϒjzECEgn o"ot}/&P\CXgtlN B<<4K ' DH2l a^ʾī\íMHGb d& :ݒtƸt9{"\s|P(HNb'u&y(vO[$ UgϤa(s\Jߢ#ۂ(].Wj L)&yܰ 5 HJ n:偍n$OZ#ՀpaP9.1çCL`a"c\G.'Y+V]28pt#ذY? y x+"#G[k)c?SW$q$H! ?fZ424p: 8Vx 79V)N)Uѷ'LgF͆6&vt&PRt1^@t>?^We&<Ih}}t$?FIJ/LMG,Op-_֖nllS&^+A bqnFn:$GQ_y6B,lمUZ@MBNsM0T&uq$9yەҨ, 绻 eC0R)J A*́v/X,߆=P`5hAz5Λ9)OMb,_LvCqnt1p"F-EnQϡǃYLg) ĹaK3/7gZ8O&¸L箆ڊEؚ6͚WU#8P_JDŽլͣΞ2 ~$'6GKa 0^fk҂vp1Y;%v{IPx޳AT՜yҔ TKpО'ժݑԑf< GSx,<),vP`Z"\=âĈ]Pbp q{5[1:/O|H=qy *HK#^}p ̲b<I($7)\<" ll{{\PNJ|];RvvcV'3qcnOPA..~/͖HKׇ_PG~OQl/(v][R^:GƦU%A9vRK@sS!XU,C E<,~@dj#cQ vAZȊsM {vl'k#gkUO )a%w^Fנ^u 愮K LY_D%]7RXXTGf(ʝ-@J fߤ,D3X%?J-NI[g-y*$Q8 ԉp,=QFLObXrt V)b|SN*d #v/-rF b$Uk#*?爤D%8?Be*T+[ey+@!fCuyG8}樫`sߗQ.IdʼnHW g\K'aЗl=Cck2oQjN}_>-f\s] *{̍ rbt-dPu.jm1feTR5 9 D[@( Pwb5}Aʍ5.-x =H`^gCf? Y/F|sF}& Z쉒}i[v}tZj"Db*ӈ\K3b#9BNRP 50 D <-of3iRgLYi`ϩA|K - $>"e>i.J-n b endstream endobj 156 0 obj << /Length1 2276 /Length2 16257 /Length3 0 /Length 17601 /Filter /FlateDecode >> stream xڴeTJ5 wݝgcww   }N` sY`@AB/dbkxrr [f&z)1 Bh<N9@`abFHA@7 TE[G02RDl,j'[ mhlehe0.oF -`476ؚT51e 5[a';;["&AWj*Aoo?}ˉ j)13Y tp}zc[_ T`# #/~[+w5/a@&oŕ*gO@#O6oR%&&OMhcnW, Ao`C#/'Є_'?=6K]me^.c 'Gh69Z8U0ag,@TTeD/g +O=!QٷQކT d"bkc|o:m{@. dbGw';F5PJL̀`h3i׬131agk05vzY߾!x8:`'?9&1;*Ue~c)5YLqW/q'kkyC )a6n_?Tml g(n 4Q%$2X9XQsFs{986V #/MHE4h{nۚX,CC7a`agx0M i02lo);'ώr8"FlFQo`qX*H#~7[۹`47d}#dbkmmf#? +ޔ0w3f|kڽ3Y.V dceۿYi` ߖ` i 4meN߹o|[?b~#OorVl [k?p{ު8:HC*sU?_q/vjXV9Cfߟ߷?m]=ٸop)H5׭vly @+aᗭ1oerSpXd 7Q6t,BdG\ќ R@_o[YI=D?P&E JkBĕডGd1lu5ty߲yZlSbFD;X_.Hu˾-¸07c:XΣw|cD-LcHDզMSTR;^=Ѻmn  ']?bG,()&lY<ʿ*+-`7nG hrR 8Tzc9QZX=OA|@TIs[nK6(H-i[b;Ӥ$_\S8ue>mnbDgܓK I ŏW5kά:Xȃ!cD]@THV8wVYR6 MƕbeD҆ rrA[p:|AkO9)sDd(QF q ' Y -K+w"z.Z7+TKqdHKq1f[qP.%I8>8q$bgi&b1-%x!'鍬*!gs.v?(0 W+,RuU!MoMhpZSNLxjpG]/l3#G+ ϧ#4i8C<ŏs?)Epgν *PgͨkG^7d,B;W1b\cTM|tu*DKayhf[qi_mQK=h9ҊUI6ъ5к0!?]d5 $rb>途jN6|~?EQ8?+\8ͭw!5zZJBTXΞj!zςv E/^UiJ)~6/^U% Ft* $} GxQ.9qV&Z5ޠrȯ̎uKdaovԄU׌mQ-U#m%V45 {;+ O+MAۺMjǯ ftQ 6O13 kU&n_5\y6fioq9'{DڸТy%Xn.Ls ؙBS͇|8f  &Vx@Y8.&qG=@&NGp!ٰ,&c}X@,I'BGZn0 XmPQ;qdM837 kr5r;WQWe}X]ѨK|C' {9x, aJ=;)SD"ַGKɞ;dV-?L`): "b㊑o9%.W^x$/!0cU3,տtN}sX34;?q۔W!NOTzoB\J.qk}n4#$Qj O^@CCrX&(e⮖|-~׍ @\`?z +3 .k-Drg_Uv(py =e,Zӎ 9(5a|ZKTC;BdRQmY9/Lt-&4%{GHm~Ntf8c :JY٬M';XEN›M[ Y\K 9-tzjbZd<}HIZ@:^uG)&vSi#w-f!$om'RFK@| 3~<)rF\+JX*FߎgXt ̀9he_awn)Hv1uz$z͆#G8G&$q'[<#3CO:G&C;wy+C\R$qݕܟ Mh!um9Aj_bq<)z2AO+fRU2ܛYeb)x4R& fq5fY3Y%l,7NAv_>/Ll}\] n@> lVw$ +25^aN@F7bjF[~u?OUaW\ߜ z?@Ǻ~)a3|ڄ47WGLAB!Eqcz=u9Ef7GS2rv{#m_2~QIL_ au:9B4{_~}gfFlk@)MkYZfcU<]wv9ڵ`Kyטp1/)w*OHHACά*یoxUޖiFw{;.]CX+D֪)%rR_p 0og&Rdưƒ04u*) ^EMj{L63ġm>Sj?qrrG=s?Y nfV*`eMWCE*vu(4ǍcK711DFxjHֻ/#$% U#2^xCz .P?ޕHz~O7 [lWo%uih9jو̸ЅvR(Wi6CH}ċǯ:R;wpK,ȯU|tx{ ^Ҏ!^ｋ-GpF*g!BRb_v2#ԦƢR}\bt$A8[i Lc10rn!f6eі۟"lr/g>hzeXwb|9e}ϬymRMCpBN 8 K0F8LyIPdbIK_m䨾.u/)Bh aHօ YJ(@mMuؒ|a4g_>L0oSRX+i\jHњIɄp)\ۮgҜ Ɇ;&Pj;q]QrGNoii[ldk+TSM<4%-PNq pZf$C{ PEK^kBV&s>W|@8Tq SmNpJp|&G򕌐L_ vPzޢPĀ.n!l|q+S{f%8T FL[#A{E"}l$b6Ԡ\<G^OtsiK.VJ{ܜƤ>׏Ol~(û۹s1_lm]YReMc#OFFKN7\&޿-4X2Wˉi o5O#D \ DJh ?Ė$$ siftr 8A%~Df|"DE%Oqd'ZS|sg럨;.f[y^n"u#ѕﰨVMN.T55=suWqYww" FՀ 33 37hU2$L%|LI Jp=9RqE+OQQYš|1DYZ(DW}KΉOhVñ':k /m΍H~EưUCر{;r+Zkg17j6̴Wf2ķL!!Ǽ%v6Bk(>U~NIw!qn%5R)H%]Lb^Y gL56qggbƶRF4rv(> e$70޻yoR>(q8یn&Ug:[wZdB?C7Qy _Bh}FH"ILWЄ nߦEb#̈́eCҫrO"bFSpeM'\f 5GO1(YO\nkgz9'Uma\FiRah8s ]gAH.~d0kv}_gnّ5}LMW ϼPzȶB/VsOIwk{? /15>Lڎ5)G>"c-}m}%IPޏXX@ F30H` ) !/j fJ Tg1-eG,%h9*N254r:P VhUTTt'*[x$FJ~%JJ~i||Nm!o<2N-30A4'jf^6@aGNMEi=˻%Ƌ ˑ䪢ն>T!]JOTu8H؀V s-}u(3^ ,$tI t^؁nwFh72t Cs?a8I_9gl$$Ԫce[e> J^݉\ȪbR1.ˊ ݵC<{w^ap61.H_91[R%`_洋n JsvxqE%q .ܨN!hn_~,"Y( <441"wWRR;[*sͷLX(M0m!IN) ߎI:T>5!f&B tee PԺ^Ƴ*]q@7Vd%fJWﴘE+'PϦKp&H*T.lUܛ{_'耚_ҵw-ftg{cqNK!7A JV a70_pW}4d{$|F F #o£5MOΞJ-PK85|VvKTkRITˀ$EBj}J2fFK \=4{h4 Cq+w :hiiK$ ? j?lp)Y dWR*w~y~^TL<Vs+Q1(];7[+9&:B9%SMPl A:A#jbvf+q&Ҵ"-]:4v\ 3~M"qh%YZDТOqc& PȾ#7~XߋIJ$ Δ H3u0\PGv+Iy譓A;0\0`%ZpF͔!Tc_oEUe(:xH22"r (_pF"oJ麶:S-0Z m־r;2_. +VSpjH=Pc V#<'$ˉe U}O}y 4`VjBhα  e:Rv̦s/XVT% kf6.4=kd=l.dbsM#jgZb]jܠD6,wkFj>PEBQw +;cʅ1cQvM}ojD1!7%N)Zi,Q$&+J=dLyAKxhc@91/p|IIޟ,T6ĴlCK~ʙB6F~e:~Z&M69)g8ɞ@9L+^Aş폝 C<9O[Fl-[gobVDFm`ye)iw4Gfn 5NŞha-Z/:g#2 ?at5,:/F, ! oCnKżDs\GLѰ=J栨V LAs} 0b랬fHv!Ftˆx/mb#u] Z݈yEDK΅r[~YW)[಴C'j|Wß17mR҅%^[A y 5]G.1le|lz}jR&J舞 Er{ُd;>p;- pwү؇7▰|z=⏁cBHi5\EckWy&>g1L}p&zT@ 6 KTi?ė7|g|c CAPދ}lv5N~#1>xY"6\?"g>8^̧c5f]1ɨĞN R$PxZFtӽګF>Yx]~A|:^—pRl <8 ӷgNG€`FQzDW_xxDb :]*V;}6ל*l1PIBokqb<侰G3i1+}cAp#I32|2Yq!fUJ}{@oE%/)pEZT[Í:r"^C6_yRL()TZغ*jN5PUs_? t73>kzLӋe}_t;Jqx@ .,sdVo޶֛zM 4ݬ`ؽ@7e4EP,HO7)h~xqSuDl EjhU(8BOtY C7_M 2V"w{q#(y9 O0.Լ]d%-S CFp>=G&~λt}$`4%s ? J3ѤJ|};+X}O,Vm.ZǠfM&D\Ui~( Q@`^:na"JKS%; Z}lW֖,2r*T#R/OQKQd>eP-ɅMKmB%$wVचz/ߩeE! 4?}@7J9i|#wmDhlϾV4@ˉZ1J[. 4: r9{o ݴzI:FKYh ݵ1T%f}"UaI mT̠bmH%$&PwrbvVVg2-j6!Ĺ1b׆zE|G5;>/oNN ]q8`puXs a^w,pj?3@O(k~\ %+Q#K޻21U!3O2jH N* ~u' zh^ FY3jQ y^#YZHe%;H~mNf-gĩ)$If|U_li #(첋Gi bS䃫6H?_7}7u)ɼ}O't/ڔ1GjVtԧ66;o.lMA=wRLܱ׷_=KfǢDjNjw} Y_AhO;)P# l&a^E5^S"6K65YӞ`utuN5TnP3Z-vc\6ѰF7|zǁ0/JΦDM XO F4u8KGs%pL }{_Bb/oloFS^uߛeW^ntáѲ,{R'"c$_Q@',x{ ;Cd|KĪ3WWM7@YrQmĤ f^Y&P -N%V+٭A L-5%n\:-GQB␇=* ;jg}xBG~գ hc8:B!}WgtШQS[6w)w0ٞYaya#Z縒]΢OL4_$ mxYcr?joʢV i"DVx S %>YԵ)&:iASRGe6dA!&}ڒňo79f1/'F:#(i0ɽǛ|>FDsd!aV\$UeV \+F'fIQS| 7{T8DxSqFu1lp|B6=?!ӻ])~QIpŽiӸzQ^e#WK1oPezZlq@E*gӢuQ}u/K9=pSZFۥL *?P= bE*cIZ k*OCz>*B>ل.SKD~bdqSP4 8,_)Pʈ2L~"3Lړ֖/$YW4\ȝSS @"c_n财m_ètHy:[}b䟯$򤢨#ˋ ]l&&ķOw\~+B!=~b8R|Gb^ʚSſC Uw\SB+Rx =>؍7p@2@PM;X_2xq^+)#X ȍ1 R\xPA XPY+?{QwQ S37:15xsKHLhcft$d鬍]SriSd|b?9t\ Nn,n\C-Ko1OP"3Vټie6SBûvra\4Sŋ|'#CTY a;1VJK\3VL[Ctk GOl\;fGoFMq5yQz.dxB;20tC&̔JTAhú}7 |ԋϏ!9ϓ!O1 .C_ޠݣn3%al*;+>9FJaܘ`òKg:}Q.y*@TJ@IT/H5baG*NGAtf1!Ч{3FwV!Q*55Z.+P&`mˢ!7-lШ޳g׭" WlBLPxQn!g[$FJe@MǦ:w7:$"҆ME ;b#+\N?s *Ahv5%cfd,aʓ]s5zO_5X][|%zµ\$ce`LWۓ}ԠK.,iMquT!>báMOXί FxrKHo RTo'68Ǫ_!/-vZS YY]^x#J;lRXyJ>(3 AK)JU'yd;z\7kAcDWV?0-'PgRp)$yʚk!՚ law/R"B@}l*`*ݮ. R5ٚ<տHMO|f;Y$']ck˫TLmvsu~g4Oizt!  aa@M  ѺLCȄ>t;u: ;LF8;.ugvZwN~1岡DڪA颺J^<_h'GcТ*NiU1j%{kheJ1 ըQ"%O۞ںeKt~:r1Jd&-c k[eW^͝w\Sm7$Ե $go R77G [ӑ9x{Tٗ'٪Q1Kfǀ'n;\б,zO{=*O&'F}61KHBy8+WC-m;|g~j:]HȹT Ճ›&y|l8%DA`u@F0Hja~%;;A!}pӼqG@ɇn'$-cxR|2eiKe}&&0&줿JL( ngf Bþ3^PXjEbEٺZ4[Zػ!'&|pÔ>6h쵣J@e#e;Kq˩c'a1 DJniaߎ:Tƙœ.'NŴ@-BYKK_Vs rt< AE]hƒ>3EP6ЎnMF$*ٞPYA{zF\K("\ӻPj':DW*h.U/Tlt DBkE$(A?W o\e۳sn%L.z1E3=M?tՅqWQbC7)p" ECd_溔OO[W2⇠NK3賺qZ{^}m.1ݴY{'^Uz9ȯ4 `"i=l942r~}*X&N{W QH|Z Aqa7ww;>jDx -OcPh;7̡uBCr;v~Ew IrF oϢ lk5dh.>$gg\NȆL9Oϙ%pXhsO*"2O0I;Њu? FSO=1 GЗ=r&bi$:F59Lؒp}۔i'$ Cc5tOf$ھRO0Tke'\X4vuUuE(ܶE}ћP}`ޡX":0 Le+3h:DVj]E.&S,<,b?>E*zbyz^3.#'h{{ ^)n|{ғ4'*UoC8eP&`"giiNf%6'@) Ǫ9 osfqN}N 8VQ ~іPl'TbB,<nݛ.&#*%_U;ڙyyIw i K#kFyi?]/2~+'ugFdWك542`{FIX~O8tE>@ؼOt3  ܓSWmAD4Y#Dq9鯪Haiɧ9] ~Q@6ȘPܲ!2|6 .F,x*g?6vM=تK1x{z>Clõɶd7%{(7ft2k W5Y ;{"HC\/*]:/M식&tM$&P{f?Id'FW2ʝ*5C h /h€b/\bVnO W}tjȴ)1cND [  X"jI t*}t[}6ٲ?B=vtU R@6li`e%xܗ`_)0r:/O.n'7 ڱ#S*B> endobj 137 0 obj << /Type /ObjStm /N 45 /First 379 /Length 2670 /Filter /FlateDecode >> stream xZ[s6~ׯc;6;8錯MIi-Ѷ6T9HOw&!q98h M„%Yx F`C"r Cp#)Z"OqDpkGL)"0b tpMB<%% [;*% (Di(caDGS')aDs҆hhRHa0/&!j,1Ca l=,6BqHd 1 A  '3-eŒg"Vj8 Xh(5( G^TI#`{e &MmA$tr ֱ XK ڴVJ2ڢ*׶"YpN7fȘo'E:6jA5*U:R.hO? $:Y?/" I#7+5czI3zCtBszK9]Ђ.2_ڷJ@KZњÎ~2\~?>=?bbu+{`{IҭzBg ^rŬX=}ZjOW.,+#t8Ibnd -:fE!+yA_S 8:u`p~:nVY^#q|2Ͳธ3cW5SIV*÷o /Ve//˼*~^u>9]*O=*VZeM2iW^-&yY2__~ӿ;|{(>w1Dދ׾hrV[}lmѬ!mn"o '>Szur|-dW|/KRRJb|$kydr:I}@} I}:<<2?3'D 8zOb,) yJSֳ/$qMv#aw]GzR=;z=/YxWĆ{H#b^vݖXCB g5NrC >XKL.Y6wyW,V[6H.y%Fæ(WO*GGoVF>m¸^wE1&ݔ8w>Z9)ahvU)pnXo|MOyUOHSA}+QE^]PV'ӣ|ݹͣi/ ^c]p^oջ}sηfG9 %pu7\9mkd7ŸA|d,͈۫Mi(:M8{0D_m\4>$6 .2G>#'4}lohj-;h<!@@bPȶ@}MEs@z+"[5nCZڒ5p?k4do[/KW >>%ѽ/gJbݸC[_~-{[oK/ח^3TC)lA]n ȣ$x_sM! Uo]EPn7haf{Z^^_LRn\L6d+r+w6` ] /Length 426 /Filter /FlateDecode >> stream x%ҹVSQBdP2IqEGKGGs-J)XtZXh淾GD_ t(sMl // for NULL #include #include typedef double optimfn(int n, double *par, void *ex); typedef void optimgr(int n, double *par, double *gr, void *ex); typedef void (*lbfgsb3_fn)(int n, int lmm, double *x, double *lower, double *upper, int *nbd, double *Fmin, optimfn fn, optimgr gr, int *fail, void *ex, double factr, double pgtol, int *fncount, int *grcount, int maxit, char *msg, int trace, int nREPORT, double atol, double rtol, double *g); void lbfgsb3C(int n, int lmm, double *x, double *lower, double *upper, int *nbd, double *Fmin, optimfn fn, optimgr gr, int *fail, void *ex, double factr, double pgtol, int *fncount, int *grcount, int maxit, char *msg, int trace, int nREPORT, double atol, double rtol, double *g){ static lbfgsb3_fn fun=NULL; if (fun == NULL) fun = (lbfgsb3_fn) R_GetCCallable("lbfgsb3c","lbfgsb3C_"); fun(n, lmm, x, lower, upper, nbd, Fmin, fn, gr, fail, ex, factr, pgtol, fncount, grcount, maxit, msg, trace, nREPORT, atol, rtol, g); } #endif lbfgsb3c/inst/include/lbfgsb3x.h0000644000176200001440000000216613577744616016253 0ustar liggesusers#ifndef __LBFGSB3_H__ #define __LBFGSB3_H__ #include // for NULL #include #include typedef double optimfn(int n, double *par, void *ex); typedef void optimgr(int n, double *par, double *gr, void *ex); typedef void (*lbfgsb3_fn)(int n, int lmm, double *x, double *lower, double *upper, int *nbd, double *Fmin, optimfn fn, optimgr gr, int *fail, void *ex, double factr, double pgtol, int *fncount, int *grcount, int maxit, char *msg, int trace, int nREPORT, double atol, double rtol, double *g); void lbfgsb3C(int n, int lmm, double *x, double *lower, double *upper, int *nbd, double *Fmin, optimfn fn, optimgr gr, int *fail, void *ex, double factr, double pgtol, int *fncount, int *grcount, int maxit, char *msg, int trace, int nREPORT, double atol, double rtol, double *g){ static lbfgsb3_fn fun=NULL; if (fun == NULL) fun = (lbfgsb3_fn) R_GetCCallable("lbfgsb3x","lbfgsb3C_"); fun(n, lmm, x, lower, upper, nbd, Fmin, fn, gr, fail, ex, factr, pgtol, fncount, grcount, maxit, msg, trace, nREPORT, atol, rtol, g); } #endif lbfgsb3c/inst/drlbfgsb3.R0000644000176200001440000000057713511470416014722 0ustar liggesusers## drlbfgsb3.R source("lbfgsb3.R") source("grose.R") n <- 25L x<-rep(3,n) u <- rep(10, n) l <- rep(-10, n) it <- seq(1,n,by=2) l[it] <- 1 ## odd variables have 1 as lower bound ans <- lbfgsb3(x, grose.f, gr=grose.g, lower = l, upper = u, gs=100) print(ans) tmp<-readline("try with numderiv") ansn <- lbfgsb3(x, grose.f, lower = l, upper = u, gs=100) print(ansn) lbfgsb3c/inst/lbfgsb140731.f0000644000176200001440000040234513511470416015014 0ustar liggesusersc c L-BFGS-B is released under the “New BSD License” (aka “Modified BSD License” c or “3-clause license”) c Please read attached file License.txt c c=========== L-BFGS-B (version 3.0. April 25, 2011 =================== c c This is a modified version of L-BFGS-B. Minor changes in the updated c code appear preceded by a line comment as follows c c c-jlm-jn c c Major changes are described in the accompanying paper: c c Jorge Nocedal and Jose Luis Morales, Remark on "Algorithm 778: c L-BFGS-B: Fortran Subroutines for Large-Scale Bound Constrained c Optimization" (2011). To appear in ACM Transactions on c Mathematical Software, c c The paper describes an improvement and a correction to Algorithm 778. c It is shown that the performance of the algorithm can be improved c significantly by making a relatively simple modication to the subspace c minimization phase. The correction concerns an error caused by the use c of routine dpmeps to estimate machine precision. c c The total work space **wa** required by the new version is c c 2*m*n + 11m*m + 5*n + 8*m c c the old version required c c 2*m*n + 12m*m + 4*n + 12*m c c c J. Nocedal Department of Electrical Engineering and c Computer Science. c Northwestern University. Evanston, IL. USA c c c J.L Morales Departamento de Matematicas, c Instituto Tecnologico Autonomo de Mexico c Mexico D.F. Mexico. c c March 2011 c c============================================================================= subroutine setulb(n, m, x, l, u, nbd, f, g, factr, pgtol, wa, iwa, + itask, iprint, icsave, lsave, isave, dsave) logical lsave(4) integer n, m, iprint, itask, icsave, + nbd(n), iwa(3*n), isave(44) double precision f, factr, pgtol, x(n), l(n), u(n), g(n), c c-jlm-jn + wa(2*m*n + 5*n + 11*m*m + 8*m), dsave(29) c ************ c c Subroutine setulb c c This subroutine partitions the working arrays wa and iwa, and c then uses the limited memory BFGS method to solve the bound c constrained optimization problem by calling mainlb. c (The direct method will be used in the subspace minimization.) c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c x is a double precision array of dimension n. c On entry x is an approximation to the solution. c On exit x is the current approximation. c c l is a double precision array of dimension n. c On entry l is the lower bound on x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound on x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c f is a double precision variable. c On first entry f is unspecified. c On final exit f is the value of the function at x. c c g is a double precision array of dimension n. c On first entry g is unspecified. c On final exit g is the value of the gradient at x. c c factr is a double precision variable. c On entry factr >= 0 is specified by the user. The iteration c will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision, which is automatically c generated by the code. Typical values for factr: 1.d+12 for c low accuracy; 1.d+7 for moderate accuracy; 1.d+1 for extremely c high accuracy. c On exit factr is unchanged. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c On exit pgtol is unchanged. c c wa is a double precision working array of length c (2mmax + 5)nmax + 12mmax^2 + 12mmax. c c iwa is an integer working array of length 3nmax. c c itask is an integer indicating c the current job when entering and quitting this subroutine. c c iprint is an integer variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c icsave is a working integer c c lsave is a logical working array of dimension 4. c On exit with 'task' = NEW_X, the following information is c available: c If lsave(1) = .true. then the initial X has been replaced by c its projection in the feasible set; c If lsave(2) = .true. then the problem is constrained; c If lsave(3) = .true. then each variable has upper and lower c bounds; c c isave is an integer working array of dimension 44. c On exit with 'task' = NEW_X, the following information is c available: c isave(22) = the total number of intervals explored in the c search of Cauchy points; c isave(26) = the total number of skipped BFGS updates before c the current iteration; c isave(30) = the number of current iteration; c isave(31) = the total number of BFGS updates prior the current c iteration; c isave(33) = the number of intervals explored in the search of c Cauchy point in the current iteration; c isave(34) = the total number of function and gradient c evaluations; c isave(36) = the number of function value or gradient c evaluations in the current iteration; c if isave(37) = 0 then the subspace argmin is within the box; c if isave(37) = 1 then the subspace argmin is beyond the box; c isave(38) = the number of free variables in the current c iteration; c isave(39) = the number of active constraints in the current c iteration; c n + 1 - isave(40) = the number of variables leaving the set of c active constraints in the current iteration; c isave(41) = the number of variables entering the set of active c constraints in the current iteration. c c dsave is a double precision working array of dimension 29. c On exit with 'task' = NEW_X, the following information is c available: c dsave(1) = current 'theta' in the BFGS matrix; c dsave(2) = f(x) in the previous iteration; c dsave(3) = factr*epsmch; c dsave(4) = 2-norm of the line search direction vector; c dsave(5) = the machine precision epsmch generated by the code; c dsave(7) = the accumulated time spent on searching for c Cauchy points; c dsave(8) = the accumulated time spent on c subspace minimization; c dsave(9) = the accumulated time spent on line search; c dsave(11) = the slope of the line search function at c the current point of line search; c dsave(12) = the maximum relative step length imposed in c line search; c dsave(13) = the infinity norm of the projected gradient; c dsave(14) = the relative step length in the line search; c dsave(15) = the slope of the line search function at c the starting point of the line search; c dsave(16) = the square of the 2-norm of the line search c direction vector. c c Subprograms called: c c L-BFGS-B Library ... mainlb. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: a c limited memory FORTRAN code for solving bound constrained c optimization problems'', Tech. Report, NAM-11, EECS Department, c Northwestern University, 1994. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ c-jlm-jn integer lws,lr,lz,lt,ld,lxp,lwa, + lwy,lsy,lss,lwt,lwn,lsnd write(6,*) ' The incoming task no. is ', itask if ((itask .lt. 1) .or. (itask .gt. 26)) stop(999) c if (task .eq. 'START') then if (itask .eq. 2) then isave(1) = m*n isave(2) = m**2 isave(3) = 4*m**2 isave(4) = 1 ! ws m*n isave(5) = isave(4) + isave(1) ! wy m*n isave(6) = isave(5) + isave(1) ! wsy m**2 isave(7) = isave(6) + isave(2) ! wss m**2 isave(8) = isave(7) + isave(2) ! wt m**2 isave(9) = isave(8) + isave(2) ! wn 4*m**2 isave(10) = isave(9) + isave(3) ! wsnd 4*m**2 isave(11) = isave(10) + isave(3) ! wz n isave(12) = isave(11) + n ! wr n isave(13) = isave(12) + n ! wd n isave(14) = isave(13) + n ! wt n isave(15) = isave(14) + n ! wxp n isave(16) = isave(15) + n ! wa 8*m endif lws = isave(4) lwy = isave(5) lsy = isave(6) lss = isave(7) lwt = isave(8) lwn = isave(9) lsnd = isave(10) lz = isave(11) lr = isave(12) ld = isave(13) lt = isave(14) lxp = isave(15) lwa = isave(16) call mainlb(n,m,x,l,u,nbd,f,g,factr,pgtol, + wa(lws),wa(lwy),wa(lsy),wa(lss), wa(lwt), + wa(lwn),wa(lsnd),wa(lz),wa(lr),wa(ld),wa(lt),wa(lxp), + wa(lwa), + iwa(1),iwa(n+1),iwa(2*n+1),itask,iprint, + icsave,lsave,isave(22),dsave) write(6,*) " itask on return is ",itask return end c======================= The end of setulb ============================= c mainlb here subroutine mainlb(n, m, x, l, u, nbd, f, g, factr, pgtol, ws, wy, + sy, ss, wt, wn, snd, z, r, d, t, xp, wa, + index, iwhere, indx2, itask, + iprint, icsave, lsave, isave, dsave) implicit none logical lsave(4) integer n, m, iprint, itask, icsave, nbd(n), index(n), + iwhere(n), indx2(n), isave(23) double precision f, factr, pgtol, + x(n), l(n), u(n), g(n), z(n), r(n), d(n), t(n), c-jlm-jn + xp(n), + wa(8*m), + ws(n, m), wy(n, m), sy(m, m), ss(m, m), + wt(m, m), wn(2*m, 2*m), snd(2*m, 2*m), dsave(29) c ************ c c Subroutine mainlb c c This subroutine solves bound constrained optimization problems by c using the compact formula of the limited memory BFGS updates. c c n is an integer variable. c On entry n is the number of variables. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric c corrections allowed in the limited memory matrix. c On exit m is unchanged. c c x is a double precision array of dimension n. c On entry x is an approximation to the solution. c On exit x is the current approximation. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c f is a double precision variable. c On first entry f is unspecified. c On final exit f is the value of the function at x. c c g is a double precision array of dimension n. c On first entry g is unspecified. c On final exit g is the value of the gradient at x. c c factr is a double precision variable. c On entry factr >= 0 is specified by the user. The iteration c will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision, which is automatically c generated by the code. c On exit factr is unchanged. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c On exit pgtol is unchanged. c c ws, wy, sy, and wt are double precision working arrays used to c store the following information defining the limited memory c BFGS matrix: c ws, of dimension n x m, stores S, the matrix of s-vectors; c wy, of dimension n x m, stores Y, the matrix of y-vectors; c sy, of dimension m x m, stores S'Y; c ss, of dimension m x m, stores S'S; c yy, of dimension m x m, stores Y'Y; c wt, of dimension m x m, stores the Cholesky factorization c of (theta*S'S+LD^(-1)L'); see eq. c (2.26) in [3]. c c wn is a double precision working array of dimension 2m x 2m c used to store the LEL^T factorization of the indefinite matrix c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c where E = [-I 0] c [ 0 I] c c snd is a double precision working array of dimension 2m x 2m c used to store the lower triangular part of c N = [Y' ZZ'Y L_a'+R_z'] c [L_a +R_z S'AA'S ] c c z(n),r(n),d(n),t(n), xp(n),wa(8*m) are double precision working arrays. c z is used at different times to store the Cauchy point and c the Newton point. c xp is used to safeguard the projected Newton direction c c sg(m),sgo(m),yg(m),ygo(m) are double precision working arrays. c c index is an integer working array of dimension n. c In subroutine freev, index is used to store the free and fixed c variables at the Generalized Cauchy Point (GCP). c c iwhere is an integer working array of dimension n used to record c the status of the vector x for GCP computation. c iwhere(i)=0 or -3 if x(i) is free and has bounds, c 1 if x(i) is fixed at l(i), and l(i) .ne. u(i) c 2 if x(i) is fixed at u(i), and u(i) .ne. l(i) c 3 if x(i) is always fixed, i.e., u(i)=x(i)=l(i) c -1 if x(i) is always free, i.e., no bounds on it. c c indx2 is an integer working array of dimension n. c Within subroutine cauchy, indx2 corresponds to the array iorder. c In subroutine freev, a list of variables entering and leaving c the free set is stored in indx2, and it is passed on to c subroutine formk with this information. c c task is a working string of characters of length 60 indicating c the current job when entering and leaving this subroutine. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c icsave is a working integer c c lsave is a logical working array of dimension 4. c c isave is an integer working array of dimension 23. c c dsave is a double precision working array of dimension 29. c c c Subprograms called c c L-BFGS-B Library ... cauchy, subsm, lnsrlb, formk, c c errclb, prn1lb, prn2lb, prn3lb, active, projgr, c c freev, cmprlb, matupd, formt. c c Minpack2 Library ... timer c c Linpack Library ... dcopy, ddot. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c [3] R. Byrd, J. Nocedal and R. Schnabel "Representations of c Quasi-Newton Matrices and their use in Limited Memory Methods'', c Mathematical Programming 63 (1994), no. 4, pp. 129-156. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ logical prjctd,cnstnd,boxed,updatd,wrk character*3 word integer i,k,nintol,itfile,iback,nskip, + head,col,iter,itail,iupdat, + nseg,nfgv,info,ifun, + iword,nfree,nact,ileave,nenter double precision theta,fold,ddot,dr,rr,tol, + xstep,sbgnrm,ddum,dnorm,dtd,epsmch, + cpu1,cpu2,cachyt,sbtime,lnscht,time1,time2, + gd,gdold,stp,stpmx,time double precision one,zero parameter (one=1.0d0,zero=0.0d0) if (itask .eq. 2) then epsmch = epsilon(one) call timer(time1) c Initialize counters and scalars when task='START'. c for the limited memory BFGS matrices: col = 0 head = 1 theta = one iupdat = 0 updatd = .false. iback = 0 itail = 0 iword = 0 nact = 0 ileave = 0 nenter = 0 fold = zero dnorm = zero cpu1 = zero gd = zero stpmx = zero sbgnrm = zero stp = zero gdold = zero dtd = zero c for operation counts: iter = 0 nfgv = 0 nseg = 0 nintol = 0 nskip = 0 nfree = n ifun = 0 c for stopping tolerance: tol = factr*epsmch c for measuring running time: cachyt = 0 sbtime = 0 lnscht = 0 c 'word' records the status of subspace solutions. word = '---' c 'info' records the termination information. info = 0 itfile = 8 if (iprint .ge. 1) then c open a summary file 'iterate.dat' open (8, file = 'iterate.dat', status = 'unknown') endif c Check the input arguments for errors. call errclb(n,m,factr,l,u,nbd,itask,info,k) c ERROR return if ((itask .ge. 9) .and. (itask .le. 19)) then call prn3lb(n,x,f,itask,iprint,info,itfile, + iter,nfgv,nintol,nskip,nact,sbgnrm, + zero,nseg,word,iback,stp,xstep,k, + cachyt,sbtime,lnscht) return endif call prn1lb(n,m,l,u,x,iprint,itfile,epsmch) c Initialize iwhere & project x onto the feasible set. call active(n,l,u,nbd,x,iwhere,iprint,prjctd,cnstnd,boxed) c The end of the initialization. else c restore local variables. prjctd = lsave(1) cnstnd = lsave(2) boxed = lsave(3) updatd = lsave(4) nintol = isave(1) itfile = isave(3) iback = isave(4) nskip = isave(5) head = isave(6) col = isave(7) itail = isave(8) iter = isave(9) iupdat = isave(10) nseg = isave(12) nfgv = isave(13) info = isave(14) ifun = isave(15) iword = isave(16) nfree = isave(17) nact = isave(18) ileave = isave(19) nenter = isave(20) theta = dsave(1) fold = dsave(2) tol = dsave(3) dnorm = dsave(4) epsmch = dsave(5) cpu1 = dsave(6) cachyt = dsave(7) sbtime = dsave(8) lnscht = dsave(9) time1 = dsave(10) gd = dsave(11) stpmx = dsave(12) sbgnrm = dsave(13) stp = dsave(14) gdold = dsave(15) dtd = dsave(16) c After returning from the driver go to the point where execution c is to resume. c if (task(1:5) .eq. 'FG_LN') goto 666 if (itask .eq. 20) goto 666 c if (task(1:5) .eq. 'NEW_X') goto 777 if (itask .eq. 1) goto 777 c if (task(1:5) .eq. 'FG_ST') goto 111 if (itask .eq. 21) goto 111 c if (task(1:4) .eq. 'STOP') then c if (itask .eq. 3) then c if (task(7:9) .eq. 'CPU') then c NOTE: Cannot happen in R driver. c restore the previous iterate. c call dcopy(n,t,1,x,1) c call dcopy(n,r,1,g,1) c f = fold c endif c goto 999 c endif endif c Compute f0 and g0. c task = 'FG_START' itask = 21 c return to the driver to calculate f and g; reenter at 111. goto 1000 111 continue nfgv = 1 c Compute the infinity norm of the (-) projected gradient. call projgr(n,l,u,nbd,x,g,sbgnrm) if (iprint .ge. 1) then write (6,1002) iter,f,sbgnrm write (itfile,1003) iter,nfgv,sbgnrm,f endif if (sbgnrm .le. pgtol) then c terminate the algorithm. c task = 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL' itask = 7 goto 999 endif c ----------------- the beginning of the loop -------------------------- 222 continue if (iprint .ge. 99) write (6,1001) iter + 1 iword = -1 c if (.not. cnstnd .and. col .gt. 0) then c skip the search for GCP. call dcopy(n,x,1,z,1) wrk = updatd nseg = 0 goto 333 endif cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Compute the Generalized Cauchy Point (GCP). c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc call timer(cpu1) call cauchy(n,x,l,u,nbd,g,indx2,iwhere,t,d,z, + m,wy,ws,sy,wt,theta,col,head, + wa(1),wa(2*m+1),wa(4*m+1),wa(6*m+1),nseg, + iprint, sbgnrm, info, epsmch) if (info .ne. 0) then c singular triangular system detected; refresh the lbfgs memory. if(iprint .ge. 1) write (6, 1005) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. call timer(cpu2) cachyt = cachyt + cpu2 - cpu1 goto 222 endif call timer(cpu2) cachyt = cachyt + cpu2 - cpu1 nintol = nintol + nseg c Count the entering and leaving variables for iter > 0; c find the index set of free and active variables at the GCP. call freev(n,nfree,index,nenter,ileave,indx2, + iwhere,wrk,updatd,cnstnd,iprint,iter) nact = n - nfree 333 continue c If there are no free variables or B=theta*I, then c skip the subspace minimization. if (nfree .eq. 0 .or. col .eq. 0) goto 555 cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Subspace minimization. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc call timer(cpu1) c Form the LEL^T factorization of the indefinite c matrix K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] if (wrk) call formk(n,nfree,index,nenter,ileave,indx2,iupdat, + updatd,wn,snd,m,ws,wy,sy,theta,col,head,info) if (info .ne. 0) then c nonpositive definiteness in Cholesky factorization; c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1006) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. call timer(cpu2) sbtime = sbtime + cpu2 - cpu1 goto 222 endif c compute r=-Z'B(xcp-xk)-Z'g (using wa(2m+1)=W'(xcp-x) c from 'cauchy'). call cmprlb(n,m,x,g,ws,wy,sy,wt,z,r,wa,index, + theta,col,head,nfree,cnstnd,info) if (info .ne. 0) goto 444 c-jlm-jn call the direct method. call subsm( n, m, nfree, index, l, u, nbd, z, r, xp, ws, wy, + theta, x, g, col, head, iword, wa, wn, iprint, info) 444 continue if (info .ne. 0) then c singular triangular system detected; c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1005) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. call timer(cpu2) sbtime = sbtime + cpu2 - cpu1 goto 222 endif call timer(cpu2) sbtime = sbtime + cpu2 - cpu1 555 continue cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Line search and optimality tests. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c Generate the search direction d:=z-x. do 40 i = 1, n d(i) = z(i) - x(i) 40 continue call timer(cpu1) 666 continue call lnsrlb(n,l,u,nbd,x,f,fold,gd,gdold,g,d,r,t,z,stp,dnorm, + dtd,xstep,stpmx,iter,ifun,iback,nfgv,info,itask, + boxed,cnstnd,icsave,isave(22),dsave(17)) if (info .ne. 0 .or. iback .ge. 20) then c restore the previous iterate. call dcopy(n,t,1,x,1) call dcopy(n,r,1,g,1) f = fold if (col .eq. 0) then c abnormal termination. if (info .eq. 0) then info = -9 c restore the actual number of f and g evaluations etc. nfgv = nfgv - 1 ifun = ifun - 1 iback = iback - 1 endif c task = 'ABNORMAL_TERMINATION_IN_LNSRCH' itask = 5 iter = iter + 1 goto 999 else c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1008) if (info .eq. 0) nfgv = nfgv - 1 info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. c task = 'RESTART_FROM_LNSRCH' itask = 22 call timer(cpu2) lnscht = lnscht + cpu2 - cpu1 goto 222 endif c else if (task(1:5) .eq. 'FG_LN') then else if (itask .eq. 20) then c return to the driver for calculating f and g; reenter at 666. goto 1000 else c calculate and print out the quantities related to the new X. call timer(cpu2) lnscht = lnscht + cpu2 - cpu1 iter = iter + 1 c Compute the infinity norm of the projected (-)gradient. call projgr(n,l,u,nbd,x,g,sbgnrm) c Print iteration information. call prn2lb(n,x,f,g,iprint,itfile,iter,nfgv,nact, + sbgnrm,nseg,word,iword,iback,stp,xstep) goto 1000 endif 777 continue c Test for termination. if (sbgnrm .le. pgtol) then c terminate the algorithm. c task = 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL' itask = 7 goto 999 endif ddum = max(abs(fold), abs(f), one) if ((fold - f) .le. tol*ddum) then c terminate the algorithm. c task = 'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH' itask = 8 if (iback .ge. 10) info = -5 c i.e., to issue a warning if iback>10 in the line search. goto 999 endif c Compute d=newx-oldx, r=newg-oldg, rr=y'y and dr=y's. do 42 i = 1, n r(i) = g(i) - r(i) 42 continue rr = ddot(n,r,1,r,1) if (stp .eq. one) then dr = gd - gdold ddum = -gdold else dr = (gd - gdold)*stp call dscal(n,stp,d,1) ddum = -gdold*stp endif if (dr .le. epsmch*ddum) then c skip the L-BFGS update. nskip = nskip + 1 updatd = .false. if (iprint .ge. 1) write (6,1004) dr, ddum goto 888 endif cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Update the L-BFGS matrix. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc updatd = .true. iupdat = iupdat + 1 c Update matrices WS and WY and form the middle matrix in B. call matupd(n,m,ws,wy,sy,ss,d,r,itail, + iupdat,col,head,theta,rr,dr,stp,dtd) c Form the upper half of the pds T = theta*SS + L*D^(-1)*L'; c Store T in the upper triangular of the array wt; c Cholesky factorize T to J*J' with c J' stored in the upper triangular of wt. call formt(m,wt,sy,ss,col,theta,info) if (info .ne. 0) then c nonpositive definiteness in Cholesky factorization; c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1007) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. goto 222 endif c Now the inverse of the middle matrix in B is c [ D^(1/2) O ] [ -D^(1/2) D^(-1/2)*L' ] c [ -L*D^(-1/2) J ] [ 0 J' ] 888 continue c -------------------- the end of the loop ----------------------------- goto 222 999 continue call timer(time2) time = time2 - time1 call prn3lb(n,x,f,itask,iprint,info,itfile, + iter,nfgv,nintol,nskip,nact,sbgnrm, + time,nseg,word,iback,stp,xstep,k, + cachyt,sbtime,lnscht) 1000 continue c Save local variables. lsave(1) = prjctd lsave(2) = cnstnd lsave(3) = boxed lsave(4) = updatd isave(1) = nintol isave(3) = itfile isave(4) = iback isave(5) = nskip isave(6) = head isave(7) = col isave(8) = itail isave(9) = iter isave(10) = iupdat isave(12) = nseg isave(13) = nfgv isave(14) = info isave(15) = ifun isave(16) = iword isave(17) = nfree isave(18) = nact isave(19) = ileave isave(20) = nenter dsave(1) = theta dsave(2) = fold dsave(3) = tol dsave(4) = dnorm dsave(5) = epsmch dsave(6) = cpu1 dsave(7) = cachyt dsave(8) = sbtime dsave(9) = lnscht dsave(10) = time1 dsave(11) = gd dsave(12) = stpmx dsave(13) = sbgnrm dsave(14) = stp dsave(15) = gdold dsave(16) = dtd 1001 format (//,'ITERATION ',i5) 1002 format + (/,'At iterate',i5,4x,'f= ',1p,d12.5,4x,'|proj g|= ',1p,d12.5) 1003 format (2(1x,i4),5x,'-',5x,'-',3x,'-',5x,'-',5x,'-',8x,'-',3x, + 1p,2(1x,d10.3)) 1004 format (' ys=',1p,e10.3,' -gs=',1p,e10.3,' BFGS update SKIPPED') 1005 format (/, +' Singular triangular system detected;',/, +' refresh the lbfgs memory and restart the iteration.') 1006 format (/, +' Nonpositive definiteness in Cholesky factorization in formk;',/, +' refresh the lbfgs memory and restart the iteration.') 1007 format (/, +' Nonpositive definiteness in Cholesky factorization in formt;',/, +' refresh the lbfgs memory and restart the iteration.') 1008 format (/, +' Bad direction in the line search;',/, +' refresh the lbfgs memory and restart the iteration.') return end c======================= The end of mainlb ============================= subroutine active(n, l, u, nbd, x, iwhere, iprint, + prjctd, cnstnd, boxed) logical prjctd, cnstnd, boxed integer n, iprint, nbd(n), iwhere(n) double precision x(n), l(n), u(n) c ************ c c Subroutine active c c This subroutine initializes iwhere and projects the initial x to c the feasible set if necessary. c c iwhere is an integer array of dimension n. c On entry iwhere is unspecified. c On exit iwhere(i)=-1 if x(i) has no bounds c 3 if l(i)=u(i) c 0 otherwise. c In cauchy, iwhere is given finer gradations. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer nbdd,i double precision zero parameter (zero=0.0d0) c Initialize nbdd, prjctd, cnstnd and boxed. nbdd = 0 prjctd = .false. cnstnd = .false. boxed = .true. c Project the initial x to the easible set if necessary. do 10 i = 1, n if (nbd(i) .gt. 0) then if (nbd(i) .le. 2 .and. x(i) .le. l(i)) then if (x(i) .lt. l(i)) then prjctd = .true. x(i) = l(i) endif nbdd = nbdd + 1 else if (nbd(i) .ge. 2 .and. x(i) .ge. u(i)) then if (x(i) .gt. u(i)) then prjctd = .true. x(i) = u(i) endif nbdd = nbdd + 1 endif endif 10 continue c Initialize iwhere and assign values to cnstnd and boxed. do 20 i = 1, n if (nbd(i) .ne. 2) boxed = .false. if (nbd(i) .eq. 0) then c this variable is always free iwhere(i) = -1 c otherwise set x(i)=mid(x(i), u(i), l(i)). else cnstnd = .true. if (nbd(i) .eq. 2 .and. u(i) - l(i) .le. zero) then c this variable is always fixed iwhere(i) = 3 else iwhere(i) = 0 endif endif 20 continue if (iprint .ge. 0) then if (prjctd) write (6,*) + 'The initial X is infeasible. Restart with its projection.' if (.not. cnstnd) + write (6,*) 'This problem is unconstrained.' endif if (iprint .gt. 0) write (6,1001) nbdd 1001 format (/,'At X0 ',i9,' variables are exactly at the bounds') return end c======================= The end of active ============================= subroutine bmv(m, sy, wt, col, v, p, info) integer m, col, info double precision sy(m, m), wt(m, m), v(2*col), p(2*col) c ************ c c Subroutine bmv c c This subroutine computes the product of the 2m x 2m middle matrix c in the compact L-BFGS formula of B and a 2m vector v; c it returns the product in p. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c sy is a double precision array of dimension m x m. c On entry sy specifies the matrix S'Y. c On exit sy is unchanged. c c wt is a double precision array of dimension m x m. c On entry wt specifies the upper triangular matrix J' which is c the Cholesky factor of (thetaS'S+LD^(-1)L'). c On exit wt is unchanged. c c col is an integer variable. c On entry col specifies the number of s-vectors (or y-vectors) c stored in the compact L-BFGS formula. c On exit col is unchanged. c c v is a double precision array of dimension 2col. c On entry v specifies vector v. c On exit v is unchanged. c c p is a double precision array of dimension 2col. c On entry p is unspecified. c On exit p is the product Mv. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return, c = nonzero for abnormal return when the system c to be solved by dtrsl is singular. c c Subprograms called: c c Linpack ... dtrsl. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,k,i2 double precision sum if (col .eq. 0) return c PART I: solve [ D^(1/2) O ] [ p1 ] = [ v1 ] c [ -L*D^(-1/2) J ] [ p2 ] [ v2 ]. c solve Jp2=v2+LD^(-1)v1. p(col + 1) = v(col + 1) do 20 i = 2, col i2 = col + i sum = 0.0d0 do 10 k = 1, i - 1 sum = sum + sy(i,k)*v(k)/sy(k,k) 10 continue p(i2) = v(i2) + sum 20 continue c Solve the triangular system call dtrsl(wt,m,col,p(col+1),11,info) if (info .ne. 0) return c solve D^(1/2)p1=v1. do 30 i = 1, col p(i) = v(i)/sqrt(sy(i,i)) 30 continue c PART II: solve [ -D^(1/2) D^(-1/2)*L' ] [ p1 ] = [ p1 ] c [ 0 J' ] [ p2 ] [ p2 ]. c solve J^Tp2=p2. call dtrsl(wt,m,col,p(col+1),01,info) if (info .ne. 0) return c compute p1=-D^(-1/2)(p1-D^(-1/2)L'p2) c =-D^(-1/2)p1+D^(-1)L'p2. do 40 i = 1, col p(i) = -p(i)/sqrt(sy(i,i)) 40 continue do 60 i = 1, col sum = 0.d0 do 50 k = i + 1, col sum = sum + sy(k,i)*p(col+k)/sy(i,i) 50 continue p(i) = p(i) + sum 60 continue return end c======================== The end of bmv =============================== subroutine cauchy(n, x, l, u, nbd, g, iorder, iwhere, t, d, xcp, + m, wy, ws, sy, wt, theta, col, head, p, c, wbp, + v, nseg, iprint, sbgnrm, info, epsmch) implicit none integer n, m, head, col, nseg, iprint, info, + nbd(n), iorder(n), iwhere(n) double precision theta, epsmch, + x(n), l(n), u(n), g(n), t(n), d(n), xcp(n), + wy(n, col), ws(n, col), sy(m, m), + wt(m, m), p(2*m), c(2*m), wbp(2*m), v(2*m) c ************ c c Subroutine cauchy c c For given x, l, u, g (with sbgnrm > 0), and a limited memory c BFGS matrix B defined in terms of matrices WY, WS, WT, and c scalars head, col, and theta, this subroutine computes the c generalized Cauchy point (GCP), defined as the first local c minimizer of the quadratic c c Q(x + s) = g's + 1/2 s'Bs c c along the projected gradient direction P(x-tg,l,u). c The routine returns the GCP in xcp. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c x is a double precision array of dimension n. c On entry x is the starting point for the GCP computation. c On exit x is unchanged. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c g is a double precision array of dimension n. c On entry g is the gradient of f(x). g must be a nonzero vector. c On exit g is unchanged. c c iorder is an integer working array of dimension n. c iorder will be used to store the breakpoints in the piecewise c linear path and free variables encountered. On exit, c iorder(1),...,iorder(nleft) are indices of breakpoints c which have not been encountered; c iorder(nleft+1),...,iorder(nbreak) are indices of c encountered breakpoints; and c iorder(nfree),...,iorder(n) are indices of variables which c have no bound constraits along the search direction. c c iwhere is an integer array of dimension n. c On entry iwhere indicates only the permanently fixed (iwhere=3) c or free (iwhere= -1) components of x. c On exit iwhere records the status of the current x variables. c iwhere(i)=-3 if x(i) is free and has bounds, but is not moved c 0 if x(i) is free and has bounds, and is moved c 1 if x(i) is fixed at l(i), and l(i) .ne. u(i) c 2 if x(i) is fixed at u(i), and u(i) .ne. l(i) c 3 if x(i) is always fixed, i.e., u(i)=x(i)=l(i) c -1 if x(i) is always free, i.e., it has no bounds. c c t is a double precision working array of dimension n. c t will be used to store the break points. c c d is a double precision array of dimension n used to store c the Cauchy direction P(x-tg)-x. c c xcp is a double precision array of dimension n used to return the c GCP on exit. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c ws, wy, sy, and wt are double precision arrays. c On entry they store information that defines the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c sy(m,m) stores S'Y; c wt(m,m) stores the c Cholesky factorization of (theta*S'S+LD^(-1)L'). c On exit these arrays are unchanged. c c theta is a double precision variable. c On entry theta is the scaling factor specifying B_0 = theta I. c On exit theta is unchanged. c c col is an integer variable. c On entry col is the actual number of variable metric c corrections stored so far. c On exit col is unchanged. c c head is an integer variable. c On entry head is the location of the first s-vector (or y-vector) c in S (or Y). c On exit col is unchanged. c c p is a double precision working array of dimension 2m. c p will be used to store the vector p = W^(T)d. c c c is a double precision working array of dimension 2m. c c will be used to store the vector c = W^(T)(xcp-x). c c wbp is a double precision working array of dimension 2m. c wbp will be used to store the row of W corresponding c to a breakpoint. c c v is a double precision working array of dimension 2m. c c nseg is an integer variable. c On exit nseg records the number of quadratic segments explored c in searching for the GCP. c c sg and yg are double precision arrays of dimension m. c On entry sg and yg store S'g and Y'g correspondingly. c On exit they are unchanged. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c sbgnrm is a double precision variable. c On entry sbgnrm is the norm of the projected gradient at x. c On exit sbgnrm is unchanged. c c info is an integer variable. c On entry info is 0. c On exit info = 0 for normal return, c = nonzero for abnormal return when the the system c used in routine bmv is singular. c c Subprograms called: c c L-BFGS-B Library ... hpsolb, bmv. c c Linpack ... dscal dcopy, daxpy. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ logical xlower,xupper,bnded integer i,j,col2,nfree,nbreak,pointr, + ibp,nleft,ibkmin,iter double precision f1,f2,dt,dtm,tsum,dibp,zibp,dibp2,bkmin, + tu,tl,wmc,wmp,wmw,ddot,tj,tj0,neggi,sbgnrm, + f2_org double precision one,zero parameter (one=1.0d0,zero=0.0d0) c Check the status of the variables, reset iwhere(i) if necessary; c compute the Cauchy direction d and the breakpoints t; initialize c the derivative f1 and the vector p = W'd (for theta = 1). if (sbgnrm .le. zero) then if (iprint .ge. 0) write (6,*) 'Subgnorm = 0. GCP = X.' call dcopy(n,x,1,xcp,1) return endif bnded = .true. nfree = n + 1 nbreak = 0 ibkmin = 0 bkmin = zero col2 = 2*col f1 = zero if (iprint .ge. 99) write (6,3010) c We set p to zero and build it up as we determine d. do 20 i = 1, col2 p(i) = zero 20 continue c In the following loop we determine for each variable its bound c status and its breakpoint, and update p accordingly. c Smallest breakpoint is identified. do 50 i = 1, n neggi = -g(i) if (iwhere(i) .ne. 3 .and. iwhere(i) .ne. -1) then c if x(i) is not a constant and has bounds, c compute the difference between x(i) and its bounds. if (nbd(i) .le. 2) tl = x(i) - l(i) if (nbd(i) .ge. 2) tu = u(i) - x(i) c If a variable is close enough to a bound c we treat it as at bound. xlower = nbd(i) .le. 2 .and. tl .le. zero xupper = nbd(i) .ge. 2 .and. tu .le. zero c reset iwhere(i). iwhere(i) = 0 if (xlower) then if (neggi .le. zero) iwhere(i) = 1 else if (xupper) then if (neggi .ge. zero) iwhere(i) = 2 else if (abs(neggi) .le. zero) iwhere(i) = -3 endif endif pointr = head if (iwhere(i) .ne. 0 .and. iwhere(i) .ne. -1) then d(i) = zero else d(i) = neggi f1 = f1 - neggi*neggi c calculate p := p - W'e_i* (g_i). do 40 j = 1, col p(j) = p(j) + wy(i,pointr)* neggi p(col + j) = p(col + j) + ws(i,pointr)*neggi pointr = mod(pointr,m) + 1 40 continue if (nbd(i) .le. 2 .and. nbd(i) .ne. 0 + .and. neggi .lt. zero) then c x(i) + d(i) is bounded; compute t(i). nbreak = nbreak + 1 iorder(nbreak) = i t(nbreak) = tl/(-neggi) if (nbreak .eq. 1 .or. t(nbreak) .lt. bkmin) then bkmin = t(nbreak) ibkmin = nbreak endif else if (nbd(i) .ge. 2 .and. neggi .gt. zero) then c x(i) + d(i) is bounded; compute t(i). nbreak = nbreak + 1 iorder(nbreak) = i t(nbreak) = tu/neggi if (nbreak .eq. 1 .or. t(nbreak) .lt. bkmin) then bkmin = t(nbreak) ibkmin = nbreak endif else c x(i) + d(i) is not bounded. nfree = nfree - 1 iorder(nfree) = i if (abs(neggi) .gt. zero) bnded = .false. endif endif 50 continue c The indices of the nonzero components of d are now stored c in iorder(1),...,iorder(nbreak) and iorder(nfree),...,iorder(n). c The smallest of the nbreak breakpoints is in t(ibkmin)=bkmin. if (theta .ne. one) then c complete the initialization of p for theta not= one. call dscal(col,theta,p(col+1),1) endif c Initialize GCP xcp = x. call dcopy(n,x,1,xcp,1) if (nbreak .eq. 0 .and. nfree .eq. n + 1) then c is a zero vector, return with the initial xcp as GCP. if (iprint .gt. 100) write (6,1010) (xcp(i), i = 1, n) return endif c Initialize c = W'(xcp - x) = 0. do 60 j = 1, col2 c(j) = zero 60 continue c Initialize derivative f2. f2 = -theta*f1 f2_org = f2 if (col .gt. 0) then call bmv(m,sy,wt,col,p,v,info) if (info .ne. 0) return f2 = f2 - ddot(col2,v,1,p,1) endif dtm = -f1/f2 tsum = zero nseg = 1 if (iprint .ge. 99) + write (6,*) 'There are ',nbreak,' breakpoints ' c If there are no breakpoints, locate the GCP and return. if (nbreak .eq. 0) goto 888 nleft = nbreak iter = 1 tj = zero c------------------- the beginning of the loop ------------------------- 777 continue c Find the next smallest breakpoint; c compute dt = t(nleft) - t(nleft + 1). tj0 = tj if (iter .eq. 1) then c Since we already have the smallest breakpoint we need not do c heapsort yet. Often only one breakpoint is used and the c cost of heapsort is avoided. tj = bkmin ibp = iorder(ibkmin) else if (iter .eq. 2) then c Replace the already used smallest breakpoint with the c breakpoint numbered nbreak > nlast, before heapsort call. if (ibkmin .ne. nbreak) then t(ibkmin) = t(nbreak) iorder(ibkmin) = iorder(nbreak) endif c Update heap structure of breakpoints c (if iter=2, initialize heap). endif call hpsolb(nleft,t,iorder,iter-2) tj = t(nleft) ibp = iorder(nleft) endif dt = tj - tj0 if (dt .ne. zero .and. iprint .ge. 100) then write (6,4011) nseg,f1,f2 write (6,5010) dt write (6,6010) dtm endif c If a minimizer is within this interval, locate the GCP and return. if (dtm .lt. dt) goto 888 c Otherwise fix one variable and c reset the corresponding component of d to zero. tsum = tsum + dt nleft = nleft - 1 iter = iter + 1 dibp = d(ibp) d(ibp) = zero if (dibp .gt. zero) then zibp = u(ibp) - x(ibp) xcp(ibp) = u(ibp) iwhere(ibp) = 2 else zibp = l(ibp) - x(ibp) xcp(ibp) = l(ibp) iwhere(ibp) = 1 endif if (iprint .ge. 100) write (6,*) 'Variable ',ibp,' is fixed.' if (nleft .eq. 0 .and. nbreak .eq. n) then c all n variables are fixed, c return with xcp as GCP. dtm = dt goto 999 endif c Update the derivative information. nseg = nseg + 1 dibp2 = dibp**2 c Update f1 and f2. c temporarily set f1 and f2 for col=0. f1 = f1 + dt*f2 + dibp2 - theta*dibp*zibp f2 = f2 - theta*dibp2 if (col .gt. 0) then c update c = c + dt*p. call daxpy(col2,dt,p,1,c,1) c choose wbp, c the row of W corresponding to the breakpoint encountered. pointr = head do 70 j = 1,col wbp(j) = wy(ibp,pointr) wbp(col + j) = theta*ws(ibp,pointr) pointr = mod(pointr,m) + 1 70 continue c compute (wbp)Mc, (wbp)Mp, and (wbp)M(wbp)'. call bmv(m,sy,wt,col,wbp,v,info) if (info .ne. 0) return wmc = ddot(col2,c,1,v,1) wmp = ddot(col2,p,1,v,1) wmw = ddot(col2,wbp,1,v,1) c update p = p - dibp*wbp. call daxpy(col2,-dibp,wbp,1,p,1) c complete updating f1 and f2 while col > 0. f1 = f1 + dibp*wmc f2 = f2 + 2.0d0*dibp*wmp - dibp2*wmw endif f2 = max(epsmch*f2_org,f2) if (nleft .gt. 0) then dtm = -f1/f2 goto 777 c to repeat the loop for unsearched intervals. else if(bnded) then f1 = zero f2 = zero dtm = zero else dtm = -f1/f2 endif c------------------- the end of the loop ------------------------------- 888 continue if (iprint .ge. 99) then write (6,*) write (6,*) 'GCP found in this segment' write (6,4010) nseg,f1,f2 write (6,6010) dtm endif if (dtm .le. zero) dtm = zero tsum = tsum + dtm c Move free variables (i.e., the ones w/o breakpoints) and c the variables whose breakpoints haven't been reached. call daxpy(n,tsum,d,1,xcp,1) 999 continue c Update c = c + dtm*p = W'(x^c - x) c which will be used in computing r = Z'(B(x^c - x) + g). if (col .gt. 0) call daxpy(col2,dtm,p,1,c,1) if (iprint .gt. 100) write (6,1010) (xcp(i),i = 1,n) if (iprint .ge. 99) write (6,2010) 1010 format ('Cauchy X = ',/,(4x,1p,6(1x,d11.4))) 2010 format (/,'---------------- exit CAUCHY----------------------',/) 3010 format (/,'---------------- CAUCHY entered-------------------') 4010 format ('Piece ',i3,' --f1, f2 at start point ',1p,2(1x,d11.4)) 4011 format (/,'Piece ',i3,' --f1, f2 at start point ', + 1p,2(1x,d11.4)) 5010 format ('Distance to the next break point = ',1p,d11.4) 6010 format ('Distance to the stationary point = ',1p,d11.4) return end c====================== The end of cauchy ============================== subroutine cmprlb(n, m, x, g, ws, wy, sy, wt, z, r, wa, index, + theta, col, head, nfree, cnstnd, info) logical cnstnd integer n, m, col, head, nfree, info, index(n) double precision theta, + x(n), g(n), z(n), r(n), wa(4*m), + ws(n, m), wy(n, m), sy(m, m), wt(m, m) c ************ c c Subroutine cmprlb c c This subroutine computes r=-Z'B(xcp-xk)-Z'g by using c wa(2m+1)=W'(xcp-x) from subroutine cauchy. c c Subprograms called: c c L-BFGS-B Library ... bmv. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,j,k,pointr double precision a1,a2 if (.not. cnstnd .and. col .gt. 0) then do 26 i = 1, n r(i) = -g(i) 26 continue else do 30 i = 1, nfree k = index(i) r(i) = -theta*(z(k) - x(k)) - g(k) 30 continue call bmv(m,sy,wt,col,wa(2*m+1),wa(1),info) if (info .ne. 0) then info = -8 return endif pointr = head do 34 j = 1, col a1 = wa(j) a2 = theta*wa(col + j) do 32 i = 1, nfree k = index(i) r(i) = r(i) + wy(k,pointr)*a1 + ws(k,pointr)*a2 32 continue pointr = mod(pointr,m) + 1 34 continue endif return end c======================= The end of cmprlb ============================= subroutine errclb(n, m, factr, l, u, nbd, itask, info, k) c character*255 task integer n, m, info, k, nbd(n) double precision factr, l(n), u(n) c ************ c c Subroutine errclb c c This subroutine checks the validity of the input data. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i double precision one,zero parameter (one=1.0d0,zero=0.0d0) c Check the input arguments for errors. c if (n .le. 0) task = 'ERROR: N .LE. 0' if (n .le. 0) then write (6, *) ' ERROR: N .LE. 0' stop endif c if (m .le. 0) task = 'ERROR: M .LE. 0' if (m .le. 0) then write (6, *) ' ERROR: M .LE. 0' stop endif c if (factr .lt. zero) task = 'ERROR: FACTR .LT. 0' if (factr .le. zero) then write (6, *) ' ERROR: FACTR .LT. 0' stop endif c Check the validity of the arrays nbd(i), u(i), and l(i). do 10 i = 1, n if (nbd(i) .lt. 0 .or. nbd(i) .gt. 3) then c return c task = 'ERROR: INVALID NBD' itask = 12 info = -6 k = i endif if (nbd(i) .eq. 2) then if (l(i) .gt. u(i)) then c return c task = 'ERROR: NO FEASIBLE SOLUTION' itask = 14 info = -7 k = i endif endif 10 continue return end c======================= The end of errclb ============================= subroutine formk(n, nsub, ind, nenter, ileave, indx2, iupdat, + updatd, wn, wn1, m, ws, wy, sy, theta, col, + head, info) integer n, nsub, m, col, head, nenter, ileave, iupdat, + info, ind(n), indx2(n) double precision theta, wn(2*m, 2*m), wn1(2*m, 2*m), + ws(n, m), wy(n, m), sy(m, m) logical updatd c ************ c c Subroutine formk c c This subroutine forms the LEL^T factorization of the indefinite c c matrix K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] c The matrix K can be shown to be equal to the matrix M^[-1]N c occurring in section 5.1 of [1], as well as to the matrix c Mbar^[-1] Nbar in section 5.3. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c nsub is an integer variable c On entry nsub is the number of subspace variables in free set. c On exit nsub is not changed. c c ind is an integer array of dimension nsub. c On entry ind specifies the indices of subspace variables. c On exit ind is unchanged. c c nenter is an integer variable. c On entry nenter is the number of variables entering the c free set. c On exit nenter is unchanged. c c ileave is an integer variable. c On entry indx2(ileave),...,indx2(n) are the variables leaving c the free set. c On exit ileave is unchanged. c c indx2 is an integer array of dimension n. c On entry indx2(1),...,indx2(nenter) are the variables entering c the free set, while indx2(ileave),...,indx2(n) are the c variables leaving the free set. c On exit indx2 is unchanged. c c iupdat is an integer variable. c On entry iupdat is the total number of BFGS updates made so far. c On exit iupdat is unchanged. c c updatd is a logical variable. c On entry 'updatd' is true if the L-BFGS matrix is updatd. c On exit 'updatd' is unchanged. c c wn is a double precision array of dimension 2m x 2m. c On entry wn is unspecified. c On exit the upper triangle of wn stores the LEL^T factorization c of the 2*col x 2*col indefinite matrix c [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c wn1 is a double precision array of dimension 2m x 2m. c On entry wn1 stores the lower triangular part of c [Y' ZZ'Y L_a'+R_z'] c [L_a+R_z S'AA'S ] c in the previous iteration. c On exit wn1 stores the corresponding updated matrices. c The purpose of wn1 is just to store these inner products c so they can be easily updated and inserted into wn. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c ws, wy, sy, and wtyy are double precision arrays; c theta is a double precision variable; c col is an integer variable; c head is an integer variable. c On entry they store the information defining the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c sy(m,m) stores S'Y; c wtyy(m,m) stores the Cholesky factorization c of (theta*S'S+LD^(-1)L') c theta is the scaling factor specifying B_0 = theta I; c col is the number of variable metric corrections stored; c head is the location of the 1st s- (or y-) vector in S (or Y). c On exit they are unchanged. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return; c = -1 when the 1st Cholesky factorization failed; c = -2 when the 2st Cholesky factorization failed. c c Subprograms called: c c Linpack ... dcopy, dpofa, dtrsl. c c c References: c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: a c limited memory FORTRAN code for solving bound constrained c optimization problems'', Tech. Report, NAM-11, EECS Department, c Northwestern University, 1994. c c (Postscript files of these papers are available via anonymous c ftp to eecs.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer m2,ipntr,jpntr,iy,is,jy,js,is1,js1,k1,i,k, + col2,pbegin,pend,dbegin,dend,upcl double precision ddot,temp1,temp2,temp3,temp4 double precision one,zero parameter (one=1.0d0,zero=0.0d0) c Form the lower triangular part of c WN1 = [Y' ZZ'Y L_a'+R_z'] c [L_a+R_z S'AA'S ] c where L_a is the strictly lower triangular part of S'AA'Y c R_z is the upper triangular part of S'ZZ'Y. if (updatd) then if (iupdat .gt. m) then c shift old part of WN1. do 10 jy = 1, m - 1 js = m + jy call dcopy(m-jy,wn1(jy+1,jy+1),1,wn1(jy,jy),1) call dcopy(m-jy,wn1(js+1,js+1),1,wn1(js,js),1) call dcopy(m-1,wn1(m+2,jy+1),1,wn1(m+1,jy),1) 10 continue endif c put new rows in blocks (1,1), (2,1) and (2,2). pbegin = 1 pend = nsub dbegin = nsub + 1 dend = n iy = col is = m + col ipntr = head + col - 1 if (ipntr .gt. m) ipntr = ipntr - m jpntr = head do 20 jy = 1, col js = m + jy temp1 = zero temp2 = zero temp3 = zero c compute element jy of row 'col' of Y'ZZ'Y do 15 k = pbegin, pend k1 = ind(k) temp1 = temp1 + wy(k1,ipntr)*wy(k1,jpntr) 15 continue c compute elements jy of row 'col' of L_a and S'AA'S do 16 k = dbegin, dend k1 = ind(k) temp2 = temp2 + ws(k1,ipntr)*ws(k1,jpntr) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 16 continue wn1(iy,jy) = temp1 wn1(is,js) = temp2 wn1(is,jy) = temp3 jpntr = mod(jpntr,m) + 1 20 continue c put new column in block (2,1). jy = col jpntr = head + col - 1 if (jpntr .gt. m) jpntr = jpntr - m ipntr = head do 30 i = 1, col is = m + i temp3 = zero c compute element i of column 'col' of R_z do 25 k = pbegin, pend k1 = ind(k) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 25 continue ipntr = mod(ipntr,m) + 1 wn1(is,jy) = temp3 30 continue upcl = col - 1 else upcl = col endif c modify the old parts in blocks (1,1) and (2,2) due to changes c in the set of free variables. ipntr = head do 45 iy = 1, upcl is = m + iy jpntr = head do 40 jy = 1, iy js = m + jy temp1 = zero temp2 = zero temp3 = zero temp4 = zero do 35 k = 1, nenter k1 = indx2(k) temp1 = temp1 + wy(k1,ipntr)*wy(k1,jpntr) temp2 = temp2 + ws(k1,ipntr)*ws(k1,jpntr) 35 continue do 36 k = ileave, n k1 = indx2(k) temp3 = temp3 + wy(k1,ipntr)*wy(k1,jpntr) temp4 = temp4 + ws(k1,ipntr)*ws(k1,jpntr) 36 continue wn1(iy,jy) = wn1(iy,jy) + temp1 - temp3 wn1(is,js) = wn1(is,js) - temp2 + temp4 jpntr = mod(jpntr,m) + 1 40 continue ipntr = mod(ipntr,m) + 1 45 continue c modify the old parts in block (2,1). ipntr = head do 60 is = m + 1, m + upcl jpntr = head do 55 jy = 1, upcl temp1 = zero temp3 = zero do 50 k = 1, nenter k1 = indx2(k) temp1 = temp1 + ws(k1,ipntr)*wy(k1,jpntr) 50 continue do 51 k = ileave, n k1 = indx2(k) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 51 continue if (is .le. jy + m) then wn1(is,jy) = wn1(is,jy) + temp1 - temp3 else wn1(is,jy) = wn1(is,jy) - temp1 + temp3 endif jpntr = mod(jpntr,m) + 1 55 continue ipntr = mod(ipntr,m) + 1 60 continue c Form the upper triangle of WN = [D+Y' ZZ'Y/theta -L_a'+R_z' ] c [-L_a +R_z S'AA'S*theta] m2 = 2*m do 70 iy = 1, col is = col + iy is1 = m + iy do 65 jy = 1, iy js = col + jy js1 = m + jy wn(jy,iy) = wn1(iy,jy)/theta wn(js,is) = wn1(is1,js1)*theta 65 continue do 66 jy = 1, iy - 1 wn(jy,is) = -wn1(is1,jy) 66 continue do 67 jy = iy, col wn(jy,is) = wn1(is1,jy) 67 continue wn(iy,iy) = wn(iy,iy) + sy(iy,iy) 70 continue c Form the upper triangle of WN= [ LL' L^-1(-L_a'+R_z')] c [(-L_a +R_z)L'^-1 S'AA'S*theta ] c first Cholesky factor (1,1) block of wn to get LL' c with L' stored in the upper triangle of wn. call dpofa(wn,m2,col,info) if (info .ne. 0) then info = -1 return endif c then form L^-1(-L_a'+R_z') in the (1,2) block. col2 = 2*col do 71 js = col+1 ,col2 call dtrsl(wn,m2,col,wn(1,js),11,info) 71 continue c Form S'AA'S*theta + (L^-1(-L_a'+R_z'))'L^-1(-L_a'+R_z') in the c upper triangle of (2,2) block of wn. do 72 is = col+1, col2 do 74 js = is, col2 wn(is,js) = wn(is,js) + ddot(col,wn(1,is),1,wn(1,js),1) 74 continue 72 continue c Cholesky factorization of (2,2) block of wn. call dpofa(wn(col+1,col+1),m2,col,info) if (info .ne. 0) then info = -2 return endif return end c======================= The end of formk ============================== subroutine formt(m, wt, sy, ss, col, theta, info) integer m, col, info double precision theta, wt(m, m), sy(m, m), ss(m, m) c ************ c c Subroutine formt c c This subroutine forms the upper half of the pos. def. and symm. c T = theta*SS + L*D^(-1)*L', stores T in the upper triangle c of the array wt, and performs the Cholesky factorization of T c to produce J*J', with J' stored in the upper triangle of wt. c c Subprograms called: c c Linpack ... dpofa. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,j,k,k1 double precision ddum double precision zero parameter (zero=0.0d0) c Form the upper half of T = theta*SS + L*D^(-1)*L', c store T in the upper triangle of the array wt. do 52 j = 1, col wt(1,j) = theta*ss(1,j) 52 continue do 55 i = 2, col do 54 j = i, col k1 = min(i,j) - 1 ddum = zero do 53 k = 1, k1 ddum = ddum + sy(i,k)*sy(j,k)/sy(k,k) 53 continue wt(i,j) = ddum + theta*ss(i,j) 54 continue 55 continue c Cholesky factorize T to J*J' with c J' stored in the upper triangle of wt. call dpofa(wt,m,col,info) if (info .ne. 0) then info = -3 endif return end c======================= The end of formt ============================== subroutine freev(n, nfree, index, nenter, ileave, indx2, + iwhere, wrk, updatd, cnstnd, iprint, iter) integer n, nfree, nenter, ileave, iprint, iter, + index(n), indx2(n), iwhere(n) logical wrk, updatd, cnstnd c ************ c c Subroutine freev c c This subroutine counts the entering and leaving variables when c iter > 0, and finds the index set of free and active variables c at the GCP. c c cnstnd is a logical variable indicating whether bounds are present c c index is an integer array of dimension n c for i=1,...,nfree, index(i) are the indices of free variables c for i=nfree+1,...,n, index(i) are the indices of bound variables c On entry after the first iteration, index gives c the free variables at the previous iteration. c On exit it gives the free variables based on the determination c in cauchy using the array iwhere. c c indx2 is an integer array of dimension n c On entry indx2 is unspecified. c On exit with iter>0, indx2 indicates which variables c have changed status since the previous iteration. c For i= 1,...,nenter, indx2(i) have changed from bound to free. c For i= ileave+1,...,n, indx2(i) have changed from free to bound. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer iact,i,k nenter = 0 ileave = n + 1 if (iter .gt. 0 .and. cnstnd) then c count the entering and leaving variables. do 20 i = 1, nfree k = index(i) c write(6,*) ' k = index(i) ', k c write(6,*) ' index = ', i if (iwhere(k) .gt. 0) then ileave = ileave - 1 indx2(ileave) = k if (iprint .ge. 100) write (6,*) + 'Variable ',k,' leaves the set of free variables' endif 20 continue do 22 i = 1 + nfree, n k = index(i) if (iwhere(k) .le. 0) then nenter = nenter + 1 indx2(nenter) = k if (iprint .ge. 100) write (6,*) + 'Variable ',k,' enters the set of free variables' endif 22 continue if (iprint .ge. 99) write (6,*) + n+1-ileave,' variables leave; ',nenter,' variables enter' endif wrk = (ileave .lt. n+1) .or. (nenter .gt. 0) .or. updatd c Find the index set of free and active variables at the GCP. nfree = 0 iact = n + 1 do 24 i = 1, n if (iwhere(i) .le. 0) then nfree = nfree + 1 index(nfree) = i else iact = iact - 1 index(iact) = i endif 24 continue if (iprint .ge. 99) write (6,*) + nfree,' variables are free at GCP ',iter + 1 return end c======================= The end of freev ============================== subroutine hpsolb(n, t, iorder, iheap) integer iheap, n, iorder(n) double precision t(n) c ************ c c Subroutine hpsolb c c This subroutine sorts out the least element of t, and puts the c remaining elements of t in a heap. c c n is an integer variable. c On entry n is the dimension of the arrays t and iorder. c On exit n is unchanged. c c t is a double precision array of dimension n. c On entry t stores the elements to be sorted, c On exit t(n) stores the least elements of t, and t(1) to t(n-1) c stores the remaining elements in the form of a heap. c c iorder is an integer array of dimension n. c On entry iorder(i) is the index of t(i). c On exit iorder(i) is still the index of t(i), but iorder may be c permuted in accordance with t. c c iheap is an integer variable specifying the task. c On entry iheap should be set as follows: c iheap .eq. 0 if t(1) to t(n) is not in the form of a heap, c iheap .ne. 0 if otherwise. c On exit iheap is unchanged. c c c References: c Algorithm 232 of CACM (J. W. J. Williams): HEAPSORT. c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c ************ integer i,j,k,indxin,indxou double precision ddum,out if (iheap .eq. 0) then c Rearrange the elements t(1) to t(n) to form a heap. do 20 k = 2, n ddum = t(k) indxin = iorder(k) c Add ddum to the heap. i = k 10 continue if (i.gt.1) then j = i/2 if (ddum .lt. t(j)) then t(i) = t(j) iorder(i) = iorder(j) i = j goto 10 endif endif t(i) = ddum iorder(i) = indxin 20 continue endif c Assign to 'out' the value of t(1), the least member of the heap, c and rearrange the remaining members to form a heap as c elements 1 to n-1 of t. if (n .gt. 1) then i = 1 out = t(1) indxou = iorder(1) ddum = t(n) indxin = iorder(n) c Restore the heap 30 continue j = i+i if (j .le. n-1) then if (t(j+1) .lt. t(j)) j = j+1 if (t(j) .lt. ddum ) then t(i) = t(j) iorder(i) = iorder(j) i = j goto 30 endif endif t(i) = ddum iorder(i) = indxin c Put the least member in t(n). t(n) = out iorder(n) = indxou endif return end c====================== The end of hpsolb ============================== subroutine lnsrlb(n, l, u, nbd, x, f, fold, gd, gdold, g, d, r, t, + z, stp, dnorm, dtd, xstep, stpmx, iter, ifun, + iback, nfgv, info, itask, boxed, cnstnd, icsave, + isave, dsave) logical boxed, cnstnd integer n, iter, ifun, iback, nfgv, info, itask, icsave, + nbd(n), isave(2) double precision f, fold, gd, gdold, stp, dnorm, dtd, xstep, + stpmx, x(n), l(n), u(n), g(n), d(n), r(n), t(n), + z(n), dsave(13) c ********** c c Subroutine lnsrlb c c This subroutine calls subroutine dcsrch from the Minpack2 library c to perform the line search. Subroutine dscrch is safeguarded so c that all trial points lie within the feasible region. c c Subprograms called: c c Minpack2 Library ... dcsrch. c c Linpack ... dtrsl, ddot. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ********** integer i double precision ddot,a1,a2 double precision one,zero,big parameter (one=1.0d0,zero=0.0d0,big=1.0d+10) double precision ftol,gtol,xtol parameter (ftol=1.0d-3,gtol=0.9d0,xtol=0.1d0) c if (task(1:5) .eq. 'FG_LN') goto 556 if (itask .eq. 20) goto 556 dtd = ddot(n,d,1,d,1) dnorm = sqrt(dtd) c Determine the maximum step length. stpmx = big if (cnstnd) then if (iter .eq. 0) then stpmx = one else do 43 i = 1, n a1 = d(i) if (nbd(i) .ne. 0) then if (a1 .lt. zero .and. nbd(i) .le. 2) then a2 = l(i) - x(i) if (a2 .ge. zero) then stpmx = zero else if (a1*stpmx .lt. a2) then stpmx = a2/a1 endif else if (a1 .gt. zero .and. nbd(i) .ge. 2) then a2 = u(i) - x(i) if (a2 .le. zero) then stpmx = zero else if (a1*stpmx .gt. a2) then stpmx = a2/a1 endif endif endif 43 continue endif endif if (iter .eq. 0 .and. .not. boxed) then stp = min(one/dnorm, stpmx) else stp = one endif call dcopy(n,x,1,t,1) call dcopy(n,g,1,r,1) fold = f ifun = 0 iback = 0 c csave = 'START' icsave = 2 556 continue gd = ddot(n,g,1,d,1) if (ifun .eq. 0) then gdold=gd if (gd .ge. zero) then c the directional derivative >=0. c Line search is impossible. write(6,*)' ascent direction in projection gd = ', gd info = -4 return endif endif call dcsrch(f,gd,stp,ftol,gtol,xtol,zero,stpmx,icsave,isave,dsave) xstep = stp*dnorm c if (csave(1:4) .ne. 'CONV' .and. csave(1:4) .ne. 'WARN') then c2345678901234567890123456789012345678901234567890123456789012345678901234567890 if ((icsave .lt. 6) .or. ((icsave .gt. 8) .and. + (icsave .lt. 23))) then c task = 'FG_LNSRCH' itask = 20 ifun = ifun + 1 nfgv = nfgv + 1 iback = ifun - 1 if (stp .eq. one) then call dcopy(n,z,1,x,1) else do 41 i = 1, n x(i) = stp*d(i) + t(i) 41 continue endif else c task = 'NEW_X' itask = 1 endif return end c======================= The end of lnsrlb ============================= subroutine matupd(n, m, ws, wy, sy, ss, d, r, itail, + iupdat, col, head, theta, rr, dr, stp, dtd) integer n, m, itail, iupdat, col, head double precision theta, rr, dr, stp, dtd, d(n), r(n), + ws(n, m), wy(n, m), sy(m, m), ss(m, m) c ************ c c Subroutine matupd c c This subroutine updates matrices WS and WY, and forms the c middle matrix in B. c c Subprograms called: c c Linpack ... dcopy, ddot. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer j,pointr double precision ddot double precision one parameter (one=1.0d0) c Set pointers for matrices WS and WY. if (iupdat .le. m) then col = iupdat itail = mod(head+iupdat-2,m) + 1 else itail = mod(itail,m) + 1 head = mod(head,m) + 1 endif c Update matrices WS and WY. call dcopy(n,d,1,ws(1,itail),1) call dcopy(n,r,1,wy(1,itail),1) c Set theta=yy/ys. theta = rr/dr c Form the middle matrix in B. c update the upper triangle of SS, c and the lower triangle of SY: if (iupdat .gt. m) then c move old information do 50 j = 1, col - 1 call dcopy(j,ss(2,j+1),1,ss(1,j),1) call dcopy(col-j,sy(j+1,j+1),1,sy(j,j),1) 50 continue endif c add new information: the last row of SY c and the last column of SS: pointr = head do 51 j = 1, col - 1 sy(col,j) = ddot(n,d,1,wy(1,pointr),1) ss(j,col) = ddot(n,ws(1,pointr),1,d,1) pointr = mod(pointr,m) + 1 51 continue if (stp .eq. one) then ss(col,col) = dtd else ss(col,col) = stp*stp*dtd endif sy(col,col) = dr return end c======================= The end of matupd ============================= subroutine prn1lb(n, m, l, u, x, iprint, itfile, epsmch) integer n, m, iprint, itfile double precision epsmch, x(n), l(n), u(n) c ************ c c Subroutine prn1lb c c This subroutine prints the input data, initial point, upper and c lower bounds of each variable, machine precision, as well as c the headings of the output. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i if (iprint .ge. 0) then write (6,7001) epsmch write (6,*) 'N = ',n,' M = ',m if (iprint .ge. 1) then write (itfile,2001) epsmch write (itfile,*)'N = ',n,' M = ',m write (itfile,9001) if (iprint .gt. 100) then write (6,1004) 'L =',(l(i),i = 1,n) write (6,1004) 'X0 =',(x(i),i = 1,n) write (6,1004) 'U =',(u(i),i = 1,n) endif endif endif 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) 2001 format ('RUNNING THE L-BFGS-B CODE',/,/, + 'it = iteration number',/, + 'nf = number of function evaluations',/, + 'nseg = number of segments explored during the Cauchy search',/, + 'nact = number of active bounds at the generalized Cauchy point' + ,/, + 'sub = manner in which the subspace minimization terminated:' + ,/,' con = converged, bnd = a bound was reached',/, + 'itls = number of iterations performed in the line search',/, + 'stepl = step length used',/, + 'tstep = norm of the displacement (total step)',/, + 'projg = norm of the projected gradient',/, + 'f = function value',/,/, + ' * * *',/,/, + 'Machine precision =',1p,d10.3) 7001 format ('RUNNING THE L-BFGS-B CODE',/,/, + ' * * *',/,/, + 'Machine precision =',1p,d10.3) 9001 format (/,3x,'it',3x,'nf',2x,'nseg',2x,'nact',2x,'sub',2x,'itls', + 2x,'stepl',4x,'tstep',5x,'projg',8x,'f') return end c======================= The end of prn1lb ============================= subroutine prn2lb(n, x, f, g, iprint, itfile, iter, nfgv, nact, + sbgnrm, nseg, word, iword, iback, stp, xstep) character*3 word integer n, iprint, itfile, iter, nfgv, nact, nseg, + iword, iback double precision f, sbgnrm, stp, xstep, x(n), g(n) c ************ c c Subroutine prn2lb c c This subroutine prints out new information after a successful c line search. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,imod c 'word' records the status of subspace solutions. if (iword .eq. 0) then c the subspace minimization converged. word = 'con' else if (iword .eq. 1) then c the subspace minimization stopped at a bound. word = 'bnd' else if (iword .eq. 5) then c the truncated Newton step has been used. word = 'TNT' else word = '---' endif if (iprint .ge. 99) then write (6,*) 'LINE SEARCH',iback,' times; norm of step = ',xstep write (6,2001) iter,f,sbgnrm if (iprint .gt. 100) then write (6,1004) 'X =',(x(i), i = 1, n) write (6,1004) 'G =',(g(i), i = 1, n) endif else if (iprint .gt. 0) then imod = mod(iter,iprint) if (imod .eq. 0) write (6,2001) iter,f,sbgnrm endif if (iprint .ge. 1) write (itfile,3001) + iter,nfgv,nseg,nact,word,iback,stp,xstep,sbgnrm,f 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) 2001 format + (/,'At iterate',i5,4x,'f= ',1p,d12.5,4x,'|proj g|= ',1p,d12.5) 3001 format(2(1x,i4),2(1x,i5),2x,a3,1x,i4,1p,2(2x,d7.1),1p,2(1x,d10.3)) return end c======================= The end of prn2lb ============================= subroutine prn3lb(n, x, f, itask, iprint, info, itfile, + iter, nfgv, nintol, nskip, nact, sbgnrm, + time, nseg, word, iback, stp, xstep, k, + cachyt, sbtime, lnscht) c character*255 task character*3 word integer n, iprint, info, itfile, iter, nfgv, nintol, + nskip, nact, nseg, iback, k, itask double precision f, sbgnrm, time, stp, xstep, cachyt, sbtime, + lnscht, x(n) c ************ c c Subroutine prn3lb c c This subroutine prints out information when either a built-in c convergence test is satisfied or when an error message is c generated. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i c if (task(1:5) .eq. 'ERROR') goto 999 if ((itask .ge. 9) .and. (itask .le. 19)) goto 999 if (iprint .ge. 0) then write (6,3003) write (6,3004) write(6,3005) n,iter,nfgv,nintol,nskip,nact,sbgnrm,f if (iprint .ge. 100) then write (6,1004) 'X =',(x(i),i = 1,n) endif if (iprint .ge. 1) write (6,*) ' F =',f endif 999 continue if (iprint .ge. 0) then write (6,3009) itask if (info .ne. 0) then if (info .eq. -1) write (6,9011) if (info .eq. -2) write (6,9012) if (info .eq. -3) write (6,9013) if (info .eq. -4) write (6,9014) if (info .eq. -5) write (6,9015) if (info .eq. -6) write (6,*)' Input nbd(',k,') is invalid.' if (info .eq. -7) + write (6,*)' l(',k,') > u(',k,'). No feasible solution.' if (info .eq. -8) write (6,9018) if (info .eq. -9) write (6,9019) endif if (iprint .ge. 1) write (6,3007) cachyt,sbtime,lnscht write (6,3008) time if (iprint .ge. 1) then if (info .eq. -4 .or. info .eq. -9) then write (itfile,3002) + iter,nfgv,nseg,nact,word,iback,stp,xstep endif write (itfile,3009) itask if (info .ne. 0) then if (info .eq. -1) write (itfile,9011) if (info .eq. -2) write (itfile,9012) if (info .eq. -3) write (itfile,9013) if (info .eq. -4) write (itfile,9014) if (info .eq. -5) write (itfile,9015) if (info .eq. -8) write (itfile,9018) if (info .eq. -9) write (itfile,9019) endif write (itfile,3008) time endif endif 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) 3002 format(2(1x,i4),2(1x,i5),2x,a3,1x,i4,1p,2(2x,d7.1),6x,'-',10x,'-') 3003 format (/, + ' * * *',/,/, + 'Tit = total number of iterations',/, + 'Tnf = total number of function evaluations',/, + 'Tnint = total number of segments explored during', + ' Cauchy searches',/, + 'Skip = number of BFGS updates skipped',/, + 'Nact = number of active bounds at final generalized', + ' Cauchy point',/, + 'Projg = norm of the final projected gradient',/, + 'F = final function value',/,/, + ' * * *') 3004 format (/,3x,'N',4x,'Tit',5x,'Tnf',2x,'Tnint',2x, + 'Skip',2x,'Nact',5x,'Projg',8x,'F') 3005 format (i5,2(1x,i6),(1x,i6),(2x,i4),(1x,i5),1p,2(2x,d10.3)) 3007 format (/,' Cauchy time',1p,e10.3,' seconds.',/ + ' Subspace minimization time',1p,e10.3,' seconds.',/ + ' Line search time',1p,e10.3,' seconds.') 3008 format (/,' Total User time',1p,e10.3,' seconds.',/) 3009 format (/,i4) 9011 format (/, +' Matrix in 1st Cholesky factorization in formk is not Pos. Def.') 9012 format (/, +' Matrix in 2st Cholesky factorization in formk is not Pos. Def.') 9013 format (/, +' Matrix in the Cholesky factorization in formt is not Pos. Def.') 9014 format (/, +' Derivative >= 0, backtracking line search impossible.',/, +' Previous x, f and g restored.',/, +' Possible causes: 1 error in function or gradient evaluation;',/, +' 2 rounding errors dominate computation.') 9015 format (/, +' Warning: more than 10 function and gradient',/, +' evaluations in the last line search. Termination',/, +' may possibly be caused by a bad search direction.') 9018 format (/,' The triangular system is singular.') 9019 format (/, +' Line search cannot locate an adequate point after 20 function',/ +,' and gradient evaluations. Previous x, f and g restored.',/, +' Possible causes: 1 error in function or gradient evaluation;',/, +' 2 rounding error dominate computation.') return end c======================= The end of prn3lb ============================= subroutine projgr(n, l, u, nbd, x, g, sbgnrm) integer n, nbd(n) double precision sbgnrm, x(n), l(n), u(n), g(n) c ************ c c Subroutine projgr c c This subroutine computes the infinity norm of the projected c gradient. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i double precision gi double precision one,zero parameter (one=1.0d0,zero=0.0d0) sbgnrm = zero do 15 i = 1, n gi = g(i) if (nbd(i) .ne. 0) then if (gi .lt. zero) then if (nbd(i) .ge. 2) gi = max((x(i)-u(i)),gi) else if (nbd(i) .le. 2) gi = min((x(i)-l(i)),gi) endif endif sbgnrm = max(sbgnrm,abs(gi)) 15 continue return end c======================= The end of projgr ============================= subroutine subsm ( n, m, nsub, ind, l, u, nbd, x, d, xp, ws, wy, + theta, xx, gg, + col, head, iword, wv, wn, iprint, info ) implicit none integer n, m, nsub, col, head, iword, iprint, info, + ind(nsub), nbd(n) double precision theta, + l(n), u(n), x(n), d(n), xp(n), xx(n), gg(n), + ws(n, m), wy(n, m), + wv(2*m), wn(2*m, 2*m) c ********************************************************************** c c This routine contains the major changes in the updated version. c The changes are described in the accompanying paper c c Jose Luis Morales, Jorge Nocedal c "Remark On Algorithm 788: L-BFGS-B: Fortran Subroutines for Large-Scale c Bound Constrained Optimization". Decemmber 27, 2010. c c J.L. Morales Departamento de Matematicas, c Instituto Tecnologico Autonomo de Mexico c Mexico D.F. c c J, Nocedal Department of Electrical Engineering and c Computer Science. c Northwestern University. Evanston, IL. USA c c January 17, 2011 c c ********************************************************************** c c c Subroutine subsm c c Given xcp, l, u, r, an index set that specifies c the active set at xcp, and an l-BFGS matrix B c (in terms of WY, WS, SY, WT, head, col, and theta), c this subroutine computes an approximate solution c of the subspace problem c c (P) min Q(x) = r'(x-xcp) + 1/2 (x-xcp)' B (x-xcp) c c subject to l<=x<=u c x_i=xcp_i for all i in A(xcp) c c along the subspace unconstrained Newton direction c c d = -(Z'BZ)^(-1) r. c c The formula for the Newton direction, given the L-BFGS matrix c and the Sherman-Morrison formula, is c c d = (1/theta)r + (1/theta*2) Z'WK^(-1)W'Z r. c c where c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c Note that this procedure for computing d differs c from that described in [1]. One can show that the matrix K is c equal to the matrix M^[-1]N in that paper. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c nsub is an integer variable. c On entry nsub is the number of free variables. c On exit nsub is unchanged. c c ind is an integer array of dimension nsub. c On entry ind specifies the coordinate indices of free variables. c On exit ind is unchanged. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is a integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c x is a double precision array of dimension n. c On entry x specifies the Cauchy point xcp. c On exit x(i) is the minimizer of Q over the subspace of c free variables. c c d is a double precision array of dimension n. c On entry d is the reduced gradient of Q at xcp. c On exit d is the Newton direction of Q. c c xp is a double precision array of dimension n. c used to safeguard the projected Newton direction c c xx is a double precision array of dimension n c On entry it holds the current iterate c On output it is unchanged c gg is a double precision array of dimension n c On entry it holds the gradient at the current iterate c On output it is unchanged c c ws and wy are double precision arrays; c theta is a double precision variable; c col is an integer variable; c head is an integer variable. c On entry they store the information defining the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c theta is the scaling factor specifying B_0 = theta I; c col is the number of variable metric corrections stored; c head is the location of the 1st s- (or y-) vector in S (or Y). c On exit they are unchanged. c c iword is an integer variable. c On entry iword is unspecified. c On exit iword specifies the status of the subspace solution. c iword = 0 if the solution is in the box, c 1 if some bound is encountered. c c wv is a double precision working array of dimension 2m. c c wn is a double precision array of dimension 2m x 2m. c On entry the upper triangle of wn stores the LEL^T factorization c of the indefinite matrix c c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] c On exit wn is unchanged. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return, c = nonzero for abnormal return c when the matrix K is ill-conditioned. c c Subprograms called: c c Linpack dtrsl. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer pointr,m2,col2,ibd,jy,js,i,j,k double precision alpha, xk, dk, temp1, temp2 double precision one,zero parameter (one=1.0d0,zero=0.0d0) c double precision dd_p if (nsub .le. 0) return if (iprint .ge. 99) write (6,1001) c Compute wv = W'Zd. pointr = head do 20 i = 1, col temp1 = zero temp2 = zero do 10 j = 1, nsub k = ind(j) temp1 = temp1 + wy(k,pointr)*d(j) temp2 = temp2 + ws(k,pointr)*d(j) 10 continue wv(i) = temp1 wv(col + i) = theta*temp2 pointr = mod(pointr,m) + 1 20 continue c Compute wv:=K^(-1)wv. m2 = 2*m col2 = 2*col call dtrsl(wn,m2,col2,wv,11,info) if (info .ne. 0) return do 25 i = 1, col wv(i) = -wv(i) 25 continue call dtrsl(wn,m2,col2,wv,01,info) if (info .ne. 0) return c Compute d = (1/theta)d + (1/theta**2)Z'W wv. pointr = head do 40 jy = 1, col js = col + jy do 30 i = 1, nsub k = ind(i) d(i) = d(i) + wy(k,pointr)*wv(jy)/theta + + ws(k,pointr)*wv(js) 30 continue pointr = mod(pointr,m) + 1 40 continue call dscal( nsub, one/theta, d, 1 ) c c----------------------------------------------------------------- c Let us try the projection, d is the Newton direction iword = 0 call dcopy ( n, x, 1, xp, 1 ) c do 50 i=1, nsub k = ind(i) dk = d(i) xk = x(k) if ( nbd(k) .ne. 0 ) then c if ( nbd(k).eq.1 ) then ! lower bounds only x(k) = max( l(k), xk + dk ) if ( x(k).eq.l(k) ) iword = 1 else c if ( nbd(k).eq.2 ) then ! upper and lower bounds xk = max( l(k), xk + dk ) x(k) = min( u(k), xk ) if ( x(k).eq.l(k) .or. x(k).eq.u(k) ) iword = 1 else c if ( nbd(k).eq.3 ) then ! upper bounds only x(k) = min( u(k), xk + dk ) if ( x(k).eq.u(k) ) iword = 1 end if end if end if c else ! free variables x(k) = xk + dk end if 50 continue c if ( iword.eq.0 ) then go to 911 end if c c check sign of the directional derivative c dd_p = zero do 55 i=1, n dd_p = dd_p + (x(i) - xx(i))*gg(i) 55 continue if ( dd_p .gt.zero ) then call dcopy( n, xp, 1, x, 1 ) write(6,*) ' Positive dir derivative in projection ' write(6,*) ' Using the backtracking step ' else go to 911 endif c c----------------------------------------------------------------- c alpha = one temp1 = alpha ibd = 0 do 60 i = 1, nsub k = ind(i) dk = d(i) if (nbd(k) .ne. 0) then if (dk .lt. zero .and. nbd(k) .le. 2) then temp2 = l(k) - x(k) if (temp2 .ge. zero) then temp1 = zero else if (dk*alpha .lt. temp2) then temp1 = temp2/dk endif else if (dk .gt. zero .and. nbd(k) .ge. 2) then temp2 = u(k) - x(k) if (temp2 .le. zero) then temp1 = zero else if (dk*alpha .gt. temp2) then temp1 = temp2/dk endif endif if (temp1 .lt. alpha) then alpha = temp1 ibd = i endif endif 60 continue if (alpha .lt. one) then dk = d(ibd) k = ind(ibd) if (dk .gt. zero) then x(k) = u(k) d(ibd) = zero else if (dk .lt. zero) then x(k) = l(k) d(ibd) = zero endif endif do 70 i = 1, nsub k = ind(i) x(k) = x(k) + alpha*d(i) 70 continue cccccc 911 continue if (iprint .ge. 99) write (6,1004) 1001 format (/,'----------------SUBSM entered-----------------',/) 1004 format (/,'----------------exit SUBSM --------------------',/) return end c====================== The end of subsm =============================== subroutine dcsrch(f,g,stp,ftol,gtol,xtol,stpmin,stpmax, + itask,isave,dsave) c character*(*) task integer itask, isave(2) double precision f,g,stp,ftol,gtol,xtol,stpmin,stpmax double precision dsave(13) c ********** c c Subroutine dcsrch c c This subroutine finds a step that satisfies a sufficient c decrease condition and a curvature condition. c c Each call of the subroutine updates an interval with c endpoints stx and sty. The interval is initially chosen c so that it contains a minimizer of the modified function c c psi(stp) = f(stp) - f(0) - ftol*stp*f'(0). c c If psi(stp) <= 0 and f'(stp) >= 0 for some step, then the c interval is chosen so that it contains a minimizer of f. c c The algorithm is designed to find a step that satisfies c the sufficient decrease condition c c f(stp) <= f(0) + ftol*stp*f'(0), c c and the curvature condition c c abs(f'(stp)) <= gtol*abs(f'(0)). c c If ftol is less than gtol and if, for example, the function c is bounded below, then there is always a step which satisfies c both conditions. c c If no step can be found that satisfies both conditions, then c the algorithm stops with a warning. In this case stp only c satisfies the sufficient decrease condition. c c A typical invocation of dcsrch has the following outline: c c task = 'START' c 10 continue c call dcsrch( ... ) c if (task .eq. 'FG') then c Evaluate the function and the gradient at stp c goto 10 c end if c c NOTE: The user must no alter work arrays between calls. c c The subroutine statement is c c subroutine dcsrch(f,g,stp,ftol,gtol,xtol,stpmin,stpmax, c task,isave,dsave) c where c c f is a double precision variable. c On initial entry f is the value of the function at 0. c On subsequent entries f is the value of the c function at stp. c On exit f is the value of the function at stp. c c g is a double precision variable. c On initial entry g is the derivative of the function at 0. c On subsequent entries g is the derivative of the c function at stp. c On exit g is the derivative of the function at stp. c c stp is a double precision variable. c On entry stp is the current estimate of a satisfactory c step. On initial entry, a positive initial estimate c must be provided. c On exit stp is the current estimate of a satisfactory step c if task = 'FG'. If task = 'CONV' then stp satisfies c the sufficient decrease and curvature condition. c c ftol is a double precision variable. c On entry ftol specifies a nonnegative tolerance for the c sufficient decrease condition. c On exit ftol is unchanged. c c gtol is a double precision variable. c On entry gtol specifies a nonnegative tolerance for the c curvature condition. c On exit gtol is unchanged. c c xtol is a double precision variable. c On entry xtol specifies a nonnegative relative tolerance c for an acceptable step. The subroutine exits with a c warning if the relative difference between sty and stx c is less than xtol. c On exit xtol is unchanged. c c stpmin is a double precision variable. c On entry stpmin is a nonnegative lower bound for the step. c On exit stpmin is unchanged. c c stpmax is a double precision variable. c On entry stpmax is a nonnegative upper bound for the step. c On exit stpmax is unchanged. c c task is a character variable of length at least 60. c On initial entry task must be set to 'START'. c On exit task indicates the required action: c c If task(1:2) = 'FG' then evaluate the function and c derivative at stp and call dcsrch again. c c If task(1:4) = 'CONV' then the search is successful. c c If task(1:4) = 'WARN' then the subroutine is not able c to satisfy the convergence conditions. The exit value of c stp contains the best point found during the search. c c If task(1:5) = 'ERROR' then there is an error in the c input arguments. c c On exit with convergence, a warning or an error, the c variable task contains additional information. c c isave is an integer work array of dimension 2. c c dsave is a double precision work array of dimension 13. c c Subprograms called c c MINPACK-2 ... dcstep c c MINPACK-1 Project. June 1983. c Argonne National Laboratory. c Jorge J. More' and David J. Thuente. c c MINPACK-2 Project. October 1993. c Argonne National Laboratory and University of Minnesota. c Brett M. Averick, Richard G. Carter, and Jorge J. More'. c c ********** double precision zero,p5,p66 parameter(zero=0.0d0,p5=0.5d0,p66=0.66d0) double precision xtrapl,xtrapu parameter(xtrapl=1.1d0,xtrapu=4.0d0) logical brackt integer stage double precision finit,ftest,fm,fx,fxm,fy,fym,ginit,gtest, + gm,gx,gxm,gy,gym,stx,sty,stmin,stmax,width,width1 c Initialization block. c if (task(1:5) .eq. 'START') then if (itask .eq. 2) then c Check the input arguments for errors. c if (stp .lt. stpmin) task = 'ERROR: STP .LT. STPMIN' c if (stp .gt. stpmax) task = 'ERROR: STP .GT. STPMAX' c if (g .ge. zero) task = 'ERROR: INITIAL G .GE. ZERO' c if (ftol .lt. zero) task = 'ERROR: FTOL .LT. ZERO' c if (gtol .lt. zero) task = 'ERROR: GTOL .LT. ZERO' c if (xtol .lt. zero) task = 'ERROR: XTOL .LT. ZERO' c if (stpmin .lt. zero) task = 'ERROR: STPMIN .LT. ZERO' c if (stpmax .lt. stpmin) task = 'ERROR: STPMAX .LT. STPMIN' if (stp .lt. stpmin) itask = 16 if (stp .gt. stpmax) itask = 15 if (g .ge. zero) itask = 11 if (ftol .lt. zero) itask = 9 if (gtol .lt. zero) itask = 10 if (xtol .lt. zero) itask = 19 if (stpmin .lt. zero) itask = 18 if (stpmax .lt. stpmin) itask = 17 c Exit if there are errors on input. c if (task(1:5) .eq. 'ERROR') return if ((itask .ge. 9) .and. (itask .le. 19)) return c Initialize local variables. brackt = .false. stage = 1 finit = f ginit = g gtest = ftol*ginit width = stpmax - stpmin width1 = width/p5 c The variables stx, fx, gx contain the values of the step, c function, and derivative at the best step. c The variables sty, fy, gy contain the value of the step, c function, and derivative at sty. c The variables stp, f, g contain the values of the step, c function, and derivative at stp. stx = zero fx = finit gx = ginit sty = zero fy = finit gy = ginit stmin = zero stmax = stp + xtrapu*stp c task = 'FG' itask = 4 goto 1000 else c Restore local variables. if (isave(1) .eq. 1) then brackt = .true. else brackt = .false. endif stage = isave(2) ginit = dsave(1) gtest = dsave(2) gx = dsave(3) gy = dsave(4) finit = dsave(5) fx = dsave(6) fy = dsave(7) stx = dsave(8) sty = dsave(9) stmin = dsave(10) stmax = dsave(11) width = dsave(12) width1 = dsave(13) endif c If psi(stp) <= 0 and f'(stp) >= 0 for some step, then the c algorithm enters the second stage. ftest = finit + stp*gtest if (stage .eq. 1 .and. f .le. ftest .and. g .ge. zero) + stage = 2 c Test for warnings. if (brackt .and. (stp .le. stmin .or. stp .ge. stmax)) c + task = 'WARNING: ROUNDING ERRORS PREVENT PROGRESS' + itask = 23 if (brackt .and. stmax - stmin .le. xtol*stmax) c + task = 'WARNING: XTOL TEST SATISFIED' + itask = 26 if (stp .eq. stpmax .and. f .le. ftest .and. g .le. gtest) c + task = 'WARNING: STP = STPMAX' + itask = 24 if (stp .eq. stpmin .and. (f .gt. ftest .or. g .ge. gtest)) c + task = 'WARNING: STP = STPMIN' + itask = 25 c Test for convergence. if (f .le. ftest .and. abs(g) .le. gtol*(-ginit)) c + task = 'CONVERGENCE' + itask = 6 c Test for termination. c if (task(1:4) .eq. 'WARN' .or. task(1:4) .eq. 'CONV') goto 1000 if ((itask .ge. 23) .or. ((itask .le. 8) + .and. (itask .ge. 6))) goto 1000 c A modified function is used to predict the step during the c first stage if a lower function value has been obtained but c the decrease is not sufficient. if (stage .eq. 1 .and. f .le. fx .and. f .gt. ftest) then c Define the modified function and derivative values. fm = f - stp*gtest fxm = fx - stx*gtest fym = fy - sty*gtest gm = g - gtest gxm = gx - gtest gym = gy - gtest c Call dcstep to update stx, sty, and to compute the new step. call dcstep(stx,fxm,gxm,sty,fym,gym,stp,fm,gm, + brackt,stmin,stmax) c Reset the function and derivative values for f. fx = fxm + stx*gtest fy = fym + sty*gtest gx = gxm + gtest gy = gym + gtest else c Call dcstep to update stx, sty, and to compute the new step. call dcstep(stx,fx,gx,sty,fy,gy,stp,f,g, + brackt,stmin,stmax) endif c Decide if a bisection step is needed. if (brackt) then if (abs(sty-stx) .ge. p66*width1) stp = stx + p5*(sty - stx) width1 = width width = abs(sty-stx) endif c Set the minimum and maximum steps allowed for stp. if (brackt) then stmin = min(stx,sty) stmax = max(stx,sty) else stmin = stp + xtrapl*(stp - stx) stmax = stp + xtrapu*(stp - stx) endif c Force the step to be within the bounds stpmax and stpmin. stp = max(stp,stpmin) stp = min(stp,stpmax) c If further progress is not possible, let stp be the best c point obtained during the search. if (brackt .and. (stp .le. stmin .or. stp .ge. stmax) + .or. (brackt .and. stmax-stmin .le. xtol*stmax)) stp = stx c Obtain another function and derivative. c task = 'FG' itask = 4 1000 continue c Save local variables. if (brackt) then isave(1) = 1 else isave(1) = 0 endif isave(2) = stage dsave(1) = ginit dsave(2) = gtest dsave(3) = gx dsave(4) = gy dsave(5) = finit dsave(6) = fx dsave(7) = fy dsave(8) = stx dsave(9) = sty dsave(10) = stmin dsave(11) = stmax dsave(12) = width dsave(13) = width1 return end c====================== The end of dcsrch ============================== subroutine dcstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt, + stpmin,stpmax) logical brackt double precision stx,fx,dx,sty,fy,dy,stp,fp,dp,stpmin,stpmax c ********** c c Subroutine dcstep c c This subroutine computes a safeguarded step for a search c procedure and updates an interval that contains a step that c satisfies a sufficient decrease and a curvature condition. c c The parameter stx contains the step with the least function c value. If brackt is set to .true. then a minimizer has c been bracketed in an interval with endpoints stx and sty. c The parameter stp contains the current step. c The subroutine assumes that if brackt is set to .true. then c c min(stx,sty) < stp < max(stx,sty), c c and that the derivative at stx is negative in the direction c of the step. c c The subroutine statement is c c subroutine dcstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt, c stpmin,stpmax) c c where c c stx is a double precision variable. c On entry stx is the best step obtained so far and is an c endpoint of the interval that contains the minimizer. c On exit stx is the updated best step. c c fx is a double precision variable. c On entry fx is the function at stx. c On exit fx is the function at stx. c c dx is a double precision variable. c On entry dx is the derivative of the function at c stx. The derivative must be negative in the direction of c the step, that is, dx and stp - stx must have opposite c signs. c On exit dx is the derivative of the function at stx. c c sty is a double precision variable. c On entry sty is the second endpoint of the interval that c contains the minimizer. c On exit sty is the updated endpoint of the interval that c contains the minimizer. c c fy is a double precision variable. c On entry fy is the function at sty. c On exit fy is the function at sty. c c dy is a double precision variable. c On entry dy is the derivative of the function at sty. c On exit dy is the derivative of the function at the exit sty. c c stp is a double precision variable. c On entry stp is the current step. If brackt is set to .true. c then on input stp must be between stx and sty. c On exit stp is a new trial step. c c fp is a double precision variable. c On entry fp is the function at stp c On exit fp is unchanged. c c dp is a double precision variable. c On entry dp is the the derivative of the function at stp. c On exit dp is unchanged. c c brackt is an logical variable. c On entry brackt specifies if a minimizer has been bracketed. c Initially brackt must be set to .false. c On exit brackt specifies if a minimizer has been bracketed. c When a minimizer is bracketed brackt is set to .true. c c stpmin is a double precision variable. c On entry stpmin is a lower bound for the step. c On exit stpmin is unchanged. c c stpmax is a double precision variable. c On entry stpmax is an upper bound for the step. c On exit stpmax is unchanged. c c MINPACK-1 Project. June 1983 c Argonne National Laboratory. c Jorge J. More' and David J. Thuente. c c MINPACK-2 Project. October 1993. c Argonne National Laboratory and University of Minnesota. c Brett M. Averick and Jorge J. More'. c c ********** double precision zero,p66,two,three parameter(zero=0.0d0,p66=0.66d0,two=2.0d0,three=3.0d0) double precision gamma,p,q,r,s,sgnd,stpc,stpf,stpq,theta sgnd = dp*(dx/abs(dx)) c First case: A higher function value. The minimum is bracketed. c If the cubic step is closer to stx than the quadratic step, the c cubic step is taken, otherwise the average of the cubic and c quadratic steps is taken. if (fp .gt. fx) then theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dx/s)*(dp/s)) if (stp .lt. stx) gamma = -gamma p = (gamma - dx) + theta q = ((gamma - dx) + gamma) + dp r = p/q stpc = stx + r*(stp - stx) stpq = stx + ((dx/((fx - fp)/(stp - stx) + dx))/two)* + (stp - stx) if (abs(stpc-stx) .lt. abs(stpq-stx)) then stpf = stpc else stpf = stpc + (stpq - stpc)/two endif brackt = .true. c Second case: A lower function value and derivatives of opposite c sign. The minimum is bracketed. If the cubic step is farther from c stp than the secant step, the cubic step is taken, otherwise the c secant step is taken. else if (sgnd .lt. zero) then theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dx/s)*(dp/s)) if (stp .gt. stx) gamma = -gamma p = (gamma - dp) + theta q = ((gamma - dp) + gamma) + dx r = p/q stpc = stp + r*(stx - stp) stpq = stp + (dp/(dp - dx))*(stx - stp) if (abs(stpc-stp) .gt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif brackt = .true. c Third case: A lower function value, derivatives of the same sign, c and the magnitude of the derivative decreases. else if (abs(dp) .lt. abs(dx)) then c The cubic step is computed only if the cubic tends to infinity c in the direction of the step or if the minimum of the cubic c is beyond stp. Otherwise the cubic step is defined to be the c secant step. theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) c The case gamma = 0 only arises if the cubic does not tend c to infinity in the direction of the step. gamma = s*sqrt(max(zero,(theta/s)**2-(dx/s)*(dp/s))) if (stp .gt. stx) gamma = -gamma p = (gamma - dp) + theta q = (gamma + (dx - dp)) + gamma r = p/q if (r .lt. zero .and. gamma .ne. zero) then stpc = stp + r*(stx - stp) else if (stp .gt. stx) then stpc = stpmax else stpc = stpmin endif stpq = stp + (dp/(dp - dx))*(stx - stp) if (brackt) then c A minimizer has been bracketed. If the cubic step is c closer to stp than the secant step, the cubic step is c taken, otherwise the secant step is taken. if (abs(stpc-stp) .lt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif if (stp .gt. stx) then stpf = min(stp+p66*(sty-stp),stpf) else stpf = max(stp+p66*(sty-stp),stpf) endif else c A minimizer has not been bracketed. If the cubic step is c farther from stp than the secant step, the cubic step is c taken, otherwise the secant step is taken. if (abs(stpc-stp) .gt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif stpf = min(stpmax,stpf) stpf = max(stpmin,stpf) endif c Fourth case: A lower function value, derivatives of the same sign, c and the magnitude of the derivative does not decrease. If the c minimum is not bracketed, the step is either stpmin or stpmax, c otherwise the cubic step is taken. else if (brackt) then theta = three*(fp - fy)/(sty - stp) + dy + dp s = max(abs(theta),abs(dy),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dy/s)*(dp/s)) if (stp .gt. sty) gamma = -gamma p = (gamma - dp) + theta q = ((gamma - dp) + gamma) + dy r = p/q stpc = stp + r*(sty - stp) stpf = stpc else if (stp .gt. stx) then stpf = stpmax else stpf = stpmin endif endif c Update the interval which contains a minimizer. if (fp .gt. fx) then sty = stp fy = fp dy = dp else if (sgnd .lt. zero) then sty = stx fy = fx dy = dx endif stx = stp fx = fp dx = dp endif c Compute the new step. stp = stpf return end